A conveyor belt tear detection method of improved regional convolutional neural network

Abstract

The application discloses a method for improving a conveyor belt tear detection of a region convolutional neural network, a feature extraction module is set as a lightweight convolutional network MobileNetV2 to effectively reduce model parameters, a large-scale convolution kernel is adopted to expand a network receptive field and enhance the model feature extraction capability, a feature pyramid structure is introduced to fuse feature information at different levels, and the model is optimized to improve the recognition effect on slight damage, and the feature weight is adjusted in combination with an attention mechanism to highlight the feature information of the damage target. Experimental results show that the average detection accuracy of the algorithm for the conveyor belt damage target is improved by 2.78%, the detection speed is improved by 22.9%, and the algorithm has good real-time detection effect.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image detection, and particularly relates to a conveyor belt tearing detection method based on improved regional convolutional neural network. BACKGROUND

[0002] In the mining of coal mines, the conveyor belt plays an important role in the long-distance transportation of coal, but the conveyor belt itself is relatively fragile, and tearing often occurs during transportation. At present, the detection methods for conveyor belt tearing mainly include contact detection and non-contact detection. The contact detection generally indirectly judges the running state of the conveyor belt by monitoring the pressure on the surface of the conveyor belt in real time. However, this method only issues an interruption signal when the conveyor belt produces a large tearing deformation, and the detection sensitivity is low. The non-contact detection mainly uses machine vision methods, uses edge detection, image segmentation, etc. to obtain the texture features of the damaged area and classifies them. However, this method is prone to misjudgment when identifying images with similar background textures to the damaged target of the conveyor belt, resulting in false positives and false negatives.

[0003] In recent years, deep learning has rapidly developed in the field of target detection due to its good generalization ability and automatic learning of feature distribution. The target detection algorithm based on deep learning is mainly divided into regression-based method and region-based method. The former uses the output layer feature to directly predict the position and category of the target, which meets the real-time requirement of detection. For example, the YOLO(you only look once) algorithm uses a full convolution module to extract features and uses an equally divided grid to complete regression calculation, which realizes the classification and positioning of targets. However, each grid has good robustness for a single target, but the detection accuracy is low for overlapping multi-class targets. Based on this, the multi-scale detection idea is used to detect the features of different levels of convolution modules, which can consider the information of targets of different sizes and improve the accuracy of the network. However, this method does not consider the correlation between features, which may cause multiple detections of the same target. The region-based deep learning target detection method introduces a region generation module to generate target candidate regions using extracted feature information, and uses a fully connected layer to fine-tune the candidate regions. Since the prediction weight is adjusted twice, the problem of low recognition accuracy of single prediction in the regression method is effectively solved, and the positioning and classification accuracy of the target is improved. For example, the region-based fully convolutional network(R-FCN) is proposed, which considers the characteristics of classification and positioning tasks and introduces a position-sensitive region pooling module to encode the position information of the generated candidate regions, making the extracted features sensitive to the position. However, this method has many parameters in the position-sensitive region pooling module, which prolongs the inference time of the model. To solve the slow calculation problem of the selective search algorithm, the candidate region generation module is improved, and the shared convolution method is used in the detection process to solve the problem of repeated calculation of the convolution module and further improve the detection efficiency of the model. However, to achieve the accuracy of detection, the region convolutional neural network and its related algorithms have too many parameters in the fully connected layer at the end of the network, which will consume a lot of calculation time and seriously affect the detection speed of the damaged target. A light-weight region convolutional neural network(Light-Head R-CNN) is proposed, which uses large-scale separable convolution to reduce the dimension of the feature map in the region generation and reduces the parameters of the fully connected layer in the network prediction, further improving the detection performance of the network and achieving the balance between detection accuracy and speed.But in practical application, the Light-Head R-CNN network still has the following problems:

[0004] (1) The network adopts a deep convolution layer and an excessively large output feature dimension when extracting features. In the prediction process, there are still many parameters in the full connection layer. Therefore, the calculation complexity of the network is high, the training and inference time of the model is slow, and the real-time requirement of detection cannot be met.

[0005] (2) The network only uses deep features to generate target candidate regions, and does not combine the detailed information of the shallow features of the network. With the increase of the degree of feature abstraction, the small target information of the shallow layer of the network gradually disappears, so the recognition accuracy of the algorithm for micro damage is not high. SUMMARY

[0006] The application provides a conveyor belt tear detection method for improving a region convolutional neural network.

[0007] The application is implemented as follows: a conveyor belt tear detection method for improving a region convolutional neural network comprises the following steps:

[0008] S1: a CCD industrial camera is used to collect a conveyor belt image with a region light source, a damaged part existing in the conveyor belt image is labeled for a damage type, a conveyor belt damage data set is formed, and the damage types include scratches, scratches and tears;

[0009] S2: a light network Light-Head R-CNN based on a region method is constructed, including a feature extraction module, a multi-scale feature fusion module, an attention module and a classification regression module;

[0010] The feature extraction module is set as a light convolutional network MoblieNetV2, uses a deep convolution to decompose features channel by channel, and adopts a point convolution to change the feature dimension;

[0011] The multi-scale feature fusion module introduces a feature pyramid structure, and adopts two ways of top-down and horizontal connection to fuse feature information of different levels;

[0012] The attention module calculates the weight information of the feature channel, and weights the corresponding feature channel of the feature map to complete the adjustment of the weight;

[0013] The classification regression module obtains feature information and classifies the feature image;

[0014] S3: input the conveyor belt damage data set into the Light-Head R-CNN network for training until the loss value is reduced to the minimum and tends to be stable; during the training process, the feature information of the scratch, scratch and tearing part is obtained, and the damage type of the conveyor belt image is predicted;

[0015] S4: real-time acquisition of the conveyor belt image, inputting the conveyor belt image into the trained Light-Head R-CNN network;

[0016] If the image features contain tearing, the conveyor belt is immediately stopped;

[0017] If the image features contain scratches, a warning is issued, and the conveyor belt is not controlled to stop;

[0018] If the image features contain scratches, the conveyor belt is normally operated.

[0019] In step S2, the MobileNetV2 network is composed of one standard convolutional layer and 17 reverse residual modules, each of which contains a 1x1 convolution, a 5x5 deep convolution and a Relu6 activation function.

[0020] In step S2, the fusion process of the multi-scale feature fusion module is represented as:

[0021]

[0022] wherein, is the feature map of the i-1 layer after fusion, is the four scale feature maps output by the feature extraction module, up(.) represents the upsampling operation, and the bilinear interpolation is selected to continuously upsample the feature map by 2 times for fusion; Attention(.) represents the attention module, which adjusts the weight of the fused feature map; represents the addition operation;

[0023] After the low-level feature map and the high-level feature map are adjusted to the same dimension by the upsampling operation and the 1x1 convolution respectively, the attention mechanism is used to adjust the channel weight, highlighting the target area after fusion.

[0024] In step S2, the attention module is added in the fusion process, and the global pooling operation is used in the weight extraction process to compress the feature map and obtain the global information of the feature. For any input feature map X H×W×C , the calculation formula is as follows:

[0025] G max = max(X(i,j))

[0026]

[0027] Where H and W are the length and width of the feature map, and C is the dimension of the feature map; Gmax and Gavg represent the feature vectors obtained after global max pooling and global average pooling operations, respectively, and they have the same dimension, i.e., Gmax, Gavg ∈ R. 1×1×C .

[0028] In step S2, a 1×1 convolution is used to aggregate the concatenated feature vectors and adjust the feature dimensions; the ReLU activation function is used to further improve the non-linearity of the network; finally, the Sigmoid function is used to normalize the weight parameters, and the calculation formula is as follows:

[0029] Y = σ(δ(conv) 1×1 ([G max G avg ])))

[0030] In the formula, Y∈R 1×1×C represents the final output weight vector of the network, σ and δ represent the Sigmoid activation function and the ReLU activation function, respectively, [.] represents the connection operation, and conv1×1 represents the 1×1 convolution operation.

[0031] In step S2, the channel information of the original feature map is adjusted using the feature weights Y learned by the network to obtain the enhanced attention feature map G:

[0032]

[0033] in, This is a product operation, which means adding the feature weights learned by the network to the original feature map to complete the adaptive allocation of channel weights.

[0034] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention discloses an improved method for conveyor belt tear detection using a regional convolutional neural network. Addressing the problem of high computational complexity caused by deep convolutional layers during feature extraction, a lightweight network, MobileNetV2, is introduced. Depthwise separable convolutions preserve feature representation capabilities, significantly reducing model parameters. Simultaneously, to address the issue of a small receptive field, the size of the deep convolutional kernel is increased to enhance the receptive field increment and improve the network's feature extraction capability. To address the problem of low accuracy in detecting minor damage, a feature pyramid structure is introduced. Multi-scale feature fusion enables the network to simultaneously learn deep and shallow feature information, helping to enhance the detection capability of minor damage. To address the accuracy loss caused by feature scale transformation during feature fusion, an attention module is used to adjust feature weights, allowing the network to effectively select important channel information and highlight features of the target region. Attached Figure Description

[0035] Figure 1 Figure 1 is a general network architecture diagram of the present application;

[0036] Figure 2 Figure 2 is a MobileNetV2 network architecture diagram of the present application;

[0037] Figure 3 Figure 3 is a reverse residual module structure diagram of the present application;

[0038] Figure 4 Figure 4 is a multi-scale feature fusion module structure diagram of the present application;

[0039] Figure 5 Figure 5 is an attention module structure diagram of the present application;

[0040] Figure 6 Figure 6 is a network loss function diagram of the present application;

[0041] Figure 7 Figure 7 is a multi-class average precision comparison result diagram of the present application;

[0042] Figure 8 Figure 8 is a conveyor belt damage sample detection effect comparison diagram of the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0044] In the description of the present application, it should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, in the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0045] EMBODIMENT

[0046] As shown in Figure 1 the present application adopts a lightweight network Light-Head R-CNN based on a region method to detect conveyor belt defects, and constructs a lightweight network Light-Head R-CNN based on a region method;

[0047] Firstly, the feature extraction module of the network is set to a lightweight convolutional network MobileNetV2, the features are decomposed channel by channel using deep convolution, and the feature dimension is changed using point convolution, which can bring a significant reduction in computational complexity and greatly improve the detection efficiency of the algorithm.

[0048] To improve the detection accuracy of the algorithm, in the feature fusion module, two ways of top-down and horizontal connection are used to fuse feature information of different levels to improve the recognition accuracy of different damaged targets.

[0049] The attention mechanism is used to adjust the feature weight, so that the network focuses on learning key features and suppresses irrelevant features.

[0050] By replacing the feature extraction network with a lightweight convolutional network MobileNetV2 to improve the detection speed, and expanding the receptive field of the network to enhance the feature extraction capability, such as the "feature extraction module" in Figure 1 ;

[0051] The feature pyramid structure is introduced to fuse feature information of different levels to improve the detection ability of small damage, such as the "multi-scale feature fusion module" in Figure 1 ;

[0052] The attention mechanism is combined to adjust the feature weight after fusion, and the features of the target area are highlighted, such as the "attention module" in Figure 1 .

[0053] Through the above improvements, the present application can realize real-time and accurate detection of damaged targets.

[0054] To further improve the detection efficiency of the algorithm, the present application uses a lightweight convolutional network MobileNetV2 to replace the feature extraction module of Light-Head R-CNN, wherein the MobileNetV2 network structure is as shown in Figure 2 .

[0055] As can be seen from Figure 2 , the MobileNetV2 network mainly consists of a standard convolutional layer and 17 inverted residual modules.

[0056] Among them, the inverted residual module is as shown in Figure 3As shown, each module contains 1x1 convolution, 3x3 depth-wise convolution (Dwise) and Relu6 activation function. The reverse residual module in the left figure needs summation operation, so the input and output feature dimensions are the same. The right figure is a down-sampling residual block, the size of the output feature will be halved. MobileNetV2 draws on the idea of residual network, and the use of shortcut connections in the reverse residual module can effectively prevent gradient disappearance, and can pass gradient information to the deep layer of the network during back propagation, forming effective training.

[0057] In the convolutional neural network, the receptive field refers to the size of the region mapped by the pixel points of the output feature map of a certain layer of the network on the input image. A larger receptive field means that the network can better integrate the dependency between pixels and extract more rich and global feature information. The calculation method of the receptive field is as follows:

[0058]

[0059] In the formula, RFi and RFi-1 represent the size of the receptive field of the i-th and i-1-th layers of the network respectively, sk represents the step size of the k-th layer of the convolution kernel, and ki is the size of the convolution kernel of the i-th layer of the network. It can be concluded that when the network uses a larger convolution kernel size and a larger number of down-sampling convolution operations, the receptive field of the network also increases. In the MobileNetV2 network, the reverse residual module uses a 3x3 convolution operation, which brings a small increase in the field of view, and the network uses a small number of down-sampling residual modules to preserve feature information. Therefore, the entire network has a small increase in the receptive field, and cannot well integrate the context information, which seriously affects the ability to extract features.

[0060] Based on the above analysis, the present application improves the reverse residual module of the network, uses a 5x5 depth convolution instead of a 3x3 depth convolution operation on the output feature maps of different sizes of the network, and changes the size of the convolution kernel to significantly increase the receptive field of the network during network iteration, thereby extracting more rich semantic information. The resolution of the shallow feature map of the network is large, and the use of a larger convolution kernel can effectively preserve the detail information of the image and improve the positioning ability of the conveyor belt micro damage.

[0061] The present application introduces a feature pyramid network (FPN) into the network, and designs a multi-scale feature fusion module, as shown in Figure 4

[0062] ​The feature information of the network is fused by using top-down and horizontal connection, and after feature extraction, the network can output four scale feature maps, which have a step of {4, 8, 16, 32} pixels relative to the input image, and are denoted as {C2, C3, C4, C5}. In order to more comprehensively fuse the feature information of different levels, the four scale feature maps output by the network are all used. The fusion process can be represented as:

[0063]

[0064] wherein, is the feature map of the i-1th layer after fusion, corresponding to {P2, P3, P4, P5} in the figure. is the four scale feature maps {C2, C3, C4, C5} output by the feature extraction module. up(.) represents the up-sampling operation, and for the sake of simplicity, the bilinear interpolation is selected to continuously up-sample the feature map by 2 times for fusion. Attention(.) represents the attention module, which adjusts the weight of the fused feature map. represents the addition operation. After adjusting the low-level feature map and the high-level feature map to the same dimension through the up-sampling operation and the 1x1 convolution respectively, the attention mechanism is used to adjust the channel weight, highlighting the target area after fusion. Finally, the feature map C5 is subjected to the maximum pooling operation with a step of 2 to generate a feature map P6, which is used together with the fused feature maps {P2, P3, P4, P5} for training in the candidate region generation stage, further improving the prediction accuracy of the algorithm for the candidate frame.

[0065] In this way, the network effectively combines the advantages of the feature pyramid in fusing different size feature maps, and uses the attention mechanism to enhance the feature information, so that the feature map output by the network can well fuse the semantic information of the high-level feature and the detail information of the low-level feature, which helps to improve the detection accuracy of the network.

[0066] In order to improve the performance of the network, inspired by the attention mechanism, the attention module as shown in Figure 5 is added in the fusion process to improve the fusion method of the feature. The module calculates the weight information of the channel through the right branch, and then weights the corresponding channel of the left branch feature map to complete the adjustment of the weight.

[0067] The specific process is as follows:

[0068] In order to reduce the interference of spatial information and improve the sensitivity of the network to channel information, the global pooling operation is used to compress the feature map in the weight extraction process to obtain the global information of the feature. For any input feature map X H×W×C , the calculation formula is as follows:

[0069] G max= max (X(i,j))

[0070]

[0071] Wherein, H, W are the length and width of the feature map, C is the dimension of the feature map;Gmax, Gavg respectively represent the feature vectors obtained after global maximum pooling and global average pooling operation, they have the same dimension size, that is, Gmax, Gavg∈R 1×1×C .

[0072] In order to better obtain the dependence between channels, the weight information is distributed. The application uses 1*1 convolution to aggregate the connected feature vectors and adjust the dimension of the features;Using Relu activation function to further improve the nonlinear ability of the network, finally using Sigmoid function to normalize the weight parameter, the calculation formula is as follows:

[0073] Y = σ (delta (conv 1×1 ([G max ;G avg ])))

[0074] In the formula, Y∈R 1×1×C is the final output weight vector of the network, σ, delta respectively represent Sigmoid activation function and Relu activation function, [.] is the connection operation, and conv1*1 is 1*1 convolution operation.

[0075] The feature weight Y learned by the network is used to adjust the channel information of the original feature map, and the enhanced attention feature map G is obtained:

[0076]

[0077] Wherein, is the product operation, which means that the feature weight learned by the network is weighted to the original feature map, and the adaptive distribution of channel weight is completed.

[0078] The application adopts a lightweight network Light-Head R-CNN based on region method to detect the defects of the conveyor belt, and the specific detection process is as follows:

[0079] Constructing sample data set: through CCD industrial camera, the conveyor belt image with area light source is collected, and the damaged parts are labeled, forming a complete conveyor belt damage data set.

[0080] Transfer learning: based on the pre-trained MobileNetV2 network on the data set, the model is fine-tuned. The weight parameters of the first two stages of the frozen model are adjusted, and the configuration information of the model is continuously adjusted, so that the network reaches better classification accuracy.

[0081] Training network: The model structure is built and trained using a deep learning framework. During the experiment, when the loss value decreases to the minimum and tends to be stable, it indicates that the model has converged. According to the experimental results, the feature information of the scratch, scratch, and tear part is obtained, and the damage type of the conveyor belt image is predicted.

[0082] The system responds in real time: if the conveyor belt surface is normal or appears to be worn, the system operates normally; if there is a scratch, the system issues a warning; if there is a tear, the conveyor belt stops immediately.

[0083] Test example

[0084] In the experiment, the industrial camera (MV-GED200M-T) of Mide Vision was used to shoot the conveyor belt image, and the collection speed could reach 27 frames / s, which could clearly capture the image information during the operation of the belt. When the belt machine reached a stable state, a surface light source was added, and the distance between the surface light source and the conveyor belt was adjusted so that the light could uniformly illuminate the belt surface. The industrial camera captured the conveyor belt surface image in real time and transmitted it to the computer through the data line. Under the acceleration of the NVIDIA GPU, the processing module classified and located the damage information in the image, and the control module responded in real time according to the damage type.

[0085] In the experiment, 6000 belt images were collected, divided into 6 groups of experiments, each containing 1000 images. In order to increase the diversity of samples in the data set, the resolution and size of the image were controlled by adjusting the belt speed, CCD camera height and resolution. In the first group of experiments, the belt ran at a low speed of 0.5 m / s, the CCD camera height was 0.4 m, and the resolution was 640x480; in the second group of experiments, the belt still ran at a low speed of 0.5 m / s, the CCD camera height was 0.8 m, and the resolution was 1600x1200; in the third group of experiments, the belt ran at a medium speed of 1 m / s, the CCD camera height was 0.4 m, and the resolution was 640x480; in the fourth group of experiments, the belt still ran at a medium speed of 1 m / s, the CCD camera height was 0.8 m, and the resolution was 1600x1200; in the fifth group of experiments, the belt ran at a high speed of 1.5 m / s, the CCD camera height was 0.4 m, and the resolution was 640x480; in the sixth group of experiments, the belt still ran at a high speed of 1.5 m / s, the CCD camera height was 0.8 m, and the resolution was 1600x1200.

[0086] The experiment is configured under the Windows 10 operating system, runs on the PyCharm software platform, the CPU is Intel Xeon Bronze 3104, the GPU is NVIDIA GeForce RTX 1080Ti, the Python programming language is used, and the model structure is built under the PyTorch deep learning framework. After many experiments, according to the accuracy optimization principle, the momentum factor is set to 0.9, the initial learning rate is set to 0.002, and the stochastic gradient descent (SGD) is used to optimize the network parameters. In the experiment, the data is loaded in batches, and the size of the batch is set to 4, that is, 4 pictures are loaded and processed at the same time. In the training process, the accuracy tends to be stable after 30 iterations, and if the number of iterations is increased, overfitting will occur, therefore, the size of the Epoch is set to 30. 6000 collected belt images are used as training samples and are labeled, and the scale information and position category information of the sample are recorded in the xml file. After labeling, 5000 images are randomly selected from the training samples, and 1000 images are used as test samples.

[0087] The overall performance of the model is evaluated by using the multi-class average precision (mAP), and the calculation formula is as follows:

[0088]

[0089] In the formula, AP(i) is the average precision of class i, and n represents the total number of classes. In the experiment, the intersection over union (IOU) of the real position and the predicted position is used to evaluate the accuracy of the prediction, and the calculation formula is as follows:

[0090]

[0091] In the formula, G represents the manually labeled defect position, and P represents the network predicted defect position. The IOU threshold is set to 0.5, that is, when the IOU of the real position and the predicted position is greater than 0.5, it is considered that the target is successfully predicted.

[0092] In order to verify the stability of the improved model, four groups of comparative experiments are set, and the models in documents

[14] ,

[16] ,

[17] and the present application are compared. In the training process, all networks freeze the first 2 stages of convolutional layers, and do not update the parameters. The mean is 0, the standard deviation is 0.01, and the normal distribution is used to initialize the training parameters. The experiment is trained for 30 cycles, the model has reached full convergence and tends to be stable, and the results are as follows: Figure 6The network loss function graph shown.

[0093] wherein document [9] is Wang G, Zhang L, Sun H, et al. Longitudinal tear detection of conveyorbelt underuneven light based on Haar-AdaBoost and Cascade algorithm [J]. Measurement, 2021, 168(1): 108341.

[0094] Document

[14] is Chiu Y C, Tsai C Y, MD Ruan, et al. Mobilenet-SSDv2: An Improved Object Detection Model for Embedded Systems [C] / / 2020 International Conference on System Science and Engineering (ICSSE), 2020: 1-5.

[0095] Document

[16] is Ren S, He K, Girshick R, et al. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks [J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149.

[0096] Document

[17] is Li Z, Peng C, Yu G, et al. Light-Head R-CNN: in defense of two-stage object detector [J]. arXiv preprint arXiv:1711.07264, 2017.

[0097] In Figure 6 , the ordinate represents the network training loss value, and the abscissa represents the network iteration period. From Figure 6It can be seen that the loss curve of document

[14] drops slowly, and the loss value is high when the network converges, which is 0.1717, indicating that the learning ability of the network is poor, and underfitting phenomenon occurs in the training process; the loss values of document

[16] and document

[17] are 0.07293 and 0.07286 respectively when the training is completed, and the difference is not large. Compared with document

[16] and document

[17] , the algorithm of the application has a faster convergence speed after 5 iterations. When the training is completed, the loss value of the algorithm of the application is stable at 0.06988, and the loss value is the lowest. Therefore, the algorithm of the application has better stability.

[0098] To verify the effectiveness of the improved algorithm of the application, the basic algorithm of document

[17] and the algorithm of the application are compared. The average precision comparison result graph of the algorithm is shown in FIG. 2. Figure 7

[0099] Figure 7 In the figure, the vertical coordinate represents the multi-class average precision of the algorithm, and the horizontal coordinate represents the network iteration period. Figure 7 It can be seen that: in the early stage of training, the network average precision of the application model rises slowly, because the weight parameters of the feature extraction module of the application model are less, and the learning ability of the network in the initial stage is weak. After 10 iterations, the detection precision of the model of document

[17] increases slowly, while the application model still has a large upward space. At the 18th period, the precision curve of document

[17] fluctuates, indicating that the network has an unstable phenomenon in the training process, while the precision curve of the application algorithm fluctuates in a small range, and the model is relatively stable in training. When the training is completed, the average precision of the application algorithm is improved by 2.78% compared with document

[17] . Therefore, the model of the application is better than the model of document

[17] .

[0100] To verify the overall performance of the model, document [9], document

[14] , document

[16] , document

[17] and the model of the application are compared, and the comparison results are shown in Table 1. When the comparison experiment is carried out, the application selects common damage types such as wear, scratch and tear, and the test sample is 1000 images.

[0101]

[0102] Table 1

[0103] ​The detection accuracy of the comparison algorithm on three defects can be seen: the average accuracy of the literature [9] for wear, scratch and tear is much lower than other algorithms, which shows that the deep learning algorithm used in the application has more advantages compared with the traditional machine learning method. The underfitting phenomenon occurs in the training of the literature

[14] , and the network has poor learning ability for damage characteristics, so the detection accuracy is low. Compared with the literature

[16] and the literature

[17] , the detection accuracy of the model of the application for belt scratch and tear is improved, and the improvement for wear is larger. Therefore, the detection accuracy of the model of the application is better.

[0104] The average reasoning time of the comparison algorithm can be seen: the literature [9] and the literature

[14] have greater advantages in speed compared with other algorithms, but their detection accuracy is lower. The literature

[16] and the literature

[17] have improved accuracy, but the network has a large number of parameters in the feature extraction stage, which prolongs the reasoning time of the model. Compared with the literature

[16] and the literature

[17] , the reasoning time of the network of the application is reduced by 22.9%, which improves the detection efficiency of the algorithm. At the same time, the detection accuracy of the network of the application is improved to 90.9%, which can realize good detection effect. Therefore, the overall performance of the model of the application is better than other algorithms.

[0105] In order to prove the improvement effect of each improved module, the application adopts ablation experiment to analyze the performance of different modules, and the experimental results shown in table 2 are obtained. Among them, improvement 1 is to optimize the feature extraction module, improvement 2 is to add a multi-scale feature fusion module, and improvement 3 is to introduce an attention mechanism.

[0106]

[0107] Table 2

[0108] From table 2, it can be seen that after adding improvement 1 to the literature

[17] based model, the model reasoning time is reduced by 31.4%, which shows that setting the feature extraction module as a lightweight MobileNetV2 network can greatly reduce the parameter quantity of the model and effectively improve the detection efficiency and prediction speed of the model; after adding improvement 2 to the experimental one, the network optimizes the sensitivity of the model to small damage targets by fusing features of different levels, and the detection accuracy is improved by 0.44% compared with the literature

[17] and 0.94% compared with the experimental one; after adding improvement 3 to the experimental two, the detection accuracy is further improved after the feature weight is adjusted by the attention mechanism, which is improved by 2.78% and 2.34% compared with the literature

[17] and the experimental two respectively, and the reasoning time is only increased by 6.6%, without introducing large calculation cost. After adopting the above three improvements, compared with the literature

[17] based model, the application has higher real-time and accuracy, and is more stable and reliable in the detection of conveyor belt damage task.

[0109] AsFigure 8 As shown in the figure, the figure from left to right is the image of the conveyor belt of wear, scratch, tear and multiple damage targets, and the white box in the figure is the detection result. It can be seen that the model of the literature

[17] and the present application can realize the classification and positioning of the conveyor belt damage. But when the size of the damage target is small, such as Figure 8 (d) shown, the bounding box position error predicted by the literature

[17] is large, and cannot realize accurate fitting with the damage target, while the bounding box fitting degree predicted by the model of the present application is high, and is closer to the real bounding box manually annotated. Through comparative analysis, it can be seen that when there is a single type of damage or multiple types of damage in the image, the model of the present application can obtain accurate prediction.

[0110] The present application proposes an improved method based on regional convolutional neural network Light-Head R-CNN through the research on the demand of real-time detection of the conveyor belt. In view of the problem that the conveyor belt runs fast and causes the detection to be not timely, the algorithm of the present application adopts the lightweight network MobileNetV2 in the feature extraction module, effectively reduces the model parameter quantity, shortens the network prediction time, and compared with the original Light-Head R-CNN algorithm, the detection speed of the model is improved by 22.9%, which meets the real-time requirement of the conveyor belt tear detection task; In view of the problem that the poor underground environment leads to the existence of more noise in the collected image, the present application introduces the feature golden module to fuse the shallow layer information, and improves the detection accuracy of the small damage. The attention mechanism is used to improve the feature fusion mode, and the feature information of the damage area is highlighted. Compared with the Light-Head R-CNN algorithm, the average detection accuracy of the model on the wear, scratch and tear targets is improved by 3.72%, 2.28% and 2.32% respectively, which improves the recognition effect of the damage target. The experimental results show that the detection efficiency and average detection accuracy of the algorithm of the present application are improved, especially for the small damage target, which also has good recognition accuracy, and has certain reference value in practical application, and provides a new solution for the research on the conveyor belt tear detection task.

[0111] The above is only a preferred embodiment of the present application, and does not limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An improved method for conveyor belt tear detection using a regional convolutional neural network, characterized in that, Includes the following steps: S1: Acquire images of the conveyor belt with regional light sources using a CCD industrial camera, label the damaged parts in the conveyor belt images with damage types, and form a conveyor belt damage dataset. Damage types include scratches, abrasions, and tears. S2: Construct a lightweight network, Light-Head R-CNN, based on region-based methods, including a feature extraction module, a multi-scale feature fusion module, an attention module, and a classification and regression module; The feature extraction module is set to a lightweight convolutional network MoblileNetV2, which uses depthwise convolution to decompose features channel by channel and pointwise convolution to change the feature dimension. The multi-scale feature fusion module introduces a feature pyramid structure and uses two methods, top-down and horizontal connection, to fuse feature information at different levels. The attention module calculates the weight information of the feature channels and weights the corresponding feature channels of the feature map to complete the weight adjustment. The classification and regression module acquires feature information and classifies the feature images; S3: Input the conveyor belt damage dataset into the Light-Head R-CNN network for training until the loss value drops to the minimum and tends to stabilize; during the training process, obtain the feature information of scratches, lacerations and tears, and predict the damage type of the conveyor belt image; S4: Acquire images of the conveyor belt in real time and input the images into the trained Light-Head R-CNN network; If the image features contain tearing, the control conveyor belt should stop immediately; If the image features contain scratches, a warning is issued, but the conveyor belt is not stopped. If the image features contain scratches, the conveyor belt will operate normally.

2. The conveyor belt tear detection method based on an improved regional convolutional neural network according to claim 1, characterized in that, In step S2, the MobileNetV2 network consists of a standard convolutional layer and 17 inverse residual modules, each of which contains a 1×1 convolution, a 5×5 depthwise convolution, and a ReLU6 activation function.

3. The conveyor belt tear detection method based on an improved regional convolutional neural network according to claim 1, characterized in that, In step S2, the fusion process of the multi-scale feature fusion module is represented as follows: in, It is the feature map of the (i-1)th layer after fusion. These are feature maps at four scales output by the feature extraction module. up(.) represents the upsampling operation, which uses bilinear interpolation to continuously upsample the feature maps by a factor of 2 before fusion. Attention(.) represents the attention module, which adjusts the weights of the fused feature maps. This indicates an addition operation; After adjusting the low-level and high-level feature maps to the same dimension through upsampling and 1×1 convolution, an attention mechanism is used to adjust the channel weights to highlight the fused target region.

4. The conveyor belt tear detection method based on an improved regional convolutional neural network according to claim 1, characterized in that, In step S2, an attention module is added during the fusion process, and global pooling is used to compress the feature map during weight extraction to obtain global information of the features. For any input feature map X∈R H×W×C The calculation formula is as follows: G max =max(X(i,j)) Where H and W are the length and width of the feature map, and C is the dimension of the feature map; Gmax and Gavg represent the feature vectors obtained after global max pooling and global average pooling operations, respectively, and they have the same dimension, i.e., Gmax, Gavg ∈ R. 1 ×1×C .

5. The conveyor belt tear detection method based on an improved regional convolutional neural network according to claim 1, characterized in that: In step S2, a 1×1 convolution is used to aggregate the concatenated feature vectors and adjust the feature dimensions; the ReLU activation function is used to further improve the non-linearity of the network; finally, the Sigmoid function is used to normalize the weight parameters, and the calculation formula is as follows: Y=σ(δ(conv 1×1 ([G max ;G avg ]))) In the formula, Y∈R 1×1×C represents the final output weight vector of the network, σ and δ represent the Sigmoid activation function and the ReLU activation function, respectively, [.] represents the connection operation, and conv1×1 represents the 1×1 convolution operation.

6. The conveyor belt tear detection method based on an improved regional convolutional neural network according to claim 1, characterized in that, In step S2, the channel information of the original feature map is adjusted using the feature weights Y learned by the network to obtain the enhanced attention feature map G: in, This is a product operation, which means adding the feature weights learned by the network to the original feature map to complete the adaptive allocation of channel weights.

Images