Belt wear state detection method based on improved YOLOv8-EfficientNet

Abstract

The present invention relates to the technical field of target detection methods, and in particular to a belt wear state detection method based on improved YOLOv8‑EfficientNet. The present invention uses the most advanced real-time monitoring algorithm YOLOv8 target detection algorithm as a basis, combines it with the improved EfficientNet convolutional neural network, and improves the combined network. The acquired conveyor belt wear picture is preprocessed, and the preprocessing technology uses the Retinex image enhancement algorithm to perform brightness enhancement, detail protection, color protection, naturalness restoration, image denoising, detail extraction, etc. on the image to improve the clarity of the belt wear picture. Finally, the improved algorithm is deployed to the conveyor real-time detection equipment to detect the conveyor belt, and the wear parts of the conveyor belt and the degree of wear of each part can be accurately obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection methods, and in particular to a belt wear state detection method based on improved YOLOv8-EfficientNet. Background Art

[0002] As the main consumable part of the belt conveyor in the mine, the detection of its wear status consumes a lot of manpower and material resources. Considering that the special environment of the mine, which is humid, dim, and noisy, has a great impact on image acquisition, and the quality of the acquired image will affect the accuracy of the detection model, how to effectively pre-process the image is extremely important in the detection process. The more mature image enhancement technology currently used has problems of losing details, insufficient local enhancement or excessive enhancement when applied to the special environment of the mine, so it is difficult to meet the requirements.

[0003] At the same time, although deep learning is widely used in target detection, it does not necessarily have a good detection effect as the working conditions and scenarios change. In addition, there is currently no good classification to effectively define the degree of belt wear, that is, to what extent the belt is worn and its performance does not meet the working requirements. In order to solve the above problems, the present invention proposes a belt wear state detection method based on improved YOLOv8-EfficientNet. Summary of the invention

[0004] The purpose of the present invention is to provide a belt wear state detection method based on improved YOLOv8-EfficientNet, so as to quickly and accurately detect the worn parts of the conveyor belt and the degree of wear of each part.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A belt wear state detection method based on improved YOLOv8-EfficientNet includes the following contents:

[0007] S1. Use CCD camera to collect images of different wear degrees of coal mine belt conveyor belts to produce wear data sets; obtain belt performance of different wear degrees according to tensile performance test; annotate data sets using annotation software; amplify data sets, and divide data sets into training set and validation set;

[0008] S2, based on the traditional Retinex image enhancement algorithm, improve it; use Gaussian filtering and bilateral filtering as center surround functions, and after processing, perform channel fusion with the image that has undergone histogram equalization and adaptive brightness adjustment; send the image collected in S1 to the improved image enhancement algorithm for processing;

[0009] S3. Establish a belt wear state detection model based on the improved YOLOv8-EfficientNet algorithm, specifically including: replacing the YOLOv8 backbone network with the EfficientNet module, transforming it into a lightweight network model, and performing hybrid scaling on the convolutional layer without changing the layer structure in the EfficientNet module to achieve the optimal performance model structure;

[0010] S4, training the belt wear state detection model of the improved YOLOv8-EfficientNet algorithm constructed in S3, sending the training set and the verification set in the belt wear images collected in S1 to the constructed belt wear state detection model based on the improved YOLOv8-EfficientNet algorithm for training; performing performance evaluation and assessment on the trained belt wear state detection model of the improved YOLOv8-EfficientNet algorithm to obtain a well-converged network model;

[0011] S5. Evaluation model: According to the belt wear state detection model based on the improved YOLOv8-EfficientNet algorithm obtained after training, the average detection accuracy and detection speed of the model are evaluated;

[0012] S6. Apply the belt wear state detection model based on the improved YOLOv8-Efficient Net algorithm that meets the actual working conditions evaluated in S5 to the actual working conditions to perform actual detection of conveyor belt wear.

[0013] Preferably, the preparation of the wear data set described in S1 specifically refers to: using the belt wear images collected on the unloaded belt to prepare the data set, using a CCD camera to collect the wear images of the coal mine belt conveyor belt to prepare a data set for training, dividing the NN-100 nylon conveyor belt into categories according to the structural characteristics, characteristics of different wear degrees and performance data obtained from the tensile performance experiment, and marking the area and category information of the worn parts in all images.

[0014] Preferably, the amplification of the data set and the division of the data set into a training set and a verification set described in S1 specifically refer to: adding images of different angles by flipping and mirroring the belt wear images, and using a mosaic algorithm to achieve image data enhancement, expanding the data set by splicing four pictures by randomly scaling, randomly cropping, and randomly arranging them, and then dividing the belt wear image data set into a belt wear target detection training set and a belt wear target detection verification set in a ratio of 8:2.

[0015] Preferably, S3 further includes: setting a mixing factor φ in the EfficientNet module to perform mixed scaling on the network depth, width and resolution so that the network has optimal performance; inserting a spatial and channel attention mechanism by introducing a NAM attention module into the three effective feature layers of the improved YOLOv8-EfficientNet model backbone output; and using a k-means++ clustering algorithm on the data set to obtain an anchor box suitable for the target size of coal gangue.

[0016] Preferably, the EfficientNet module uses a combination of feature maps of the input image after convolution downsampling and compression 3, 4, and 5 times as the feature map input to the prediction network, which achieves good results in multi-scale target detection.

[0017] Preferably, the inserting of the spatial and channel attention mechanism by introducing the NAM attention module specifically refers to: embedding a NAM attention module at the end of each network block, and for the residual network, embedding the NAM attention module at the end of the residual structure; for the channel attention submodule, the channel and spatial attention submodule are redesigned using the batch normalization scaling factor.

[0018] Preferably, the belt wear state detection model of the improved YOLOv8-EfficientNet algorithm constructed in the training step 2 in S4 specifically refers to: setting training parameters, the number of training iterations is 300 times, the first 50 times are trained using a freezing method, and the learning rate is 0.001; the learning rate for the next 250 iterations is 0.0001, and the training process uses an annealed cosine algorithm and label smoothing.

[0019] Compared with the prior art, the present invention provides a Raw domain image and video de-moiré method based on channel modulation and spatial modulation, which has the following beneficial effects:

[0020] (1) The detection model proposed in the present invention is based on the YOLOv8 target detection algorithm, which is the most advanced real-time monitoring algorithm, so that it has high detection accuracy under normal conditions. At the same time, the improved EfficientNet convolutional neural network replaces the YOLOv8 backbone feature extraction network to ensure that its detection speed meets the real-time requirements, and introduces the NAM attention module at the output channel of its backbone feature extraction network to make the model pay more attention to the wear area.

[0021] (2) In order to solve the problem of low quality of images collected by the model in the mine, the present invention is based on the traditional Retinex image enhancement algorithm, uses Gaussian filtering and bilateral filtering as center surround functions, and then performs channel fusion processing on the processed image and the image that has undergone histogram equalization and adaptive brightness adjustment.

[0022] (3) In response to the problem that the classification of wear degrees is not clearly defined, the present invention classifies the NN-100 nylon core belt into five different wear degrees according to its structural structure and the wear texture characteristics of different wear degrees in actual working conditions, and uses a tensile performance test to test the five belts with different wear degrees, thereby providing support for the classification of wear degrees with tensile performance strength, which is more reliable.

[0023] In summary, the present invention uses a variety of advanced modules for integration, so that the new network architecture is more in line with the requirements of real-time monitoring; secondly, by improving the Retinex image enhancement algorithm, the ability of detail feature extraction is improved, and it is applied to the preprocessing operation, so that the network is greatly improved in detection accuracy, and at the same time, a reliable basis is provided for the classification of belt wear degree. The belt wear state detection method based on the improved YOLO v8-EfficientNet proposed by the present invention can quickly and accurately detect the wear position of the conveyor belt with different wear degrees in the monitoring image. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a diagram of the improved YOLOv8 detection effect mentioned in Example 1 of the present invention;

[0025] Figure 2 This is a structural diagram of the improved EfficientNet algorithm mentioned in Example 1 of the present invention;

[0026] Figure 3 This is a structural diagram of the NAM attention mechanism mentioned in Example 1 of the present invention;

[0027] Figure 4 This is a processing flow chart of the improved Retinex image enhancement algorithm mentioned in Example 1 of the present invention;

[0028] Figure 5 This is a schematic diagram of an image processed by the improved Retinex image enhancement algorithm mentioned in Example 1 of the present invention;

[0029] Figure 6 This is a schematic diagram of the conveyor belt wear identification results mentioned in Example 1 of the present invention. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0031] This study uses the most advanced real-time monitoring algorithm YOLOv8 target detection algorithm as the basis, combines it with the improved EfficientNet convolutional neural network, and improves the combined network. The acquired conveyor belt wear picture is preprocessed. The preprocessing technology uses the Retinex image enhancement algorithm to enhance the image brightness, protect details, protect colors, restore naturalness, denoise the image, extract details, etc., to improve the clarity of the belt wear picture. Finally, the improved algorithm is deployed to the conveyor real-time detection equipment to detect the conveyor belt, which can accurately determine the worn parts of the conveyor belt and the degree of wear of each part. The belt wear state detection method based on the improved YOLOv8-EfficientNet proposed in the present invention is described in combination with specific examples. The specific content is as follows.

[0032] Embodiment 1:

[0033] The present invention proposes a belt wear state detection method based on an improved YOLOv8-EfficientNet algorithm, which specifically includes the following contents:

[0034] Step 1: Collect belt wear images on the unloaded belt of the coal mine to make a data set, use annotation software to annotate the data set, and use data enhancement algorithm to expand the data set. Use CCD camera to collect images of the unloaded part of the coal mine belt conveyor belt to make a training data set. There are 1523 images in total, including five types of wear images: one layer of wear, two layers of wear, three layers of wear, four layers of wear, and five layers of wear. All images are annotated with area and category information using labelimg software.

[0035] The belt wear images of the coal mine belt conveyor are collected using a CCD camera. Images at different angles are added by flipping and mirroring to expand the data set.

[0036] The mosaic algorithm is used to enhance the data of belt wear images. Four pictures are spliced ​​by randomly scaling, randomly cropping, and randomly arranging them to increase the diversity of the data set background and thus improve the generalization ability of the trained model.

[0037] Step 2: The belt wear image collected in step 1 is processed using the improved Retinex image enhancement algorithm. Based on the original Retinex image enhancement algorithm, the bilateral filter is introduced into the Gaussian filter, which is originally a center-surround function, and then the processed image is combined with the image processed by histogram equalization and adaptive brightness adjustment to perform channel fusion processing of the three images. For the specific process, please refer to Figure 4 .

[0038] For processed image effects, see Figure 5, respectively, are the original image, the original Retinex processed image, and the improved Retinex processed image, and they are put into the detection model for experimental verification. It can be seen that the improved Retinex processed image is of great help in improving the accuracy of the detection model.

[0039] Step 3: Establish a belt wear status detection model based on the improved YOLOv8-EfficientNet algorithm.

[0040] Based on the most advanced real-time monitoring algorithm YOLOv8 target detection algorithm, the YOLOv8 target detection algorithm shows good stability in the actual production process, and its detection accuracy has outstanding advantages. This example combines the actual process of belt wear identification, optimizes the existing YOLOv8 network structure, and constructs a YOLOv8 lightweight model to identify the degree of belt wear, reduce the number of parameters and calculations, enhance feature extraction capabilities, and improve recognition detection accuracy and speed.

[0041] See also Figure 1 , the YOLOv8 target detection model uses the CSPDarknet53 backbone network to extract features. The input image is extracted through the CSPDarknet53 network structure to output three feature maps of size 80×80, 40×40, and 20×20; the feature map is input into the neck network neck for further feature integration. The neck network neck includes the SPPF module and the feature fusion FPN module. The feature map is input into the SPPF module, and the pooled feature map is obtained through the spatial pyramid. The feature map output by the backbone network is fused with the pooled feature map. The head multi-classifier module performs classified target detection based on the multi-scale fusion features output by the feature fusion module. The fusion features of the three scales of 80×80, 40×40, and 20×20 are input into the YOLOhead for classified target detection (such as Figure 1 The network structure diagram of the YOLOv8 algorithm is shown).

[0042] Inputs in the network structure of the improved YOLOv8 algorithm represents input, and the input in this embodiment is a 224×224 size picture. The present invention improves the YOLOv8 model, designs a YOLOv8-EfficientNet network model, replaces the backbone network of YOLOv8 with EfficientNet, and transforms it into a lightweight network model. By hybrid scaling the network scale in EfficientNet, the network model performance is optimized without changing the layer structure, the number of parameters is reduced, and the detection speed is improved.

[0043] See also Figure 2, build the EfficientNet-B4 network, including the data input layer and Baseline;

[0044] The data input layer inputs an image with a size of 224×224.

[0045] The Baseline includes a 3×3 ordinary convolution layer, a stacked MBConv structure, a 1×1 ordinary convolution layer, an average pooling layer, and a fully connected layer;

[0046] The 3×3 common convolution layer includes BN and activation function Swish, and the 1×1 common convolution layer includes BN and activation function Swish;

[0047] The MBConv structure consists of a 1×1 ordinary convolution, a 3×3 Depthwise Conv convolution, an SE module, and a Dropout layer; the 1×1 ordinary convolution layer contains BN and activation function Swish, and the 3×3 Depthwise Conv convolution contains BN and Swish;

[0048] The SE module consists of a global average pooling layer AvgPooling, two fully connected layers FC1 and FC2, the FC1 uses the Swish activation function, and the FC2 uses the Sigmoid activation function.

[0049] The spatial and channel attention mechanisms are inserted into the three effective feature layers of the improved YOLOv8 model backbone output.

[0050] The introduction of the NAM attention module enhances the feature extraction capability of targets in complex environments underground in mines, improves the detection accuracy, and improves the detection capability of small area wear in complex environments.

[0051] See also Figure 3, the NAM attention model uses CBAM module integration, redesigns the channel and spatial attention submodules, and then embeds a NAM module at the end of each network block. For the residual network, it is embedded at the end of the residual structure. For the channel attention submodule, we use batch normalization. The implementation of the channel attention mechanism first performs global average pooling and global maximum pooling on the input feature map at the same time to generate two feature maps. The obtained feature layers are added, and then activated by the sigmoid function to generate the final features required for channel attention. The feature map output by the channel attention module is used as the input feature of the spatial attention module. The input feature layer is subjected to global average pooling and global maximum pooling based on the channel to generate two feature maps. The feature layer is spliced ​​on the channel, and convolution is used for dimensionality reduction, sigmoid activation, and finally the features required for the spatial attention module are generated. The feature maps output by the channel and spatial modules are added, and fused with the output feature map of the backbone network. The feature map after the fusion operation is input to the neck layer of the YOLOv8 network;

[0052] Step 3: Train the belt wear state detection model of the improved YOLOv8-EfficientNet algorithm built in step 2.

[0053] Set the training parameters, the number of training iterations is 300, the first 50 iterations are trained using the freezing method, the learning rate is 0.001, and the learning rate is 0.0001 for the next 250 iterations. The training process uses the annealing cosine algorithm and label smoothing. Send the training set and validation set of the collected belt wear images to the constructed belt wear state detection model of the YOLOv8-EfficientNet algorithm for training;

[0054] Adaptive scaling of training and validation set images for adaptive belt wear image detection based on the input size set by the network;

[0055] Train the belt wear status detection model based on the improved YOLOv8-EfficientNet algorithm according to the set training parameters;

[0056] The performance of the belt wear state detection model of the trained improved YOLOv8-EfficientNet algorithm was evaluated and assessed to obtain a well-converged network model.

[0057] The set test set is sent to the trained belt wear state detection model based on the YOLOv8-EfficientNet algorithm to test the model performance;

[0058] Step 4: Evaluate the model

[0059] According to the trained belt wear state detection model based on the YOLOv8-EfficientNet algorithm, the model is evaluated from the evaluation indicators such as average detection accuracy and detection speed;

[0060] By setting the corresponding parameters, the sample images in the training set are input into the improved YOLOv8-EfficientNet algorithm, the number of iterations is set, and the best weight file for belt wear detection is obtained through training. The CIoU border loss function is used to continuously adjust the direction of network training, and the map value of the validation set is calculated to verify whether the training has achieved the expected effect. The best weight file is selected by screening the trained weight file and loaded into the improved YOLOv8-EfficientNet algorithm as a model for detecting belt wear.

[0061] Step 5: The belt wear detection model based on the improved YOLOv8-EfficientNet algorithm evaluated in step 4 that meets the actual working conditions of the coal mine is applied to the conveyor under actual working conditions. The recognition results are as follows: Figure 6 As shown, it is shown that under the detection of the technical solution of the present invention, belts in different wear states can be accurately detected, indicating that the detection effect of the algorithm proposed in this article is good and can be applied to the detection process under actual working conditions.

[0062] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A belt wear state detection method based on improved YOLOv8-EfficientNet, characterized in that: It includes the following: S1. Use CCD camera to collect images of different wear degrees of coal mine belt conveyor belts, determine the wear category according to the wear texture characteristics of different wear degrees combined with the different wear degrees divided by tensile test, and then make labels for the collected images based on the wear category, mark the wear area and category information, and make a data set; expand the data set, and divide the data set into training set and validation set; The wear categories are classified according to the structural characteristics of the NN-100 nylon conveyor belt, the belt performance data of different wear degrees obtained from the tensile performance test, and the wear texture characteristics of different wear degrees; the data set is to collect belt wear images on an unloaded belt using a CCD camera, mark the area and category information of the wear parts in all images, and produce a wear data set; The data set is expanded as described in S1, and the data set is divided into a training set and a validation set. Specifically, the belt wear image is flipped and mirrored to increase images at different angles, and the mosaic algorithm is used to enhance the image data. The four images are randomly scaled, randomly cropped, and randomly arranged to expand the data set, and then the belt wear image data set is divided into a belt wear target detection training set and a belt wear target detection validation set in a ratio of 8:

2. S2, based on the traditional Retinex image enhancement algorithm, Gaussian filtering and bilateral filtering are used as center-surround functions, and then the processed image is subjected to channel fusion processing with the image subjected to histogram equalization and adaptive brightness adjustment; The image collected in S1 is sent to the improved image enhancement algorithm for processing; S3. Establish a belt wear state detection model based on the improved YOLOv8-EfficientNet algorithm, specifically including: replacing the YOLOv8 backbone network with the EfficientNet module, transforming it into a lightweight network model, and performing hybrid scaling on the convolutional layer without changing the layer structure in the EfficientNet module to achieve the optimal performance model structure; The S3 further includes: setting a mixing factor φ in the EfficientNet module to perform mixed scaling on the network depth, width, and resolution so that the network has optimal performance; inserting a spatial and channel attention mechanism by introducing a NAM attention module into the three effective feature layers of the improved YOLOv8-EfficientNet model backbone output; using a k-means++ clustering algorithm on the data set to obtain an anchor box suitable for the belt wear position; The EfficientNet module uses a combination of feature maps of the input image after convolution downsampling and compression 3, 4, and 5 times. After compression, feature maps are calculated separately at different scales, which can improve the detection accuracy of the algorithm. The said inserting the spatial and channel attention mechanism by introducing the NAM attention module specifically refers to: embedding a NAM attention module at the end of each network block, and for the residual network, embedding the NAM attention module at the end of the residual structure; for the channel attention submodule, using the batch normalization scaling factor to redesign the channel and spatial attention submodule; S4, training the belt wear state detection model of the improved YOLOv8-EfficientNet algorithm constructed in S3, sending the training set and the verification set of the belt wear image processed in S2 to the constructed belt wear state detection model based on the improved YOLOv8-EfficientNet algorithm for training; performing performance evaluation and assessment on the trained belt wear state detection model of the improved YOLOv8-EfficientNet algorithm to obtain a well-converged network model; The belt wear state detection model of the improved YOLOv8-EfficientNet algorithm constructed in the training step 2 described in S4 specifically refers to: setting training parameters, the number of training iterations is 300 times, the first 50 iterations are trained using the freezing method, and the learning rate is 0.001; the learning rate for the next 250 iterations is 0.0001, and the training process uses the annealing cosine algorithm and label smoothing; S5. Evaluation model: According to the belt wear state detection model based on the improved YOLOv8-EfficientNet algorithm obtained after training, the average detection accuracy and detection speed of the model are evaluated; S6. Apply the belt wear state detection model based on the improved YOLOv8-Efficient Net algorithm that meets the actual working conditions evaluated in S5 to the actual working conditions, and perform actual detection of the conveyor belt wear level and position.

Images