Improved YOLOv8 coal flow visual detection method

Abstract

The invention proposes an improved YOLOv8 coal flow visual detection method, and the method comprises the following steps: obtaining a coal mine conveyor belt data set, carrying out the preprocessing of the coal mine conveyor belt data set based on an AMSRCR algorithm, and dividing the preprocessed coal mine conveyor belt data set into a training set and a test set; the method comprises the steps that an improved YOLOv8 model used for coal flow visual detection is constructed, the improved YOLOv8 model comprises a backbone network and a head network, the improved YOLOv8 model introduces a Faster Net module in a C2f module in the backbone network to replace a Bottleneck module to construct a C2f-Faster Net module, and a three-dimensional weight attention module is introduced in a third layer of the backbone network; training is carried out to obtain a trained improved YOLOv8 model; and obtaining real data to be detected, and inputting the real data to the trained improved YOLOv8 model to obtain a classification result. According to the method, the influence caused by dust shielding and uneven illumination can be solved, a clear image is obtained, the floating point operand of the network is reduced, and the system accuracy and the real-time detection efficiency are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a coal flow visual detection method based on an improved YOLOv8. Background Art

[0002] Energy consumption in the coal mining process is a key issue for the sustainable development of the industry. In the production process of coal mines, due to the difference in coal seam storage, the coal output of the fully mechanized mining face fluctuates greatly. The conveyor belt, as the main coal transportation tool, runs at a constant speed all day after startup, and is often in a state of a big horse pulling a small cart or even running empty, resulting in serious waste of energy. The study found that by building an energy-saving control strategy and adjusting the conveyor belt running speed in real time according to the coal flow of the conveyor belt, the energy consumption of the conveyor belt can be effectively reduced.

[0003] Existing conveyor belt coal flow detection methods are mainly divided into two categories: contact and non-contact. Electronic tape scales are currently the most widely used contact dynamic coal flow detection method. However, electronic tape scales have large detection errors, and it is difficult to balance sensitivity and detection accuracy. Non-contact detection methods such as nuclear scales and ultrasonic waves solve the problems of contact belt wear and equipment vibration affecting accuracy. However, nuclear scales require complex calibration, and radioactive materials are harmful to the human body, and the equipment itself has the problem of radioactive source decay, so its scope of application has been limited. The ultrasonic detection method obtains the amount of coal by calculating the distance through the time difference from the emission of sound waves to the reception of sound waves and the transmission speed of sound waves, but it has serious interference in the complex environment of coal mines.

[0004] The invention patent with publication number CN108664874A discloses an underground coal flow detection method based on image recognition, which obtains a video stream of coal materials on a belt conveyor during transportation; obtains a first image frame and a second image frame at a specific time interval from the video stream; obtains a grayscale differential image based on the grayscale difference between the pixels at corresponding positions of the first image frame and the second image frame in a preset belt detection area; analyzes the grayscale differential image to obtain the distribution characteristics of the pixels, segments the image based on the distribution characteristics of the pixels, and determines the coal material boundary; estimates the coal flow based on the coal material boundary. Rapid recognition of coal flow boundaries can be achieved using wavelet analysis, neural networks, or an algorithm combining the two. However, the image obtained by this method is not realistic, and the accuracy and reasoning speed of the algorithm are not high. Summary of the invention

[0005] In view of the technical problems that the image quality obtained by the prior art is poor and the accuracy and reasoning speed of the algorithm are not high, the present invention proposes a coal flow visual detection method based on an improved YOLOv8, preprocesses the image through a multi-scale retinal algorithm enhanced by color gain (AMSRCR), and proposes an improved YOLO-CFS (YOLO-C2f-FasterNet-SimAM) algorithm based on the YOLOv8-cls algorithm. By lightweighting the YOLOv8-cls algorithm and adding a lightweight C2f-FasterNet module and a SimAM attention mechanism, the system accuracy, real-time detection efficiency and reasoning speed are guaranteed.

[0006] In order to achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] A method for visual detection of coal flow using improved YOLOv8, comprising the following steps:

[0008] S1: Obtain a coal mine conveyor belt dataset, preprocess the coal mine conveyor belt dataset based on the AMSRCR algorithm, and divide the preprocessed coal mine conveyor belt dataset into a training set and a test set;

[0009] S2: Build an improved YOLOv8 model for coal flow visual detection, including a backbone network and a head network. The improved YOLOv8 model introduces a FasterNet module in the C2f module in the backbone network to replace the Bottleneck module to build a C2f-FasterNet module, and introduces a three-dimensional weighted attention module in the third layer of the backbone network;

[0010] S3: Input the training set into the constructed improved YOLOv8 model for training, input the test set into the constructed improved YOLOv8 model to evaluate the performance of the trained improved YOLOv8 model and adjust the parameters to obtain a trained improved YOLOv8 model;

[0011] S4: Obtain the real coal mine conveyor belt image to be detected and perform the same preprocessing as step S1, and input it into the trained improved YOLOv8 model to obtain the classification result.

[0012] The coal mine conveyor belt dataset described in step S1 is sourced from images actually shot in real coal mine conveyor belt scenes, and frames are extracted from monitoring videos of coal mine conveyor belt production and operation, to obtain 3008 images in different scenes.

[0013] The preprocessing method described in step S1 is: performing image color enhancement on the coal mine conveyor belt dataset by using the AMSRCR algorithm, adding image noise to the coal mine conveyor belt dataset after image color enhancement, and performing MixUp image enhancement, translation and inversion operations in sequence.

[0014] The calculation process of the AMSRCR algorithm is as follows: defining a reflection component calculation formula, compressing the dynamic range of the original image according to the reflection component calculation formula, convolving the collected original image with Gaussian filtering to obtain illumination information, performing multi-scale Gaussian kernel weighted superposition calculation according to the dynamic range compression result and the illumination information, calculating the color restoration factor according to the original image information, introducing a gain value and an offset value, calculating a multi-scale output image with color restoration according to the color restoration factor and the multi-scale Gaussian kernel weighted superposition calculation result, and obtaining an image that is finally enhanced in color by the AMSRCR algorithm according to the multi-scale output image with color restoration.

[0015] The calculation formula of the reflection component is:

[0016] R(x,y)=S(x,y) / L(x,y)

[0017] Among them, (x, y) is the pixel coordinate position in the original image, S(x, y) is the collected pixel information of the original image, R(x, y) is the reflection component, and L(x, y) is the ambient illumination information of the original image that needs to be eliminated;

[0018] The calculation formula for the dynamic range compression is:

[0019] log[R(x,y)]=log[S(x,y)]-log[L(x,y)]

[0020] The method of obtaining illumination information by convolving the Gaussian filter with the collected original image is:

[0021] L(x,y)=F(x,y)*S(x,y)

[0022]

[0023] Among them, * is the convolution operation, F(x,y) is the Gaussian filter function, and σ is the standard deviation of the Gaussian filter.

[0024] The formula for performing multi-scale Gaussian kernel weighted superposition calculation is:

[0025]

[0026] Among them, ω n is the weight of each scale, N is the number of scales, R MSR (x, y) is the output image with added multi-scales;

[0027] The method for calculating the color restoration factor is:

[0028]

[0029] Where α and β represent the controlled nonlinearity and gain constant, respectively. i (x, y) is the pixel value of the i-th pixel of the original image, and S represents the total number of pixels;

[0030] The method for calculating the multi-scale output image with color restoration is:

[0031] R MSRCRi(x,y) =G[C i (x,y)R MSR (x,y)+b]

[0032] Where G and b are the final gain and offset values, respectively, and R MSRCRi(x,y) Represents the multi-scale output image with color restoration.

[0033] The method for obtaining the image that is finally enhanced in color by the AMSRCR algorithm is as follows: a grayscale histogram is obtained from a multi-scale output image with color restoration, and the AMSRCR algorithm selects upper and lower clipping points for the grayscale histogram according to the frequency of occurrence of pixels in the grayscale histogram, and redistributes the pixel values ​​of the clipped pixels to [0, 255] to obtain an image that is enhanced in color by the AMSRCR algorithm.

[0034] The backbone network is constructed by constructing ten layers of sequentially connected modules: constructing conv modules in the first and second layers, constructing a three-dimensional weighted attention module in the third layer, constructing three groups of identical combination modules in the fourth to ninth layers, each group of combination modules includes a C2f-FasterNet module and a conv module constructed in sequence, and constructing a C2f-FasterNet module in the tenth layer and connecting it to the head network;

[0035] The processing process of the C2f-FasterNet module is as follows: the input features are sequentially subjected to the Split operation and the multiple layers of FasterNet modules, and then the input features are concat-connected with the output of each layer of FasterNet modules, and finally output to the next layer;

[0036] The processing process of the FasterNet module is as follows: the FasterNet module input features are sequentially passed through the PConv layer with a convolution kernel of 3*3, the conv layer with a convolution kernel of 1*1, the batch normalization operation and the ReLU activation function layer, and then residually connected with the FasterNet module input features to obtain the output and enter the next layer;

[0037] The processing process of the conv module is as follows: the input features are sequentially output through the conv2d module, batch normalization operation and ReLU activation function layer, and then enter the next layer;

[0038] The head network includes a detection head and a classification head operating in parallel. The detection head uses convolution and deconvolution operations to generate detection results based on the output feature map of the backbone network, including bounding box regression of each anchor box and the confidence of the target existence. The classification head uses global average pooling, full connection operation and activation function to classify each feature map based on the output feature map of the backbone network, and outputs the probability distribution of each category.

[0039] The processing process of the three-dimensional weighted attention module is as follows: the input feature map X is calculated by the energy function and then combined with the input feature map X of the three-dimensional weighted attention module to obtain an output;

[0040] The method for calculating the energy function is:

[0041]

[0042] in, and They represent the neurons x in the channel except the target neuron t. i , i is the index in the spatial dimension, λ is a hyperparameter, and M=H×W represents the total number of neurons on the channel.

[0043] The method of combining the input feature map X of the three-dimensional weighted attention module to obtain the output is:

[0044]

[0045] Among them, E is the combination of energy functions of all spatial and channel dimensions, sigmoid is the activation function, and ⊙ represents element-wise multiplication.

[0046] Beneficial effects of the present invention:

[0047] 1. Aiming at the dust and uneven lighting problems in the coal mine environment, the image preprocessing is carried out through the color gain enhanced multi-scale retina algorithm (AMSRCR), which can obtain clear images, solve the influence caused by dust occlusion and uneven lighting, and is conducive to the identification of coal flow images.

[0048] 2. In response to the problems of large computational complexity, slow detection time and insufficient accuracy of existing traffic detection models, the C2f-FasterNet module is proposed, which reduces the number of floating-point operations of the network, increases the detection frame rate, improves the detection accuracy, and enhances the real-time detection effect of the algorithm.

[0049] 3. In response to the problem of the model's attention to key features, the SimAM attention mechanism was proposed to increase the weight of key areas of feature extraction, thereby improving the accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 This is a flow chart of a coal flow visual detection method based on an improved YOLOv8 proposed in the present invention.

[0052] Figure 2 It is a comparison chart of the effect of the AMSRCR algorithm on image preprocessing in the present invention and the effects of other algorithms.

[0053] Figure 3 It is a structural framework diagram of the improved YOLOv8 network model in the present invention.

[0054] Figure 4 It is a schematic diagram of the structure of the SimAM attention mechanism in the present invention.

[0055] Figure 5 It is the comparison accuracy curve of different feature extraction modules in the present invention.

[0056] Figure 6 This is the comparison accuracy curve of different attention mechanisms in the present invention.

[0057] Figure 7 It is the comparison accuracy curve of different models in the present invention. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] An improved YOLOv8 coal flow visual detection method, such as Figure 1 As shown, the steps are as follows:

[0060] S1: Obtain a coal mine conveyor belt dataset, preprocess the coal mine conveyor belt dataset based on the color gain enhanced multi-scale retina algorithm (AMSRCR), and divide the preprocessed coal mine conveyor belt dataset into a training set and a test set.

[0061] The source of the coal mine conveyor belt dataset is the images actually taken in real coal mine conveyor belt scenes, as well as the frame extraction of coal mine conveyor belt production and operation monitoring videos, to obtain 3008 images in different scenes.

[0062] The coal mine conveyor belt dataset is enhanced in image color through the multi-scale retina algorithm enhanced by color gain (AMSRCR), and the coal mine conveyor belt dataset after image color enhancement is actively added with image noise, MixUp image enhancement, translation and inversion, etc., to increase the diversity of the dataset and effectively improve the generalization ability of the model.

[0063] The color gain enhanced multi-scale retinal algorithm (AMSRCR) adopted by the present invention is a color constancy algorithm. The algorithm is based on the Retinex algorithm and adds a color restoration function to solve the color distortion problem caused by the Retinex algorithm. The Retinex theory is used in graphics enhancement to set each pixel value in the image as the product of the ambient illumination information and the reflection component. After filtering the illumination information, the original reflection characteristics of the object are obtained.

[0064] The multi-scale retina algorithm with color gain enhancement has the following specific calculation process:

[0065] Define the calculation formula for the reflection component:

[0066] R(x,y)=S(x,y) / L(x,y)

[0067] Among them, (x, y) represents the pixel coordinate position in the original image, S(x, y) is the collected pixel information of the original image, R(x, y) is the reflection component, which reflects the original reflection characteristics of the object, and L(x, y) is the ambient illumination information of the original image that needs to be eliminated;

[0068] Perform logarithmic operation on the above formula to achieve dynamic range compression of the image. The formula is:

[0069] log[R(x,y)]=log[S(x,y)]-log[L(x,y)]

[0070] The illumination information L(x, y) of the image is the key to recovering the reflection component R(x, y), so the problem can be simplified to solving the illumination information L(x, y) of the image.

[0071] The illumination information is obtained by convolving the Gaussian filter with the collected original image S(x,y). The formula is as follows:

[0072] L(x,y)=F(x,y)*S(x,y)

[0073]

[0074] Among them, F(x, y) is the Gaussian filter function, σ is the pre-defined standard deviation of the Gaussian filter, and is the most important parameter in the calculation. The actual value of σ is usually greater than 50 and less than 100. The larger the σ value, the better the image smoothing effect, and the smaller σ is, the more image details are preserved.

[0075] The multi-scale retina algorithm with enhanced color gain (AMSRCR) adds weighted superposition of Gaussian kernels of different scales. On the basis of multi-scale processing, the algorithm introduces a color restoration factor. This factor adjusts the color restoration degree of the image to compensate for the color information lost in the multi-scale Retinex processing process, realizes color restoration, and solves the color distortion problem. An additional gain value and offset value are added to pay more attention to global information. This adjustment helps to restore the true color of the image and avoid color distortion. The specific calculation process is as follows:

[0076]

[0077] Where G and b are the final gain and offset values, C i (x, y) is the color restoration factor, α and β represent the controlled nonlinearity and gain constant, respectively. MSR (x, y) is the output image with multiple scales, N is the number of scales, N is generally taken as 3, representing three scales (i.e., filter processing with different standard deviations or filter ranges), ω n is the weight of each scale. i (x, y) is the pixel value of the i-th pixel in the original image, S represents the total number of pixels, R MSRCRi(x,y) Represents the multi-scale output image with color restoration.

[0078] The above multi-scale output image R with color restoration MSRCRi(x,y) There is still a situation where color restoration is poor. The output image R of the multi-scale retina algorithm with color restoration MSRCRi(x,y) The grayscale histogram is obtained by using visual library statistics. The AMSRCR algorithm selects upper and lower clipping points for the grayscale histogram according to the frequency of occurrence of pixels in the grayscale histogram, and redistributes the pixel values ​​of the clipped pixels to [0, 255], thereby obtaining an image with color enhancement by the AMSRCR algorithm. Gaussian kernel weighted superposition of different scales is added, and an additional gain value and offset value are added to pay more attention to global information. At the same time, the gain value and offset value realize color restoration and solve the color distortion problem. This process is automatically calculated, reducing manual intervention, solving the problem of image color distortion, and fully possessing the three scale information of color image enhancement mentioned above.

[0079] Table 1 Results of image preprocessing evaluation indicators

[0080]

[0081] Among them, PSNR means peak signal-to-noise ratio, SSIM means structural similarity, and RMSE means mean square error.

[0082] Experimental verification shows that the AMSRCR algorithm can achieve image dehazing in coal mine environments and image enhancement in different lighting environments. Figure 2 In order to compare the actual operation effects of image enhancement algorithms, the subjective evaluation of AMSRCR algorithm is better than other algorithms. The evaluation comparison indicators are compared using objective parameters. The results are shown in Table 1. The comprehensive index results of AMSRCR algorithm are better than other algorithms.

[0083] The preprocessed coal mine conveyor belt dataset is classified into four categories according to the coal flow rate: no coal, little coal, medium amount, and full load. The dataset is divided into training set and test set in a ratio of 8 to 2.

[0084] S2: Construct an improved YOLOv8 model for coal flow visual detection, including a backbone network and a head network. The improved YOLOv8 model introduces a FasterNet module in the C2f module in the backbone network to replace the Bottleneck module to construct a C2f-FasterNet module, and introduces a three-dimensional weighted attention module (SimAM) in the third layer of the backbone network.

[0085] The backbone network and the head network are constructed by constructing ten layers of sequentially connected modules, constructing conv modules in the first and second layers, constructing a three-dimensional weighted attention module (SimAM) in the third layer, constructing three groups of identical combination modules in the fourth to ninth layers, each group of combination modules includes a C2f-FasterNet module and a conv module constructed in sequence, and constructing a C2f-FasterNet module in the tenth layer and connecting it to the head network;

[0086] The processing process of the C2f-FasterNet module is as follows: the input features are sequentially subjected to the Split operation and the multiple layers of FasterNet modules, and then the input features are concat-connected with the output of each layer of FasterNet modules, and finally output to the next layer;

[0087] The processing process of the FasterNet module is as follows: the FasterNet module input features are sequentially passed through the PConv layer with a convolution kernel of 3*3, the conv layer with a convolution kernel of 1*1, the batch normalization operation and the ReLU activation function layer, and then residually connected with the FasterNet module input features to obtain the output and enter the next layer;

[0088] The processing process of the conv module is as follows: the input features of the conv module are sequentially output through the conv2d module, batch normalization operation and ReLU activation function layer, and then enter the next layer.

[0089] like Figure 3 As shown in the figure, the improved YOLOv8-cls network model structure uses a YOLOv8-cls model designed for image classification tasks. The C2f module of the YOLOv8s-cls model is combined with the lightweight FasterNet to obtain a C2f-FasterNet module to replace the original C2f module; the three-dimensional weighted attention mechanism SimAM is added to the third layer module of the model.

[0090] Normalization and activation layers are also indispensable for high-performance network structures. However, in order not to overly restrict the diversity of features, resulting in performance loss and increasing network running time, batch normalization and ReLU activation functions are only placed between two Conv layers in the FasterNet module. Batch normalization has the characteristic of being merged into adjacent Conv layers, which can improve the reasoning speed. Among them, PConv is a partial convolution operation, which can effectively improve the accuracy and greatly reduce the running time of the model while maintaining high calculations, thereby improving the reasoning speed of the model. PConv utilizes the redundancy in the feature mapping process. Through rule selection, it convolves some input channels while keeping other channels unchanged, reducing the number of redundant calculations and memory accesses. The Bottleneck module increases the complexity of the network structure, resulting in large amount of calculation and long running time. Therefore, in response to this problem, a C2f-FasterNet module combined with FasterNet is proposed. The FasterNet module is used to replace the previous Bottleneck module, which greatly improves the performance of the model.

[0091] The SimAM attention mechanism module is a lightweight, plug-and-play three-dimensional weighted attention module with a structure like Figure 4 As shown in the figure. This module uses weight generation as an energy function, and then derives the closed solution of this energy function to form a closed loop. No additional parameters are introduced to the network structure, so there is no need to make major changes to the original architecture to further improve the performance of the network structure in the task. By simulating the human brain, a weight is assigned to each neuron. The truly active neurons in the human brain will have spatial inhibition with the surrounding neurons. Based on this principle, neurons with high priority are found and energy functions are defined for each neuron.

[0092] The processing process of the three-dimensional weighted attention module is as follows: the input feature map X of the three-dimensional weighted attention module is calculated by the energy function and then combined with the input feature map X of the three-dimensional weighted attention module to obtain the output; the calculation process is:

[0093] Use the energy function to calculate the input feature map:

[0094]

[0095] in, and Represent the mean and variance of the neurons in the channel except t. t represents the target neuron on a channel, x i represents other neurons, i is the index in the spatial dimension, M = H × W represents the total number of neurons on the channel, and λ is a hyperparameter. The smaller the energy function, the greater the difference between this neuron and the surrounding neurons.

[0096] E is the energy function matrix, which represents the combination of energy functions in all spatial and channel dimensions. On this basis, feature refinement is performed, and the value of the energy function combination E is limited by the sigmoid function to obtain the output:

[0097]

[0098] Where X∈R C×H×W represents the input features of the three-dimensional weighted attention module, and ⊙ represents element-wise multiplication.

[0099] Adding the three-dimensional weighted attention mechanism SimAM to the third-layer module of the model not only enhances the model's ability to extract features, but also makes the network structure more flexible. Adding this module before the C2f-FasterNet module, that is, the L3 module position, can significantly improve the accuracy of the entire algorithm.

[0100] The head network includes a detection head and a classification head operating in parallel, which are responsible for the final target detection and classification tasks. The detection head uses a series of convolution and deconvolution operations to generate detection results based on the output feature map of the backbone network, including bounding box regression of each anchor box and the confidence of the target existence. The classification head uses global average pooling to reduce the dimension of the feature map based on the output feature map of the backbone network, and uses full connection operations and activation functions to classify each feature map and output the probability distribution of each category.

[0101] S3: Input the training set into the constructed improved YOLOv8 model for training, and input the test set into the constructed improved YOLOv8 model to evaluate the performance of the trained improved YOLOv8 model and adjust the parameters to obtain a trained improved YOLOv8 model.

[0102] The experimental operating system used in the present invention is Windows 11, the experimental platform GPU is NVIDIA GeForce RTX3070, the memory is 64G, the programming language is Python-3.8.18, the deep learning framework is PyTorch-2.0.0, and the computing architecture is CUDA-11.8. In the model training stage, the number of training iterations is set to 200, the initial learning rate is set to 0.01, the batch size is set to 16, the optimizer is selected to be automatic, the dropout regularization parameter is set to 0.3, and the MixUp image enhancement parameter is set to 0.2. The present invention classifies coal flow into 4 categories, which belongs to a multi-classification task. In the experiment, accuracy (Acc), model parameters (Params), floating-point operands (FLOPs) and detection frame rate frames per second (FPS) are used as evaluation indicators for performance evaluation. The calculation formula of Acc is as follows:

[0103]

[0104] Among them, TP is the correctly predicted positive sample, TN is the correctly predicted negative sample, FP is the incorrectly predicted positive sample, and FN is the incorrectly predicted negative sample.

[0105] S4: Obtain the real coal mine conveyor belt image to be detected and perform the same preprocessing as step S1, and input it into the trained improved YOLOv8 model to obtain the classification result.

[0106] Experimental comparison:

[0107] According to the coal mine conveyor belt detection model based on the improved YOLOv8-cls algorithm obtained after training, the effectiveness of the network model is comprehensively compared from multiple aspects such as model accuracy, inference speed, and model complexity. The experimental results of different feature extraction modules are shown in Table 2:

[0108] Table 2 Experimental comparison of feature extraction modules

[0109]

[0110]

[0111] Table 2 intuitively shows that the lightweight C2f-FasterNet module reduces the number of model parameters and floating-point operations, and the detection speed is increased by 14.23% compared to the original model, while the accuracy is increased by 3.8%. After further improvement, although the detection speed of the final algorithm in this article has decreased, the accuracy has increased by 1%, which effectively improves the detection effect of the model. Similarly, although the lightweight C2f-ghost has greatly reduced the model complexity, the improvement in detection accuracy is limited. Although the C2f-DynamicConv module and the C3-FasterNet module have improved the accuracy and detection frame rate compared with the original model, the effect is relatively not obvious, and the model is too complex, which is not conducive to hardware deployment. The accuracy curves of different feature extraction modules are shown in the figure below. Figure 5 shown.

[0112] Table 3 Comparison of attention mechanism experiments

[0113]

[0114] As shown in Table 3, the effect was verified by adding different attention mechanisms at the same position. The detection accuracy was improved by adding the attention mechanism, and at the same time, the detection speed of the model was affected. Among them, the number of parameters and floating-point operations of CPCA and CBAM increased, but the accuracy was slightly improved. Although the detection speed of MLCA increased instead of decreased, the accuracy was only improved by 0.1%, which did not meet the original intention of the attention mechanism to improve the detection accuracy. SimAm increased its accuracy by 0.8% on the basis of slightly reducing the detection speed, and the accuracy improvement was the most obvious. This algorithm combines the advantages of the FasterNet network and SimAm, greatly improving the accuracy while increasing the detection speed and reducing the floating-point operations of the model. The accuracy curves of the different attention mechanism models added are shown in the figure below. Figure 6 shown.

[0115] Table 4 Comparative experiments of different models

[0116]

[0117] As shown in Table 4, the effectiveness of this algorithm is verified by comparing with mainstream deep learning algorithms. YOLOv5 is the predecessor of v8, and its accuracy is similar, but the number of parameters and floating-point operations of v8 is greatly reduced, and the detection speed is improved. The detection speeds of EfficientFormerV2 and ResNet5 are both lower than those of v8-cls. In addition, the model parameters of ResNet50 are much higher than those of the algorithm of this invention, and the calculation is complex, which is not conducive to deployment on limited hardware resources. The accuracy curves of different models are shown in Figure 4. Figure 7 shown.

[0118] In summary, image preprocessing is performed to address the dust and uneven lighting problems in coal mine environments, and coal flow images that are easy to identify are obtained. In response to the large amount of computation, slow detection time, and insufficient accuracy of deep learning networks, the C2f-FasterNet module is proposed to reduce the number of floating-point operations in the network, increase the detection frame rate, and improve the detection accuracy, thereby improving the real-time detection effect of the algorithm. In order to further improve the model's attention to key features and improve detection accuracy, the SimAm attention mechanism is proposed to increase the weight of key areas of feature extraction, thereby improving accuracy.

[0119] Experimental results show that the detection accuracy of the YOLO-CFS algorithm is improved by 4.8%, the number of parameters is reduced by 28.4%, the number of floating-point operations is reduced by 33.3%, and the detection frame rate is improved by 5.9%, which proves the effectiveness and real-time performance of the algorithm and provides technical support for energy conservation and emission reduction in coal mines.

[0120] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A coal flow visual detection method based on improved YOLOv8, characterized in that: The following steps are involved: S1: Obtain a coal mine conveyor belt dataset, preprocess the coal mine conveyor belt dataset based on the AMSRCR algorithm, and divide the preprocessed coal mine conveyor belt dataset into a training set and a test set; S2: Build an improved YOLOv8 model for coal flow visual detection, including a backbone network and a head network. The improved YOLOv8 model introduces a FasterNet module in the C2f module in the backbone network to replace the Bottleneck module to build a C2f-FasterNet module, and introduces a three-dimensional weighted attention module in the third layer of the backbone network; S3: Input the training set into the constructed improved YOLOv8 model for training, input the test set into the constructed improved YOLOv8 model to evaluate the performance of the trained improved YOLOv8 model and adjust the parameters to obtain a trained improved YOLOv8 model; S4: Obtain the real coal mine conveyor belt image to be detected and perform the same preprocessing as step S1, and input it into the trained improved YOLOv8 model to obtain the classification result.

2. The improved YOLOv8 coal flow visual detection method according to claim 1 is characterized in that: The coal mine conveyor belt dataset described in step S1 is sourced from images actually shot in real coal mine conveyor belt scenes, and frames are extracted from monitoring videos of coal mine conveyor belt production and operation, to obtain 3008 images in different scenes.

3. The improved YOLOv8 coal flow visual detection method according to claim 1 or 2, characterized in that: The preprocessing method described in step S1 is: performing image color enhancement on the coal mine conveyor belt dataset by using the AMSRCR algorithm, adding image noise to the coal mine conveyor belt dataset after image color enhancement, and performing MixUp image enhancement, translation and inversion operations in sequence.

4. The improved YOLOv8 coal flow visual detection method according to claim 3 is characterized in that: The calculation process of the AMSRCR algorithm is as follows: defining a reflection component calculation formula, compressing the dynamic range of the original image according to the reflection component calculation formula, convolving the collected original image with Gaussian filtering to obtain illumination information, performing multi-scale Gaussian kernel weighted superposition calculation according to the dynamic range compression result and the illumination information, calculating the color restoration factor according to the original image information, introducing a gain value and an offset value, calculating a multi-scale output image with color restoration according to the color restoration factor and the multi-scale Gaussian kernel weighted superposition calculation result, and obtaining an image that is finally enhanced in color by the AMSRCR algorithm according to the multi-scale output image with color restoration.

5. The improved YOLOv8 coal flow visual detection method according to claim 4 is characterized in that: The calculation formula of the reflection component is: R(x,y)=S(x,y) / L(x,y) Among them, (x, y) is the pixel coordinate position in the original image, S(x, y) is the collected pixel information of the original image, R(x, y) is the reflection component, and L(x, y) is the ambient illumination information of the original image that needs to be eliminated; The calculation formula for the dynamic range compression is: log[R(x,y)]=log[S(x,y)]-log[L(x,y)] The method of obtaining illumination information by convolving the Gaussian filter with the collected original image is: L(x,y)=F(x,y)*S(x,y) Among them, * is the convolution operation, F(x,y) is the Gaussian filter function, and σ is the standard deviation of the Gaussian filter.

6. The improved YOLOv8 coal flow visual detection method according to claim 5 is characterized in that: The formula for performing multi-scale Gaussian kernel weighted superposition calculation is: Among them, ω n is the weight of each scale, N is the number of scales, R MSR (x, y) is the output image with added multi-scales; The method for calculating the color restoration factor is: Where α and β represent the controlled nonlinearity and gain constant, respectively. i (x, y) is the pixel value of the i-th pixel of the original image, and S represents the total number of pixels; The method for calculating the multi-scale output image with color restoration is: Where G and b are the final gain and offset values, respectively. Represents the multi-scale output image with color restoration.

7. The improved YOLOv8 coal flow visual detection method according to any one of claims 4 to 6, characterized in that: The method for obtaining the image that is finally enhanced in color by the AMSRCR algorithm is as follows: a grayscale histogram is obtained from a multi-scale output image with color restoration, and the AMSRCR algorithm selects upper and lower clipping points for the grayscale histogram according to the frequency of occurrence of pixels in the grayscale histogram, and redistributes the pixel values ​​of the clipped pixels to [0, 255] to obtain an image that is enhanced in color by the AMSRCR algorithm.

8. The improved YOLOv8 coal flow visual detection method according to claim 7 is characterized in that: The backbone network is constructed by constructing ten layers of sequentially connected modules: constructing conv modules in the first and second layers, constructing a three-dimensional weighted attention module in the third layer, constructing three groups of identical combination modules in the fourth to ninth layers, each group of combination modules includes a C2f-FasterNet module and a conv module constructed in sequence, and constructing a C2f-FasterNet module in the tenth layer and connecting it to the head network; The processing process of the C2f-FasterNet module is as follows: the input features are sequentially subjected to the Split operation and the multi-layer FasterNet module, and then the input features are concat-connected with the output of each layer of the FasterNet module, and finally output to the next layer; the processing process of the FasterNet module is as follows: the FasterNet module input features are sequentially subjected to the PConv layer with a convolution kernel of 3*3, the conv layer with a convolution kernel of 1*1, the batch normalization operation and the ReLU activation function layer, and then residually connected with the FasterNet module input features to obtain the output and enter the next layer; The processing process of the conv module is as follows: the input features are sequentially output through the conv2d module, batch normalization operation and ReLU activation function layer, and then enter the next layer; The head network includes a detection head and a classification head operating in parallel. The detection head uses convolution and deconvolution operations to generate detection results based on the output feature map of the backbone network, including bounding box regression of each anchor box and the confidence of the target existence. The classification head uses global average pooling, full connection operation and activation function to classify each feature map based on the output feature map of the backbone network, and outputs the probability distribution of each category.

9. The improved YOLOv8 coal flow visual detection method according to claim 8 is characterized in that: The processing process of the three-dimensional weighted attention module is as follows: the input feature map X is calculated by the energy function and then combined with the input feature map X of the three-dimensional weighted attention module to obtain an output; The method for calculating the energy function is: in, and They represent the mean and variance of the neurons xi in the channel except the target neuron t, respectively. i is the index in the spatial dimension, λ is a hyperparameter, and M=H×W represents the total number of neurons in the channel.

10. The improved YOLOv8 coal flow visual detection method according to claim 9 is characterized in that: The method of combining the input feature map X of the three-dimensional weighted attention module to obtain the output is: Among them, E is the combination of energy functions of all spatial and channel dimensions, sigmoid is the activation function, and ⊙ represents element-wise multiplication.

Images