Coal gangue detection method based on YOLOv8s model

Abstract

The invention discloses a coal gangue detection method based on a YOLOv8s model, and belongs to the field of coal gangue screening. In order to solve the problems of low efficiency, low precision and low speed in the existing coal gangue screening, the method comprises the following steps: acquiring a coal gangue target in real time by adopting an industrial camera, transmitting an acquisition result to a trained SKL-YOLOv8s network model, and outputting a coal gangue detection result; the network architecture of the SKL-YOLOv8s model comprises that a backbone network adopts KW dynamic convolution, an LSK module and a C2f module of YOLOv8s, a neck network adopts an SML-Neck structure, a head network adopts a detection head of YOLOv8, and a network loss algorithm adopts an MPDIOU algorithm. The device is used for detecting and screening the coal gangue.

Description

Technical Field

[0001] The invention relates to a coal gangue detection method based on a YOLOv8s model, and belongs to the field of coal gangue screening. Background Art

[0002] Gangue is the main impurity in the coal mining process. It has high density, low heating efficiency, and produces harmful gases when burned. Therefore, the separation of coal and gangue is an indispensable part of clean coal mining. In recent years, the entire coal industry has actively explored high-precision and high-efficiency clean sorting methods, which is also the development requirement of intelligent gangue sorting.

[0003] At present, there are four main ways to sort coal and gangue, including manual sorting, mechanical sorting, radiation sorting and image sorting. At present, manual sorting is the main sorting method in my country, which mainly relies on manual sorting to sort out coal blocks and gangue. Due to the influence of coal quality changes and human factors, the sorting rate of manual sorting is low, and there are also problems such as high labor intensity and poor working environment (including noise and dust). Therefore, it is of great significance to carry out real-time, fast and accurate intelligent detection of coal gangue. However, there is currently no efficient coal gangue recognition algorithm that can replace manual work, resulting in manual sorting of gangue still being the main sorting method. The accuracy and efficiency will directly affect the production efficiency and quality of raw coal, and reduce the overall effect of sorting coal gangue. Summary of the invention

[0004] The purpose of the present invention is to solve the problems of low efficiency, low precision and slow speed in the existing gangue screening, and to provide a gangue detection method based on the YOLOv8s model.

[0005] The coal gangue detection method based on the YOLOv8s model of the present invention comprises:

[0006] S1. Use industrial cameras to collect coal gangue targets;

[0007] S2, enhance and filter the target data collected in S1, use annotation tools to annotate different targets, and obtain a data set;

[0008] S3, divide the data set obtained in S2 into a training set, a validation set and a test set;

[0009] S4. Use the training set to train the SKL-YOLOv8s model to obtain the trained SKL-YOLOv8s network model;

[0010] The network architecture of the SKL-YOLOv8s model includes: the backbone network adopts KW dynamic convolution, LSK module and C2f module of YOLOv8s, the neck network adopts SML-Neck structure, the head network adopts YOLOv8 detection head, and the network loss algorithm adopts MPDIoU algorithm;

[0011] S5, use industrial cameras to collect gangue targets in real time, transmit the collected results to the trained SKL-YOLOv8s network model obtained in S4, and output the gangue detection results.

[0012] Preferably, the specific architecture of the KW dynamic convolution of the backbone network includes:

[0013] The KW dynamic convolution is KernelWarehouse dynamic convolution;

[0014] The number of core units in the warehouse of a backbone network is n, which are shared by one convolutional layer.

[0015] The static convolution kernel in the convolution layer is divided into a linear combination of m kernel units through kernelpartition.

[0016] Set the intermediate variable b = n / mt,

[0017] Among them, mt represents the total amount of kernel units in one layer, and the kernelpartition network model is expanded or reduced by changing the intermediate variable b;

[0018] When b < 1, kernelpartition tends to obtain a reduced model size;

[0019] When b = 1, kernelpartition tends to obtain similar model sizes;

[0020] When b>1, kernelpartition tends to obtain an enlarged model size.

[0021] Preferably, the specific architecture of the LSK module of the backbone network includes:

[0022] The LSK module is a Large Selective Kernel Network module;

[0023] It includes multiple LSKBlocks, each of which includes two residual sub-blocks, namely the large kernel selection sub-block LKS and the feed-forward network sub-block FFN;

[0024] The large kernel selection subblock LKS includes a large kernel convolution sequence and a spatial kernel selection mechanism, which decouples the large convolution kernel into a series of continuously expanding depth-wise convolutions:

[0025] The expansion rate is d, the receptive field is RF, and the following relationship exists:

[0026] k i-1 ≤k i , d1=1,

[0027] d i-1 <d i ≤RF i-1 , RF1=k1,

[0028] RF i-1 =d i (k i -1)+RF i-1 ;

[0029] Among them, k i represents the size of the i-th depth-wise convolution kernel in the large kernel convolution sequence, k i-1 represents the i-1th depth-wise convolution kernel size in the large kernel convolution sequence, k1 represents the first depth-wise convolution kernel size in the large kernel convolution sequence,

[0030] d i represents the i-th dilation rate in the large kernel convolution sequence, d i-1 represents the i-1th dilation rate in the large kernel convolution sequence, d1 represents the first dilation rate in the large kernel convolution sequence,

[0031] RF i-1 represents the i-1th receptive field in the large kernel convolution sequence, and RF1 represents the first receptive field in the large kernel convolution sequence;

[0032] Input feature map vector X, after decoupling convolution kernel:

[0033] U0=X,

[0034]

[0035] in, Indicates that the convolution kernel is k i , the expansion rate is d i The depth-wise convolution operation, U0 represents the original features of the feature map vector, U i+1 Represents the i+1th generation feature of the feature map vector;

[0036] Each convolution operation is followed by a 1×1 convolution layer F. 1×1(·) Perform channel fusion of spatial feature vectors to obtain the fusion result:

[0037] for iin[1,N];

[0038] in, Represents the characteristics of the i-th convolution kernel;

[0039] Concatenate features from multiple convolution kernels to obtain concatenated results

[0040]

[0041] Average pooling and maximum pooling are performed on the channels to extract spatial relationships and obtain spatial features:

[0042]

[0043] Among them, SA avg Represents the spatial features obtained by average pooling, SA max represents the spatial features obtained by maximum pooling, Express Perform average pooling operation. Express Perform maximum pooling operation;

[0044] The spatial pooling features are concatenated using convolution, and the pooling features of the two channels are converted into M spatial attention feature maps:

[0045]

[0046] in, represents the spatial attention feature map, F 2→M Represents a function that transforms the pooled features of a channel into M feature values;

[0047] After each spatial attention feature map passes through the Sigmoid activation function, the spatial selection mask corresponding to each decoupled large kernel is obtained:

[0048]

[0049] express The activation value of

[0050] The features of the decoupling kernel are weighted with the corresponding mask and fused through convolution F(·) to obtain the attention feature S:

[0051]

[0052] Finally, the input X is multiplied by the attention feature S and the output is:

[0053] Y = X·S;

[0054] Among them, Y represents the output result.

[0055] Preferably, the specific architecture of the SML-Neck structure of the neck network includes:

[0056] The second input data INPUT2 is passed to the upsampling module Upsample, the output of the upsampling module Upsample and the first input data INPUT1 are combined by the first Concat module, the output of the first Concat module is passed to the first C2f-KW module, the output of the first C2f-KW module is convolved by the KW dynamic convolution module Kwconv, and then combined with the third input data INPUT3 by the second Concat module, the output of the second Concat module is passed to the second C2f-KW module, the output of the first C2f-KW module is used as the first output result OUTPUT1, and the output of the second C2f-KW module is used as the second output result OUTPUT2; the first output result OUTPUT1 and the second output result OUTPUT2 are passed to the detection head;

[0057] The first output result OUTPUT1 is a 40*40 feature vector, and the second output result OUTPUT2 is a 20*20 feature vector.

[0058] Preferably, the C2f-KW module specifically comprises: using a KW dynamic convolutional layer to replace the convolutional layer in the basic module C2f module of the YOLOv8s network model.

[0059] Preferably, the first input data INPUT1 is the sixth layer of the backbone network, the second input data INPUT2 is the tenth layer of the backbone network, and the third input data INPUT3 is the ninth layer of the backbone network;

[0060] The backbone network starts from layer zero.

[0061] Preferably, the MPDIoU of the head network specifically includes:

[0062] The loss of MPDIoU is:

[0063]

[0064] L IoU = 1-IoU;

[0065] L MPDIoU =1-MPDIoU;

[0066] Among them, IoU represents the intersection over union ratio, MPDIoU represents the modified intersection over union ratio under the MPDIoU algorithm, d1 represents the distance between the upper left corner of the predicted box and the upper left corner of the real box, d2 represents the distance between the upper right corner of the predicted box and the upper right corner of the real box, h represents the height of the rectangular field of view of the industrial camera, w represents the length of the rectangular field of view of the industrial camera, L IoU Represents the loss value under the IoU algorithm, L MPDIoU Represents the loss value under the MPDIoU algorithm.

[0067] Preferably, the gangue targets described in S1 include gangue targets on the belt conveyor and gangue targets in the underground environment.

[0068] Preferably, the enhancement of the target data in S2 includes: rotation, random cropping, mapping transformation, Gaussian noise and color transformation.

[0069] Preferably, the data set division ratio described in S3 is 8:1:1, and the data set is divided into a training set, a validation set and a test set in a ratio of 8:1:1.

[0070] Advantages of the present invention: The coal gangue detection method based on the YOLOv8s model proposed in the present invention proposes a lightweight, real-time and efficient detection method for the coal gangue target detection task. The backbone network adopts KernelWarehouse, LSK module and C2f module of YOLOv8, and the neck network designs the SML-Neck structure. The detection head of MPDIoU and YOLOv8 is used. The accuracy and speed in the field of coal gangue target detection are higher than those of mainstream deep learning networks such as YOLOv5s, YOLOv5x, YOLOv7, YOLOv7x, SSD, Faster-RCNN, RT-DETR and YOLOv8s. The average number of missed recognitions of YOLOv8s and SKL-YOLOv8s is 0; the average number of missed transmissions is 2.6 and 0.6 respectively. The SKL-YOLOv8s method has a smaller amount of calculation, which is conducive to saving computing power and enables the host to process target information more efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 It is a method flow chart of the coal gangue detection method based on the YOLOv8s model described in the present invention;

[0072] Figure 2 It is a network architecture diagram of the SKL-YOLOv8s model described in the present invention;

[0073] Figure 3 It is a specific architecture diagram of the KW dynamic convolution described in the present invention;

[0074] Figure 4It is a specific architecture diagram of the LSK module of the present invention;

[0075] Figure 5 It is a specific architecture diagram of the SML-Neck structure of the present invention;

[0076] Figure 6 It is a schematic diagram of the MPDIoU of the present invention;

[0077] Figure 7 Schematic diagram of the MPDIoU detection frame of the present invention;

[0078] Figure 8 It is the accuracy curve of various network validation sets, where the horizontal axis represents the number of training iterations and the vertical axis represents the accuracy;

[0079] Fig. 9 It is a performance curve of various network verification sets, where the horizontal axis represents the frame rate and the vertical axis represents the accuracy;

[0080] Fig.10 It is a schematic diagram of the dataset image;

[0081] Fig.11 It is a comparison chart of the training process loss, where the horizontal axis represents the number of training iterations and the vertical axis represents the training loss value. DETAILED DESCRIPTION

[0082] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0083] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0084] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0085] Embodiment 1:

[0086] Combine the following Figure 1-Figure 6 The present embodiment is described. The coal gangue detection method based on the YOLOv8s model described in the present embodiment includes:

[0087] S1. Use industrial cameras to collect coal gangue targets;

[0088] S2, enhance and filter the target data collected in S1, use annotation tools to annotate different targets, and obtain a data set;

[0089] S3, divide the data set obtained in S2 into a training set, a validation set and a test set;

[0090] S4. Use the training set to train the SKL-YOLOv8s model to obtain the trained SKL-YOLOv8s network model;

[0091] The network architecture of the SKL-YOLOv8s model includes: the backbone network adopts KW dynamic convolution, LSK module and C2f module of YOLOv8s, the neck network adopts SML-Neck structure, the head network adopts YOLOv8 detection head, and the network loss algorithm adopts MPDIoU algorithm;

[0092] S5, use industrial cameras to collect gangue targets in real time, transmit the collected results to the trained SKL-YOLOv8s network model obtained in S4, and output the gangue detection results.

[0093] Furthermore, the specific architecture of the KW dynamic convolution of the backbone network includes:

[0094] The KW dynamic convolution is KernelWarehouse dynamic convolution;

[0095] The number of core units in the warehouse of a backbone network is n, which are shared by one convolutional layer.

[0096] The static convolution kernel in the convolution layer is divided into a linear combination of m kernel units through kernelpartition.

[0097] Set the intermediate variable b = n / mt, where mt represents the total number of kernel units in a layer. The kernelpartition network model can be expanded or reduced by changing the intermediate variable b.

[0098] When b < 1, kernelpartition tends to obtain a reduced model size;

[0099] When b = 1, kernelpartition tends to obtain similar model sizes;

[0100] When b>1, kernelpartition tends to obtain an enlarged model size.

[0101] Furthermore, the specific architecture of the LSK module of the backbone network includes:

[0102] The LSK module is a Large Selective Kernel Network module;

[0103] It includes multiple LSKBlocks, each of which includes two residual sub-blocks, namely the large kernel selection sub-block LKS and the feed-forward network sub-block FFN;

[0104] The large kernel selection subblock LKS includes a large kernel convolution sequence and a spatial kernel selection mechanism, which decouples the large convolution kernel into a series of continuously expanding depth-wise convolutions:

[0105] The expansion rate is d, the receptive field is RF, and the following relationship exists:

[0106] k i-1 ≤k i , d1=1,d i-1 <d i ≤RF i-1 , RF1=k1,RF i-1 =d i (k i -1)+RF i-1 ;

[0107] Among them, k i represents the size of the i-th depth-wise convolution kernel in the large kernel convolution sequence, k i-1 represents the i-1th depth-wise convolution kernel size in the large kernel convolution sequence, k1 represents the first depth-wise convolution kernel size in the large kernel convolution sequence, d i represents the i-th dilation rate in the large kernel convolution sequence, d i-1 represents the i-1th dilation rate in the large kernel convolution sequence, d1 represents the first dilation rate in the large kernel convolution sequence, RF i-1 represents the i-1th receptive field in the large kernel convolution sequence, and RF1 represents the first receptive field in the large kernel convolution sequence;

[0108] Input feature map vector X, after decoupling convolution kernel:

[0109]

[0110] in, Indicates that the convolution kernel is k i , the expansion rate is d i The depth-wise convolution operation, U0 represents the original features of the feature map vector, U i+1 Represents the i+1th generation feature of the feature map vector;

[0111] Each convolution operation is followed by a 1×1 convolution layer F. 1×1 (·) Perform channel fusion of spatial feature vectors to obtain the fusion result:

[0112]

[0113] in, Represents the characteristics of the i-th convolution kernel;

[0114] Concatenate features from multiple convolution kernels to obtain concatenated results

[0115]

[0116] Average pooling and maximum pooling are performed on the channels to extract spatial relationships and obtain spatial features:

[0117]

[0118] Among them, SA avg Represents the spatial features obtained by average pooling, SA max represents the spatial features obtained by maximum pooling, Express Perform average pooling operation. Express Perform maximum pooling operation;

[0119] The spatial pooling features are concatenated using convolution, and the pooling features of the two channels are converted into M spatial attention feature maps:

[0120]

[0121] in, represents the spatial attention feature map, F 2→M Represents a function that transforms the pooled features of a channel into M feature values;

[0122] After each spatial attention feature map passes through the Sigmoid activation function, the spatial selection mask corresponding to each decoupled large kernel is obtained:

[0123]

[0124] express The activation value of

[0125] The features of the decoupling kernel are weighted with the corresponding mask and fused through convolution F(·) to obtain the attention feature S:

[0126]

[0127] Finally, the input X is multiplied by the attention feature S and the output is:

[0128] Y = X·S;

[0129] Among them, Y represents the output result.

[0130] Furthermore, the specific architecture of the SML-Neck structure of the neck network includes:

[0131] The second input data INPUT2 is passed to the upsampling module Upsample, the output of the upsampling module Upsample and the first input data INPUT1 are combined by the first Concat module, the output of the first Concat module is passed to the first C2f-KW module, the output of the first C2f-KW module is convolved by the KW dynamic convolution module Kwconv, and then combined with the third input data INPUT3 by the second Concat module, the output of the second Concat module is passed to the second C2f-KW module, the output of the first C2f-KW module is used as the first output result OUTPUT1, and the output of the second C2f-KW module is used as the second output result OUTPUT2; the first output result OUTPUT1 and the second output result OUTPUT2 are passed to the detection head;

[0132] The first output result OUTPUT1 is a 40*40 feature vector, and the second output result OUTPUT2 is a 20*20 feature vector.

[0133] Furthermore, the C2f-KW module specifically replaces the convolution layer in the basic module C2f module of the YOLOv8s network model with the KW dynamic convolution layer.

[0134] Furthermore, the first input data INPUT1 is the sixth layer of the backbone network, the second input data INPUT2 is the tenth layer of the backbone network, and the third input data INPUT3 is the ninth layer of the backbone network;

[0135] The backbone network starts from layer zero.

[0136] Furthermore, the MPDIoU of the head network specifically includes:

[0137] The loss of MPDIoU is:

[0138]

[0139] L IoU = 1-IoU;

[0140] L MPDIoU =1-MPDIoU;

[0141] Among them, IoU represents the intersection over union ratio, MPDIoU represents the modified intersection over union ratio under the MPDIoU algorithm, d1 represents the distance between the upper left corner of the predicted box and the upper left corner of the real box, d2 represents the distance between the upper right corner of the predicted box and the upper right corner of the real box, h represents the height of the rectangular field of view of the industrial camera, w represents the length of the rectangular field of view of the industrial camera, L IoU Represents the loss value under the IoU algorithm, L MPDIoU Represents the loss value under the MPDIoU algorithm.

[0142] Furthermore, the gangue targets described in S1 include the gangue targets on the belt conveyor and the gangue targets in the underground environment.

[0143] Furthermore, the target data enhancement in S2 includes: rotation, random cropping, mapping transformation, Gaussian noise and color transformation.

[0144] Furthermore, the data set described in S3 is divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0145] In this paper, a lightweight, real-time and efficient detection method is proposed for the gangue target detection task. It is implemented based on the SKL-YOLOv8s module. The backbone network adopts KernelWarehouse, LSK module and C2f module of YOLOv8, and the SML-Neck structure is designed for the neck network. The detection head of MPDIoU and YOLOv8 is used.

[0146] KernelWarehouse (KW) can reduce network computation:

[0147] In order to improve the reasoning speed of the network model, a large number of KW dynamic convolutions are used in the backbone network to save the computational workload of the network model. KernelWarehouse proposes kernel partition and warehouse sharing to reduce the convolution kernel dimension and significantly increase the number of convolution kernels, effectively reducing the computational workload.

[0148] like Figure 3 As shown in the figure, a warehouse containing n kernel units is defined. The static convolution kernel in the convolution layer is divided into a linear combination of m kernel units through kernel partition. The linear combination in the dynamic convolution can be expressed as the combination of kernel units in the warehouse. Kernel partition and warehouse sharing reduce the number of kernel unit parameters, so the number of convolution kernels n can be larger under the same parameter budget.

[0149] Let n be the number of kernel units in the warehouse of a network, shared to l convolutional layers, and mt be the total number of kernel units in these l layers (kernel partition sets m=1, warehouse sharing sets l=1, and KernelWarehouse degenerates into ordinary dynamic convolution). Taking b=n / mt, it can be seen that KernelWarehouse can expand or reduce the network model by changing b. When b<1, KernelWarehouse tends to obtain a reduced model size; when b=1, KernelWarehouse tends to obtain a similar model size; when b>1, KernelWarehouse tends to obtain an increased model size. This model sets the scaling factor to 1. By referring to KW convolution, the computational complexity of the network model can be significantly saved and the reasoning speed of the network model can be improved.

[0150] LSKBlock can improve the accuracy of network models:

[0151] Many gangue targets are difficult to identify based on their appearance, and many successful identifications rely on the shape and other features provided by the background. The Large Selective Kernel Network (LSKNet) method uses a spatial selection mechanism to weight the features processed by a series of large kernel depth-wise convolutions, and dynamically adjusts the receptive field of feature extraction by adaptively using different large kernels, effectively capturing the information difference between the gangue target and the belt background.

[0152] LSKNet is composed of multiple LSKBlocks. Figure 2 As shown in Figure 2, each LSKBlock consists of two residual sub-blocks, namely, a large kernel selection sub-block (Large Kernel Selection, LK Selection) and a feed-forward network sub-block (Feed-forward Network, FFN).

[0153] In the LK Selection sub-block of LSKBlock, the LSK module consists of a large kernel convolution sequence and a spatial kernel selection mechanism. In order to obtain rich context features in different ranges from the input X, the large convolution kernel is decoupled into a series of continuously expanding depth-wise convolutions. Assume that the size of the i-th depth-wise convolution kernel in the sequence is k, the expansion rate is d, and the receptive field is RF, which satisfies the following relationship:

[0154] ki-1 ≤k i ; d1=1,d i-1 <d i ≤RF i-1

[0155] RF1=k1,RF i =d i (k i -1)+RF i-1

[0156] Large convolution kernels and dilation rates ensure that the receptive field can be quickly increased, and features of receptive fields of different sizes are generated in steps to facilitate subsequent selection. At the same time, large kernel decoupling can effectively reduce the number of parameters.

[0157] The input X passes through the decoupled convolution kernel:

[0158]

[0159] in, The convolution kernel is k i , the expansion rate is d i Depth-wise convolution operation. Assume that there are N decoupled convolution kernels, and each convolution operation is followed by a 1×1 convolution layer F. 1×1 (·) Perform channel fusion of spatial feature vectors:

[0160] for iin[1,N]

[0161] Concatenate features from multiple convolution kernels:

[0162]

[0163] Average pooling and maximum pooling are performed on the channels to extract spatial relationships and obtain spatial features:

[0164]

[0165] Use convolution to concatenate the spatial pooling features and convert the pooling features of 2 channels into N spatial attention feature maps:

[0166]

[0167] Each spatial attention feature map is activated by the Sigmoid function to obtain the spatial selection mask corresponding to each decoupled large kernel:

[0168]

[0169] The features of the decoupling kernel are weighted with the corresponding mask and fused through convolution F(·) to obtain the attention feature S:

[0170]

[0171] Finally, the output is obtained by multiplying the input X with the attention feature S:

[0172] Y=X·S

[0173] By dynamically adjusting the receptive field of the network, LSK can effectively capture the difference between the gangue target and the belt background and improve the recognition accuracy.

[0174] Design of Simplified-Medium-Large-Neck Structure:

[0175] As the number of Backbone layers increases, the input image becomes a smaller and smaller feature map. Large-sized feature maps can retain more details; small-sized feature maps are more suitable for detecting large targets. Feature maps of different sizes contain different feature dimensions, so it is necessary to fuse features of different sizes in the Neck part, and finally use the fused large, medium, and small sizes as output for detection.

[0176] In the gangue target detection task, gangue belongs to medium-large targets, so the small target detection head has little effect on the gangue detection task and brings a large number of parameters, which increases the computational complexity of the network. Therefore, for the gangue target detection task, the small target detection head is not used, and a simplified Neck structure is designed. The SML-Neck (Simplified-Medium-Large-Neck) structure is proposed. Figure 5 As shown in Figure 2, the SML-Neck structure pays more attention to the features of medium and large targets, simplifies the feature fusion process, and saves a lot of parameters for the model.

[0177] Using MPDIoU to improve training results:

[0178] During the training process, the traditional loss calculation method does not meet the requirements of the gangue sorting task: whether the predicted frame is K times smaller or larger than the real frame, the loss value is the same. However, in the work of grabbing and removing gangue by the sorting mechanism, the opening and closing degree of the mechanical claw must be greater than or equal to the target width. This requires that when the network is trained, the loss of small detection frames that cannot cover the target should be greater, and the loss of large detection frames that can completely cover the target should be smaller.

[0179] like Figure 6 As shown in Figure 2, MPDIoU can better train the regression of the predicted box by minimizing the upper left and lower right corner distances between the predicted box and the true box.

[0180] Let d1 represent the distance between the upper left corner of the predicted box and the upper left corner of the real box, and d2 represent the distance between the upper right corner of the predicted box and the upper right corner of the real box. Then the MPDIoU loss is:

[0181]

[0182] L IoU =1-IoU

[0183] L MPDIoU =1-MPDIoU

[0184] like Figure 7 As shown in the figure, if the center points of the two predicted boxes (red) are accurate and the aspect ratios of the two boxes are the same, the length and width of the large box is K times that of the real box, and the length and width of the small box is 1 / K times that of the real box. The calculation idea of ​​MPDIoU is more in line with the actual requirements of coal gangue sorting and can more effectively guide the regression loss of the network.

[0185] The image acquisition experimental platform recognizes the target in real time, processes the target information and transmits it to the host computer. The delay in processing and sending information will change due to fluctuations in computing power. In the case of large delays, it will affect the processing and sending of the next frame, resulting in target omissions. The belt speed is 0.3m / s. For every 100 dense and continuous coal gangue targets, the average number of missed recognitions for YOLOv8s and SKL-YOLOv8s is 0; the average number of missed transmissions is 2.6 and 0.6, respectively. The SKL-YOLOv8s method has a smaller amount of calculation, which is conducive to saving computing power and enabling the host to process target information more efficiently.

[0186] like Figure 8 As shown, it is the accuracy curve of various network verification sets, such as Fig. 9 As shown in the figure, it is the performance curve of various network verification sets. The accuracy and speed of the SKL-YOLOv8s method in the field of coal gangue target detection are higher than those of mainstream deep learning networks such as YOLOv5s, YOLOv5x, YOLOv7, YOLOv7x, SSD, Faster-RCNN, RT-DETR and YOLOv8s.

[0187] First, build the dataset:

[0188] An industrial camera was used to collect the gangue targets on the belt conveyor and the gangue targets in the underground environment, and the images were screened. In order to improve the recognition accuracy of the model, a variety of data enhancement methods were used, including rotation, random cropping, mapping transformation, Gaussian noise and color transformation. After data enhancement, 14790 gangue images were obtained. Fig.10As shown in the figure, the prefixes c, cg and multi represent coal, gangue and multi-target respectively. The dataset was annotated using the LabelImg annotation tool. The dataset is divided into training set, validation set and test set in a ratio of 8:1:1. The loss comparison of the training process is shown in the figure below. Fig.11 shown.

[0189] The hardware and software environment and parameters are shown in Table 1:

[0190] Table 1

[0191] name information operating system Ubuntu 20.04 Deep Learning Frameworks Pytorch 2.0.1 processor Intel(R)Xeon(R)Gold6226R*2 Memory 196Gb Graphics NVIDIA Tesla T4*4 Operating Environment Anaconda3 Python Version 3.9 Sorting host system Windows 10 Sorting host processor i7-13700K

[0192] In order to further verify the reasoning performance of the network, Fig. 9 As shown in the figure, the deep learning networks YOLOv5s, YOLOv5x, YOLOv7, YOLOv7x, SSD, Faster-RCNN and RT-DETR were trained respectively, and the validation set accuracy curves obtained during the training of each network were plotted. The experimental parameters were uniformly set to no pre-trained weights, Batch Size was 16, epoch was 60, confidence threshold was 0.25, and IoU threshold was 0.7. The validation set mAP@0.5:0.95 obtained by YOLOv8s and the improved network were 90.3% and 90.9% respectively. The improved network had a slight improvement in accuracy over the original network. The indicators obtained by each network using the test set are compared as shown in Table 2. Compared with the original network, SKL-YOLOv8s has a slight improvement in accuracy and a significant optimization in speed.

[0193] Table 2

[0194] Model F1(％) Recall(%) Fps mAP@0.5:0.95(%) YOLOv5s 98.8 98.8 83.3 84.2 YOLOv5x 99.2 99.3 22.3 86.7 YOLOv7 98.8 99.1 28 84.5 YOLOv7x 99.1 99.3 21.1 85.1 SSD 99.2 99 58.6 84.2 Faster-RCNN 98 99.4 7.9 82.4 YOLOv8s 99.3 99.4 80.6 91.7 RT-DETR-l 99.6 99.6 31.1 88.6 RT-DETR-x 99 99.1 14.7 88.7 SKL-YOLOv8s-MPD 99.5 99.6 120.5 91.9

[0195] In order to explore the effect of improvements in various parts of the network, an ablation experiment was conducted on the method of this application, and the indicators obtained using the test set are shown in Table 3.

[0196] Table 3

[0197] SML-Neck KWConv LSKBlock MPD F1(％) GFLOPs FPS mAP@0.5:0.95(%) 99.3 28.4 80.6 91.7 √ 99.5 22.4 84.7 91.7 √ √ 99.5 10.7 122 91.5 √ √ 99.3 22.8 83.3 91.8 √ √ 99.5 22.4 84.7 91.8 √ √ √ 99.3 14.7 111.1 91.7 √ √ √ √ 99.5 11.1 120.5 91.9

[0198] In the gangue target detection task, the SML-Neck architecture can improve the network's reasoning speed while maintaining high accuracy. The ablation experiment is mainly carried out on its basis: KW's dynamic convolution kernel reduces the amount of calculation by sharing parameters, and combined with the SML-Neck structure, it improves the network's reasoning speed; the LSK module and MPDIoU improve the model's reasoning accuracy for gangue detection.

[0199] In the present invention, the model's lightweight design and optimization effectively reduce the model's parameter quantity and computational complexity, thereby reducing the model's overall resource consumption and improving the detection speed. Secondly, while maintaining lightweight, the model can still maintain a high level of accuracy, that is, it achieves satisfactory results in various evaluation indicators.

[0200] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in other described embodiments.

Claims

1. A coal gangue detection method based on the YOLOv8s model, characterized in that: It includes: S1. Use industrial cameras to collect coal gangue targets; S2, enhance and filter the target data collected in S1, use annotation tools to annotate different targets, and obtain a data set; S3, divide the data set obtained in S2 into a training set, a validation set and a test set; S4. Use the training set to train the SKL-YOLOv8s model to obtain the trained SKL-YOLOv8s network model; The network architecture of the SKL-YOLOv8s model includes: the backbone network adopts KW dynamic convolution, LSK module and C2f module of YOLOv8s, the neck network adopts SML-Neck structure, the head network adopts YOLOv8 detection head, and the network loss algorithm adopts MPDIoU algorithm; S5, use industrial cameras to collect gangue targets in real time, transmit the collected results to the trained SKL-YOLOv8s network model obtained in S4, and output the gangue detection results.

2. The coal gangue detection method based on the YOLOv8s model according to claim 1, characterized in that: The specific architecture of the KW dynamic convolution of the backbone network includes: The KW dynamic convolution is KernelWarehouse dynamic convolution; The number of core units in the warehouse of a backbone network is n, which are shared by one convolutional layer. The static convolution kernel in the convolution layer is divided into a linear combination of m kernel units through kernelpartition. Set the intermediate variable b = n / mt, Among them, mt represents the total amount of kernel units in one layer, and the kernelpartition network model is expanded or reduced by changing the intermediate variable b; When b < 1, kernelpartition tends to obtain a reduced model size; When b=1, kernelpartition tends to obtain similar model sizes; When b>1, kernelpartition tends to obtain an enlarged model size.

3. The coal gangue detection method based on the YOLOv8s model according to claim 1, characterized in that: The specific architecture of the LSK module of the backbone network includes: The LSK module is a Large Selective Kernel Network module; It includes multiple LSKBlocks, each of which includes two residual sub-blocks, namely the large kernel selection sub-block LKS and the feed-forward network sub-block FFN; The large kernel selection subblock LKS includes a large kernel convolution sequence and a spatial kernel selection mechanism, which decouples the large convolution kernel into a series of continuously expanding depth-wise convolutions: The expansion rate is d, the receptive field is RF, and the following relationship exists: to i-1 ≤k i ,d1=1, d i-1 <d i ≤RF i-1 ,RF1=k1, RF i-1 =d i (k i -1)+RF i-1 ; Among them, k i represents the size of the i-th depth-wise convolution kernel in the large kernel convolution sequence, k i-1 represents the i-1th depth-wise convolution kernel size in the large kernel convolution sequence, k1 represents the first depth-wise convolution kernel size in the large kernel convolution sequence, d i represents the i-th dilation rate in the large kernel convolution sequence, d i-1 represents the i-1th dilation rate in the large kernel convolution sequence, d1 represents the first dilation rate in the large kernel convolution sequence, RF i-1 represents the i-1th receptive field in the large kernel convolution sequence, and RF1 represents the first receptive field in the large kernel convolution sequence; Input feature map vector X, after decoupling convolution kernel: U0=X, in, Indicates that the convolution kernel is k i , the expansion rate is d i The depth-wise convolution operation, U0 represents the original features of the feature map vector, U i+1 Represents the i+1th generation feature of the feature map vector; Each convolution operation is followed by a 1×1 convolution layer F. 1×1 (·) Perform channel fusion of spatial feature vectors to obtain the fusion result: in, Represents the characteristics of the i-th convolution kernel; Concatenate features from multiple convolution kernels to obtain concatenated results Average pooling and maximum pooling are performed on the channels to extract spatial relationships and obtain spatial features: Among them, SA avg Represents the spatial features obtained by average pooling, SA max represents the spatial features obtained by maximum pooling, Express Perform average pooling operation. Express Perform maximum pooling operation; The spatial pooling features are concatenated using convolution, and the pooling features of the two channels are converted into M spatial attention feature maps: in, represents the spatial attention feature map, F 2→M Represents a function that transforms the pooled features of a channel into M feature values; After each spatial attention feature map passes through the Sigmoid activation function, the spatial selection mask corresponding to each decoupled large kernel is obtained: express The activation value of The features of the decoupling kernel are weighted with the corresponding mask and fused through convolution F(·) to obtain the attention feature S: Finally, the input X is multiplied by the attention feature S and the output is: Y = X·S; Among them, Y represents the output result.

4. The coal gangue detection method based on the YOLOv8s model according to claim 1, characterized in that: The specific architecture of the SML-Neck structure of the neck network includes: The second input data INPUT2 is passed to the upsampling module Upsample, the output of the upsampling module Upsample and the first input data INPUT1 are combined by the first Concat module, the output of the first Concat module is passed to the first C2f-KW module, the output of the first C2f-KW module is convolved by the KW dynamic convolution module Kwconv, and then combined with the third input data INPUT3 by the second Concat module, the output of the second Concat module is passed to the second C2f-KW module, the output of the first C2f-KW module is used as the first output result OUTPUT1, and the output of the second C2f-KW module is used as the second output result OUTPUT2; the first output result OUTPUT1 and the second output result OUTPUT2 are passed to the detection head; The first output result OUTPUT1 is a 40*40 feature vector, and the second output result OUTPUT2 is a 20*20 feature vector.

5. The coal gangue detection method based on the YOLOv8s model according to claim 4, characterized in that: The C2f-KW module specifically uses a KW dynamic convolutional layer to replace the convolutional layer in the C2f module, which is the basic module of the YOLOv8s network model.

6. The coal gangue detection method based on the YOLOv8s model according to claim 4, characterized in that: The first input data INPUT1 is the sixth layer of the backbone network, the second input data INPUT2 is the tenth layer of the backbone network, and the third input data INPUT3 is the ninth layer of the backbone network; The backbone network starts from layer zero.

7. The coal gangue detection method based on the YOLOv8s model according to claim 1, characterized in that: The MPDIoU of the head network specifically includes: The loss of MPDIoU is: L IoU =1-IoU; L MPDIoU =1-MPDIoU; Among them, IoU represents the intersection over union ratio, MPDIoU represents the modified intersection over union ratio under the MPDIoU algorithm, d1 represents the distance between the upper left corner of the predicted box and the upper left corner of the real box, d2 represents the distance between the upper right corner of the predicted box and the upper right corner of the real box, h represents the height of the rectangular field of view of the industrial camera, w represents the length of the rectangular field of view of the industrial camera, L IoU Represents the loss value under the IoU algorithm, L MPDIoU Represents the loss value under the MPDIoU algorithm.

8. The coal gangue detection method based on the YOLOv8s model according to claim 1, characterized in that: The gangue targets described in S1 include the gangue targets on the belt conveyor and the gangue targets in the underground environment.

9. The coal gangue detection method based on the YOLOv8s model according to claim 1, characterized in that: The target data enhancement described in S2 includes: rotation, random cropping, mapping transformation, Gaussian noise and color transformation.

10. The coal gangue detection method based on the YOLOv8s model according to claim 1, characterized in that: The data set described in S3 is divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

Images