Coal gangue multi-scale detection method and system based on YOLOv8 network

Abstract

The invention provides a coal gangue multi-scale detection method and system based on a YOLOv8 network, and the method comprises the steps: enabling a CBAM attention mechanism to be embedded in a whole network core module C2f, and improving the small target detection capability; secondly, a fast spatial pyramid pooling module is adopted in the backbone network, more front and back features are fused, interaction of feature information is enhanced, and the multi-scale detection capability is further improved; then, through a bidirectional feature fusion module with more jump connections, the multi-scale detection capability is increased, and small target features are prevented from being lost due to network depth; and finally, convolution blocks used for feature fusion in the detection head are combined for weight reduction, regression loss is set as EIoULoss, and errors caused by the fact that the length-width ratio of a prediction frame is consistent with that of a real frame are avoided. By means of the target network model, coal gangue with the size fraction ranging from 6 mm to 50 mm can be effectively detected, cross-size-fraction detection is achieved, and the target network model can be effectively deployed in coal gangue separation equipment.

Description

Technical Field

[0001] The present invention belongs to the technical field of coal gangue detection, and in particular relates to a coal gangue multi-scale detection method and system based on a YOLOv8 network. Background Art

[0002] The mining process of coal is very complicated. It is usually found in the rock layer under the stratified rock, which is often mixed with a large amount of gangue. The mixing of gangue will reduce the combustion efficiency of coal and seriously pollute the environment. Therefore, separating gangue from the mined ore is of great significance to environmental protection and the combustion efficiency of coal. Common methods for sorting gangue are manual gangue picking, wet sorting and dry sorting. Among them, the manual gangue picking method is inefficient and unstable, and the wet sorting method has problems such as wasting water resources and polluting the environment. Therefore, the current mainstream method is the dry sorting method. In the dry sorting method, the hardness recognition method has large restrictions on the use of scenarios, the ray recognition method is dangerous, and the photoelectric recognition process is complex and has high requirements. The target detection method has high recognition efficiency and low cost, and has broad application prospects.

[0003] The target detection method includes two steps: detection and classification. Among them, the traditional target detection method is the threshold method. This method sets the threshold by the pixel value difference between the detection part and the background part of the picture to segment the target to be detected. This method can accurately detect whether the target to be detected is the target or the background from the picture in coal gangue detection, but for overlapping targets, it can only detect them as one and cannot correctly detect the number. In actual application scenarios, such as in carbon gold equipment, when the coal processing capacity is less than 200T / h, the coal overlap is not high and this method can be used, but when it is higher than 200T / h, more targets cannot be detected due to the high overlap rate and this method cannot be used. The current mainstream target detection method is to use deep learning methods, which are mainly divided into two categories: two-stage target detection algorithm; one-stage target detection algorithm.

[0004] Two-stage target detection algorithms include R-CNN, Fast R-CNN, Faster R-CNN, Mask R-CNN, etc.; one-stage target detection algorithms are mainly SSD and YOLO series. Although using deep learning methods for detection can solve the problem of overlapping target detection, in the case of multi-scale detection, if the size of the detected target differs too much, the small target features may be lost in a certain network layer, and eventually the small target cannot be recognized; by splicing pictures of various large-size coal and small-size coal and using various mainstream deep learning methods to train the model and then actually testing, it is found that when the size of the detected target differs by more than a certain multiple, small targets below a certain size may not be recognized. Summary of the invention

[0005] Based on this, the embodiments of the present invention provide a multi-scale detection method and system for coal gangue based on the YOLOv8 network, aiming to improve the YOLOv8 network model by drawing on the small target detection method, so that the improved model can optimize the detection problem of small targets in multi-scale detection while maintaining the recognition speed, and realize accurate detection of coal gangue.

[0006] A first aspect of an embodiment of the present invention provides a multi-scale detection method for coal gangue based on a YOLOv8 network, which is used to detect coal gangue with a particle size of 6 mm to 50 mm. The method includes: Acquire mixed images of coal and coal gangue, and annotate the mixed images of coal and coal gangue to obtain a data set; Input the data set into the improved YOLOv8 network model for training to obtain a target network model; Inputting the image to be detected into the target network model to determine the coal and coal gangue in the image to be detected; The improved YOLOv8 network model includes a backbone network consisting of 5 CBS modules, 4 CBC2f modules and 1 fast spatial pyramid pooling module, a bidirectional feature fusion module consisting of 2 CBS modules and 4 CBC2f modules in the neck, and finally a detection structure consisting of 3 detection heads of different sizes in the head. Among them, after a segmentation operation is performed on the input stream in the CBC2f module, the original input stream is divided into two equal parts, one of which is sent to the remaining CBS module in the CBC2f module, and the other part focuses on important features through the convolutional block attention module. Finally, the feature maps extracted by the three convolutional blocks are spliced ​​with the feature maps generated by the convolutional block attention module and finally output. The convolutional block attention module adopts an integrated channel and spatial attention mechanism; After the fast spatial pyramid pooling module performs convolution processing on the feature map output by the previous layer, the obtained feature map is subjected to maximum pooling through three pooling layers of 5×5, 9×9, and 13×13, and is concatenated with the feature maps output by the three pooling layers on the channel. At the same time, an additional convolution layer is set to extract features and the final features are jump-connected with the input features; The bidirectional feature fusion module increases the detection capability by setting jump connections between feature pyramids; The detection head includes two CBS modules connected in sequence, and convolution blocks connected to the two CBS modules respectively.

[0007] Furthermore, the convolutional block attention module is used to perform average pooling and maximum pooling on the input feature map in the spatial dimension to obtain two first feature maps, which respectively represent the average value and maximum value of the channel; The two first feature maps are input into the two-layer fully connected network respectively to generate two channel attention vectors and add them together, and then normalize them through the sigmoid function to generate the final channel attention weight; Multiply the original feature map by the channel attention weight element by element to obtain a weighted feature map; The weighted feature map is passed into the spatial attention unit, and average pooling and maximum pooling are performed on the channel dimension to obtain two second feature maps; The two second feature maps are concatenated in the channel dimension to obtain a dual-channel feature map, and then a 7×7 convolution kernel is used to perform a convolution operation to obtain a spatial attention map; The original feature map is multiplied element-by-element by the spatial attention map to obtain a weighted target feature map.

[0008] Furthermore, the convolution blocks for feature fusion in the detection head that perform regression loss and classification loss are merged together while the convolution for conversion to output remains unchanged, so as to achieve lightweighting.

[0009] Furthermore, in the step of performing average pooling and maximum pooling on the input feature map in the spatial dimension to obtain two first feature maps, the calculation formula is:

[0010] in, F avg and F max They are the pooling results after channel average pooling and maximum pooling, respectively. is the position in the cth channel The characteristic description of is the scale range of the original feature map, C is the number of channels, H is the height, and W is the width. is the size of the feature map after pooling with C channels, 1 height and width.

[0011] Furthermore, in the step of multiplying the original feature map by the channel attention weight element by element to obtain the weighted feature map, the calculation formula is:

[0012] Among them, M is the channel attention weight, σ is the sigmoid activation function, and MLP is a shared multi-layer perceptron consisting of a learnable weight matrix and a ReLU activation function.

[0013] Furthermore, the classification loss adopts BCE_Loss, which is used for multi-label classification tasks.

[0014] Furthermore, the regression loss adopts two parts: EIoU_Loss and Distribution Focal Loss, where the calculation formula of EIoU_Loss is:

[0015] in, IOU is the intersection-and-union ratio, b , b gt are the center points of frame A and frame B respectively. c is the diagonal length of the minimum bounding rectangle that can contain both the predicted box and the true box. A is the prediction box, B is the real frame, w gt and h gt is the width and height of the prediction box, w and h is the width and height of the target box, is the Euclidean distance between two points, C w , C h They are the width and height of the minimum bounding rectangle that contains both the predicted box and the true box.

[0016] A second aspect of an embodiment of the present invention provides a multi-scale detection system for coal gangue based on a YOLOv8 network, which is used to implement the multi-scale detection method for coal gangue based on a YOLOv8 network described in the first aspect, and the system includes: A labeling module is used to obtain images of a mixture of coal and coal gangue, and label the images of the mixture of coal and coal gangue to obtain a data set; A training module is used to input the data set into the improved YOLOv8 network model for training to obtain a target network model; An input module, used for inputting the image to be detected into the target network model to determine the coal and coal gangue in the image to be detected; The improved YOLOv8 network model includes a backbone network consisting of 5 CBS modules, 4 CBC2f modules and 1 fast spatial pyramid pooling module, a bidirectional feature fusion module consisting of 2 CBS modules and 4 CBC2f modules in the neck, and finally a detection structure consisting of 3 detection heads of different sizes in the head. Among them, after a segmentation operation is performed on the input stream in the CBC2f module, the original input stream is divided into two equal parts, one of which is sent to the remaining CBS module in the CBC2f module, and the other part focuses on important features through the convolutional block attention module. Finally, the feature maps extracted by the three convolutional blocks are spliced ​​with the feature maps generated by the convolutional block attention module and finally output. The convolutional block attention module adopts an integrated channel and spatial attention mechanism; After the fast spatial pyramid pooling module performs convolution processing on the feature map output by the previous layer, the obtained feature map is subjected to maximum pooling through three pooling layers of 5×5, 9×9, and 13×13, and is concatenated with the feature maps output by the three pooling layers on the channel. At the same time, an additional convolution layer is set to extract features and the final features are jump-connected with the input features; The bidirectional feature fusion module increases the detection capability by setting jump connections between feature pyramids; The detection head includes two CBS modules connected in sequence, and convolution blocks connected to the two CBS modules respectively.

[0017] A third aspect of an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-scale detection method for coal gangue based on the YOLOv8 network provided in the first aspect.

[0018] A fourth aspect of an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the multi-scale detection method for coal gangue based on the YOLOv8 network provided in the first aspect is implemented.

[0019] A multi-scale detection method and system for gangue based on the YOLOv8 network provided in the embodiment of the present invention improves and optimizes the structure from the perspective of small target detection capability under multi-scale detection based on the YOLOv8 network structure. First, the CBAM attention mechanism is embedded in the core module C2f of the entire network to increase the small target detection capability; secondly, a fast spatial pyramid pooling module is used in the backbone network to integrate more front and back features, enhance the interaction of feature information, and further improve the multi-scale detection capability; then, a bidirectional feature fusion module with more jump connections is used to increase the multi-scale detection capability and prevent the loss of small target features due to network depth; finally, the convolution blocks used for feature fusion in the detection head are merged to be lightweight, and the regression loss is set to EIoU_Loss to avoid errors due to the consistency of the aspect ratio of the predicted box and the real box. The above-mentioned target network model can effectively detect 6mm~50mm particle size gangue, realize cross-particle size detection, and can be effectively deployed in gangue sorting equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a multi-scale detection method for coal gangue based on a YOLOv8 network provided in Example 1 of the present invention; Figure 2 This is a schematic diagram of the CBC2f module structure; Figure 3 This is a schematic diagram of the structure of the convolutional block attention module; Figure 4 This is a schematic diagram of the structure of the fast spatial pyramid pooling module; Figure 5 It is a schematic diagram of the bidirectional feature fusion module; Figure 6 This is a schematic diagram of the detection head structure; Figure 7 A structural block diagram of a multi-scale detection system for coal gangue based on a YOLOv8 network provided in Embodiment 2 of the present invention; Figure 8 This is a structural block diagram of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0021] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0022] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0024] Embodiment 1 According to an embodiment of the present invention, an embodiment of a multi-scale detection method for coal gangue based on a YOLOv8 network is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0025] In this embodiment 1, a multi-scale detection method for coal gangue based on the YOLOv8 network is provided, which is used to detect coal gangue with a particle size of 6mm to 50mm and can be used in electronic devices such as computers. Figure 1 , Figure 1 A flowchart of an implementation of a multi-scale detection method for coal gangue based on a YOLOv8 network provided in Embodiment 1 of the present invention is shown, which specifically includes steps S01 to S03.

[0026] Step S01, obtaining an image of a mixture of coal and coal gangue, and annotating the image of the mixture of coal and coal gangue to obtain a data set.

[0027] Among them, you can use the image annotation tool Labelme for annotation, and divide the data set proportionally into training set, validation set and test set according to 8:1:1.

[0028] Step S02, inputting the data set into the improved YOLOv8 network model for training to obtain a target network model.

[0029] Specifically, the improved YOLOv8 network model includes a backbone network consisting of 5 CBS modules, 4 CBC2f modules and 1 fast spatial pyramid pooling module (SPPFS module), a bidirectional feature fusion module (BiPAN) consisting of 2 CBS modules and 4 CBC2f modules in the neck, and finally a detection structure consisting of 3 detection heads of different sizes sent to the head; Among them, see Figure 2, is a schematic diagram of the CBC2f module structure, wherein the input feature map and the output feature map have the same size. After a segmentation operation is performed on the input stream in the CBC2f module, the original input stream is divided into two equal parts, one of which is fed into the remaining CBS module in the CBC2f module, and the other part focuses on important features through the convolutional block attention module (CBAM). Finally, the feature maps extracted by the three convolutional blocks are spliced ​​with the feature maps generated by the convolutional block attention module, and finally output. The convolutional block attention module adopts an integrated channel and spatial attention mechanism. In this embodiment, the CBAM attention mechanism is added to the C2f module, the basic building block of the entire network model, to increase the small target detection capability. The convolutional block attention module integrates the channel and spatial attention mechanisms to help the network model focus on important features more effectively, thereby improving the model's attention to the feature channels and position information of small targets.

[0030] See also Figure 3 , is a structural diagram of the convolutional block attention module, where c is concatenation and a is addition. The function of Bottleneck is to add a part of the input to another part of the input after passing through two 3×3CBS modules and then output. Specifically, the convolutional block attention module first performs average pooling and maximum pooling on the input feature map F(C×H×W) in the spatial dimension to obtain two C×1×1 first feature maps, which represent the average and maximum values ​​of the channel respectively. The calculation formula is:

[0031] in, Favg and Fmax They are the pooling results after channel average pooling and maximum pooling, respectively. is the position in the cth channel The characteristic description of is the scale range of the original feature map, C is the number of channels, H is the height, and W is the width. is the size of the feature map after pooling with C channels, 1 height and 1 width; The two first feature maps are input into the two-layer fully connected network respectively to generate two channel attention vectors and add them together. Then they are normalized by the sigmoid function to generate the final channel attention weight M. The calculation formula is:

[0032] Where M is the channel attention weight, σ is the sigmoid activation function, and MLP is a shared multi-layer perceptron consisting of a learnable weight matrix and a ReLU activation function; The original feature map F is multiplied element by element by the channel attention weight M to obtain the weighted feature map F1, and the calculation formula is:

[0033] in, is the dot product; The weighted feature map F1 is passed into the spatial attention unit, and average pooling and maximum pooling are performed on the channel dimension to obtain two 1×H×W second feature maps Favg(i,j) and Fmax(i,j) , the calculation formula is:

[0034] The two second feature maps are concatenated in the channel dimension to obtain a dual-channel feature map , and then a 7×7 convolution kernel is used to perform a convolution operation to obtain a spatial attention map ; The original feature map F is combined with the spatial attention map Multiply element by element to get the weighted target feature map F2, the calculation formula is:

[0035] After the fast spatial pyramid pooling module performs convolution processing on the feature map output by the previous layer, the obtained feature map is max-pooled through three pooling layers of 5×5, 9×9, and 13×13, and is concatenated with the feature maps output by the three pooling layers on the channel. At the same time, an additional convolution layer is set to extract features and the final features are skip-connected with the input features. Please refer to Figure 4 , is a schematic diagram of the structure of the fast spatial pyramid pooling module; The bidirectional feature fusion module increases the detection capability by setting up skip connections between feature pyramids. Figure 5 , is a schematic diagram of a bidirectional feature fusion module. The specific processing flow of the bidirectional feature fusion module is as follows: Original feature pyramid: P={P1, P2, P3}, where P1 is a 80×80 feature map with 256 channels; P2 is a 40×40 feature map with 512 channels; P3 is a 20×20 feature map with 1024 channels.

[0036] Upsampling and fusion: P3 corresponds to the new feature map T3. After 1×1 convolution and normalization, it is upsampled by 2 times to obtain a 40×40 feature map. It is then horizontally connected with P2 and the corresponding elements are added to obtain a 40×40 feature map T2.

[0037] After convolution and normalization, T2 is upsampled by 2 times to obtain a 80×80 feature map, which is horizontally connected to P1 and the corresponding elements are added to obtain a 80×80 feature map T1.

[0038] Downsampling and skip connections: Fuse T1 and P1 to obtain a 80×80 feature map F1.

[0039] After convolution and normalization, F1 is downsampled by 2 times to obtain a 40×40 feature map, and then jump-connected with P2, and the corresponding elements are added to obtain a 40×40 feature map F2.

[0040] After convolution and normalization, F2 is downsampled by 2 times to obtain a 20×20 feature map, and then jump-connected with P3, and the corresponding elements are added to obtain a 20×20 feature map F3.

[0041] Skip connections: F3 is skip-connected with T3 to obtain a 20×20 feature map N3.

[0042] F2 is skip-connected with T2 to obtain a 40×40 feature map N2.

[0043] F1 is skip-connected with T1 to obtain a 80×80 feature map N1.

[0044] Final output: N1: 80×80 N2: 40×40 N3: 20×20; The detection head includes two CBS modules connected in sequence and convolution blocks connected to the two CBS modules respectively. The convolution blocks for feature fusion in the detection head for regression loss and classification loss are merged together, while the convolution blocks for conversion to output remain unchanged, so as to achieve the purpose of lightweight. Please refer to Figure 6 , is a schematic diagram of the detection head structure.

[0045] It should be noted that the classification loss uses BCE_Loss for multi-label classification tasks, and the regression loss uses EIoU_Loss and Distribution Focal Loss. The calculation formula of EIoU_Loss is:

[0046] in, IOU is the intersection-and-union ratio, b , bgt are the center points of frame A and frame B respectively. c is the diagonal length of the minimum bounding rectangle that can contain both the predicted box and the true box.A is the prediction box, B is the real frame, wgt and hgt is the width and height of the prediction box, w and h is the width and height of the target box, is the Euclidean distance between two points, , They are the width and height of the minimum bounding rectangle that contains both the predicted box and the true box.

[0047] In this embodiment, the evaluation indicators are precision P (Precision), recall R (Recall), mean average precision mAP (Mean Average Precision) on the test set, including mAP50 (IoU = 0.5), mAP50-95 (IoU = 0.5: 0.05: 0.95), detection speed (ms) and Parameters (model parameter quantity) to judge the performance of the network model. The calculation formulas of P, R, and mAP are as follows:

[0048]

[0049]

[0050]

[0051] Among them, TP is the number of correct targets detected by the model, FP is the number of targets incorrectly detected by the model, FN is the number of false detections and missed detections by the model, AP is the average precision of a category, and K is the number of categories.

[0052] Step S03: input the image to be detected into the target network model to determine the coal and coal gangue in the image to be detected.

[0053] In summary, the multi-scale detection method of coal gangue based on the YOLOv8 network in the above embodiment of the present invention improves and optimizes the structure from the perspective of small target detection capability under multi-scale detection based on the YOLOv8 network structure. First, the CBAM attention mechanism is embedded in the core module C2f of the entire network to increase the small target detection capability; secondly, a fast spatial pyramid pooling module is used in the backbone network to integrate more front and back features, enhance the interaction of feature information, and further improve the multi-scale detection capability; then a bidirectional feature fusion module with more jump connections is used to increase the multi-scale detection capability and prevent the loss of small target features due to network depth; finally, the convolution blocks used for feature fusion in the detection head are merged to be lightweight, and the regression loss is set to EIoU_Loss to avoid errors due to the consistency of the aspect ratio of the predicted frame and the real frame. The above-mentioned target network model can effectively detect 6mm~50mm particle size coal gangue, realize cross-particle size detection, and can be effectively deployed in coal gangue sorting equipment.

[0054] Embodiment 2 See also Figure 7 , Figure 7 It is a structural block diagram of a multi-scale detection system for coal gangue based on a YOLOv8 network provided in Example 2 of the present invention. The multi-scale detection system 200 for coal gangue based on a YOLOv8 network is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0055] Specifically, the multi-scale detection system 200 for coal gangue based on the YOLOv8 network includes: a labeling module 21, a training module 22 and an input module 23, wherein: The labeling module 21 is used to obtain the image of the mixture of coal and coal gangue, and label the image of the mixture of coal and coal gangue to obtain a data set; A training module 22 is used to input the data set into the improved YOLOv8 network model for training to obtain a target network model; An input module 23, used for inputting the image to be detected into the target network model to determine the coal and coal gangue in the image to be detected; The improved YOLOv8 network model includes a backbone network consisting of 5 CBS modules, 4 CBC2f modules and 1 fast spatial pyramid pooling module, a bidirectional feature fusion module consisting of 2 CBS modules and 4 CBC2f modules in the neck, and finally a detection structure consisting of 3 detection heads of different sizes in the head. Among them, after the input stream in the CBC2f module undergoes a segmentation operation, the original input stream is divided into two equal parts, one part of which is sent to the remaining CBS module in the CBC2f module, and the other part focuses on important features through the convolutional block attention module. Finally, the feature map extracted by the three convolutional blocks is spliced ​​with the feature map generated by the convolutional block attention module, and finally output. The convolutional block attention module adopts an integrated channel and spatial attention mechanism. The convolutional block attention module is used to perform average pooling and maximum pooling on the input feature map in the spatial dimension to obtain two first feature maps, which represent the average value and maximum value of the channel respectively. The calculation formula is:

[0056] in, F avg and F max They are the pooling results after channel average pooling and maximum pooling, respectively. is the position in the cth channel The characteristic description of is the scale range of the original feature map, C is the number of channels, H is the height, and W is the width. is the size of the feature map after pooling with C channels, 1 height and 1 width; The two first feature maps are input into the two-layer fully connected network respectively to generate two channel attention vectors and add them together, and then normalize them through the sigmoid function to generate the final channel attention weight; The original feature map is multiplied element by element by the channel attention weight to obtain a weighted feature map. The calculation formula is:

[0057] Where M is the channel attention weight, σ is the sigmoid activation function, and MLP is a shared multi-layer perceptron consisting of a learnable weight matrix and a ReLU activation function; The weighted feature map is passed into the spatial attention unit, and average pooling and maximum pooling are performed on the channel dimension to obtain two second feature maps; The two second feature maps are concatenated in the channel dimension to obtain a dual-channel feature map, and then a 7×7 convolution kernel is used to perform a convolution operation to obtain a spatial attention map; Multiply the original feature map by the spatial attention map element by element to obtain a weighted target feature map; After the fast spatial pyramid pooling module performs convolution processing on the feature map output by the previous layer, the obtained feature map is subjected to maximum pooling through three pooling layers of 5×5, 9×9, and 13×13, and is concatenated with the feature maps output by the three pooling layers on the channel. At the same time, an additional convolution layer is set to extract features and the final features are jump-connected with the input features; The bidirectional feature fusion module increases the detection capability by setting jump connections between feature pyramids; The detection head includes two CBS modules connected in sequence, and convolution blocks connected to the two CBS modules respectively. The convolution blocks for feature fusion in the detection head for regression loss and classification loss are merged together, and the convolution for conversion to output remains unchanged to achieve the purpose of lightweight. The classification loss uses BCE_Loss for multi-label classification tasks, and the regression loss uses EIoU_Loss and Distribution Focal Loss. The calculation formula of EIoU_Loss is:

[0058] in, IOU is the intersection-and-union ratio, b , b gt are the center points of frame A and frame B respectively. c is the diagonal length of the minimum bounding rectangle that can contain both the predicted box and the true box. A is the prediction box, B is the real frame, w gt and h gt is the width and height of the prediction box, w and h is the width and height of the target box, is the Euclidean distance between two points, C w , C h They are the width and height of the minimum bounding rectangle that contains both the predicted box and the true box.

[0059] Embodiment 3 Another aspect of the present invention provides an electronic device, see Figure 8 , shown is an electronic device in Embodiment 3 of the present invention, including a memory 20, a processor 10, and a computer program 30 stored in the memory and executable on the processor. When the processor 10 executes the computer program 30, the multi-scale detection method for coal gangue based on the YOLOv8 network as described above is implemented.

[0060] In some embodiments, the processor 10 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 20, such as executing access restriction programs.

[0061] Among them, the memory 20 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 20 can be an internal storage unit of an electronic device, such as a hard disk of the electronic device. In other embodiments, the memory 20 can also be an external storage device of an electronic device, such as a plug-in hard disk equipped on the electronic device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Further, the memory 20 can also include both an internal storage unit of the electronic device and an external storage device. The memory 20 can not only be used to store application software and various types of data of the electronic device, but also can be used to temporarily store data that has been output or is to be output.

[0062] It should be pointed out that Figure 8 The structure shown does not constitute a limitation on the electronic device. In other embodiments, the electronic device may include fewer or more components than those shown in the figure, or combine certain components, or arrange the components differently.

[0063] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-scale detection method for coal gangue based on the YOLOv8 network as described above.

[0064] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0065] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0066] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0067] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0068] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the attached claims.

Claims

1. A multi-scale detection method for coal gangue based on YOLOv8 network, characterized in that: Used to detect coal gangue with a particle size of 6mm to 50mm, the method comprises: Acquire mixed images of coal and coal gangue, and annotate the mixed images of coal and coal gangue to obtain a data set; Input the data set into the improved YOLOv8 network model for training to obtain a target network model; Inputting the image to be detected into the target network model to determine the coal and coal gangue in the image to be detected; The improved YOLOv8 network model includes a backbone network consisting of 5 CBS modules, 4 CBC2f modules and 1 fast spatial pyramid pooling module, 2 CBS modules at the neck, a bidirectional feature fusion module consisting of 4 CBC2f modules, and finally a detection structure consisting of 3 detection heads of different sizes sent to the head, wherein the CBC2f module is a module obtained by embedding the bidirectional feature fusion module into the C2f module, and the CBS module is a module composed of a convolution layer, a normalization layer and an activation function combination; Among them, after a segmentation operation is performed on the input stream in the CBC2f module, the original input stream is divided into two equal parts, one of which is sent to the remaining CBS module in the CBC2f module, and the other part focuses on important features through the convolutional block attention module. Finally, the feature maps extracted by the three convolutional blocks are spliced ​​with the feature maps generated by the convolutional block attention module and finally output. The convolutional block attention module adopts an integrated channel and spatial attention mechanism; After the fast spatial pyramid pooling module performs convolution processing on the feature map output by the previous layer, the obtained feature map is subjected to maximum pooling through three pooling layers of 5×5, 9×9, and 13×13, and is concatenated with the feature maps output by the three pooling layers on the channel. At the same time, an additional convolution layer is set to extract features and the final features are jump-connected with the input features; The bidirectional feature fusion module increases the detection capability by setting jump connections between feature pyramids; The detection head includes two CBS modules connected in sequence, and convolution blocks connected to the two CBS modules respectively.

2. The multi-scale detection method for coal gangue based on the YOLOv8 network according to claim 1 is characterized in that: The convolutional block attention module is used to perform average pooling and maximum pooling on the input feature map in the spatial dimension to obtain two first feature maps, which respectively represent the average value and maximum value of the channel; The two first feature maps are input into the two-layer fully connected network respectively to generate two channel attention vectors and add them together, and then normalize them through the sigmoid function to generate the final channel attention weight; Multiply the original feature map by the channel attention weight element by element to obtain a weighted feature map; The weighted feature map is passed into the spatial attention unit, and average pooling and maximum pooling are performed on the channel dimension to obtain two second feature maps; The two second feature maps are concatenated in the channel dimension to obtain a dual-channel feature map, and then a 7×7 convolution kernel is used to perform a convolution operation to obtain a spatial attention map; The original feature map is multiplied element-by-element by the spatial attention map to obtain a weighted target feature map.

3. The multi-scale detection method for coal gangue based on the YOLOv8 network according to claim 2 is characterized in that: The convolution blocks for feature fusion in the detection head that perform regression loss and classification loss are merged together, while the convolution blocks for conversion to output remain unchanged, so as to achieve the purpose of lightweight.

4. The multi-scale detection method for coal gangue based on the YOLOv8 network according to claim 3 is characterized in that: In the step of performing average pooling and maximum pooling on the input feature map in the spatial dimension to obtain two first feature maps, the calculation formula is: in, F avg and F max They are the pooling results after channel average pooling and maximum pooling, respectively. is the position in the cth channel The characteristic description of is the scale range of the original feature map, C is the number of channels, H is the height, and W is the width. is the size of the feature map after pooling with C channels, 1 height and width.

5. The multi-scale detection method for coal gangue based on the YOLOv8 network according to claim 4 is characterized in that: In the step of multiplying the original feature map by the channel attention weight element by element to obtain the weighted feature map, the calculation formula is: Among them, M is the channel attention weight, σ is the sigmoid activation function, and MLP is a shared multi-layer perceptron consisting of a learnable weight matrix and a ReLU activation function.

6. The multi-scale detection method for coal gangue based on the YOLOv8 network according to claim 5 is characterized in that: The classification loss used is BCE_Loss, which is used for multi-label classification tasks.

7. The multi-scale detection method for coal gangue based on the YOLOv8 network according to claim 6 is characterized in that: The regression loss uses two parts: EIoU_Loss and Distribution Focal Loss. The calculation formula of EIoU_Loss is: in, IOU is the intersection-and-union ratio, b , b gt are the center points of frame A and frame B respectively. c is the diagonal length of the minimum bounding rectangle that can contain both the predicted box and the true box. A is the prediction box, B is the real frame, w gt and h gt is the width and height of the prediction box, w and h is the width and height of the target box, is the Euclidean distance between two points, C w , C h They are the width and height of the minimum bounding rectangle that contains both the predicted box and the true box.

8. A multi-scale detection system for coal gangue based on YOLOv8 network, characterized in that: For implementing the multi-scale detection method of coal gangue based on the YOLOv8 network as described in any one of claims 1 to 7, the system comprises: A labeling module is used to obtain images of a mixture of coal and coal gangue, and label the images of the mixture of coal and coal gangue to obtain a data set; A training module is used to input the data set into the improved YOLOv8 network model for training to obtain a target network model; An input module, used for inputting the image to be detected into the target network model to determine the coal and coal gangue in the image to be detected; The improved YOLOv8 network model includes a backbone network consisting of 5 CBS modules, 4 CBC2f modules and 1 fast spatial pyramid pooling module, a bidirectional feature fusion module consisting of 2 CBS modules and 4 CBC2f modules in the neck, and finally a detection structure consisting of 3 detection heads of different sizes in the head. Among them, after a segmentation operation is performed on the input stream in the CBC2f module, the original input stream is divided into two equal parts, one of which is sent to the remaining CBS module in the CBC2f module, and the other part focuses on important features through the convolutional block attention module. Finally, the feature maps extracted by the three convolutional blocks are spliced ​​with the feature maps generated by the convolutional block attention module and finally output. The convolutional block attention module adopts an integrated channel and spatial attention mechanism; After the fast spatial pyramid pooling module performs convolution processing on the feature map output by the previous layer, the obtained feature map is subjected to maximum pooling through three pooling layers of 5×5, 9×9, and 13×13, and is concatenated with the feature maps output by the three pooling layers on the channel. At the same time, an additional convolution layer is set to extract features and the final features are jump-connected with the input features; The bidirectional feature fusion module increases the detection capability by setting jump connections between feature pyramids; The detection head includes two CBS modules connected in sequence, and convolution blocks connected to the two CBS modules respectively.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the multi-scale detection method for coal gangue based on the YOLOv8 network as described in any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the multi-scale detection method for coal gangue based on the YOLOv8 network as described in any one of claims 1 to 7 is implemented.

Images