Limestone granularity detection method based on YOLO-ADM

Abstract

The invention relates to a limestone granularity detection method based on YOLO-ADM, which comprises the following steps: firstly, collecting a raw material crushing section limestone data set, then carrying out data enhancement on an initial sample limestone data set, and dividing the initial sample limestone data set into a training set, a verification set and a test set; a deformable attention mechanism DAttention, a multi-scale sequence fusion module SSFF and a triple feature decoding module TFE are integrated in the YOLOv8 framework, and a YOLO-ADM model is obtained; on this basis, a training loss function is improved, and a loss function MioU based on the auxiliary frame and the minimum point distance is designed; and training the YOLO-ADM model through the training set and testing the performance of the network model through the test set. According to the method, the deformable attention DAttention is introduced into the YOLOv8 network, the YOLOv8 neck network is redesigned based on the multi-scale sequence fusion module SSFF and the triple feature decoding module TFE, the designed loss function MioU is combined, the excellent performance is shown in limestone detection, and the generalization performance is good.

Description

[0001] This application is a divisional application of the patent application entitled "Limestone particle size detection method based on YOLO-ADM". The application date of the original application is December 30, 2024, and the application number is 202411963167.8. Technical Field

[0002] The present invention relates to the technical field of image processing, and in particular to a limestone particle size detection method based on YOLO-ADM. Background Art

[0003] Limestone crushing is a critical step in cement production, providing raw materials that meet specific particle size requirements and directly impacting cement quality and performance. Due to the large variation in limestone ore particle size, this can often cause crusher blockages and production accidents. This can also lead to extended crushing times, increased current consumption, and reduced production, resulting in energy waste and reduced efficiency, impacting production safety.

[0004] Early research relied on manual labor to determine limestone ore particle size, a process that not only required significant manpower and resources but also suffered from low accuracy and efficiency, failing to meet the demands of ore utilization. With the continuous development and improvement of visual recognition algorithms, these technologies have become applicable to ore particle size identification, surpassing human accuracy in many areas. Early researchers primarily used semantic segmentation techniques, such as U-Net and PSPNet, to analyze raw ore particle size. Recent research has also seen improvements and innovations in these algorithms.

[0005] Despite significant progress in particle size measurement of processed raw ores, research on particle size measurement during limestone crushing in cement plants still faces several challenges. First, existing research relies on complex image processing techniques, which increases implementation difficulty and cost. Second, some studies that have achieved real-time statistics and online measurement of ore particle size often require high-performance hardware or specialized analytical instruments, which to some extent limits their practical application. Therefore, further research and exploration is needed on particle size measurement during limestone crushing. Summary of the Invention

[0006] In order to overcome the deficiencies of the prior art, the present invention aims to provide a limestone particle size detection method based on YOLO-ADM, which can at least solve one of the technical problems in the background technology.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] A limestone particle size detection method based on YOLO-ADM comprises the following steps:

[0009] S1. Collecting a stone block dataset: Collect images of limestone blocks under different lighting conditions at different limestone discharge ports throughout the day to create an initial sample limestone block dataset.

[0010] S2, divide the limestone block dataset obtained in S1 into training set, validation set, and test set according to the ratio of 7:2:1;

[0011] S3, perform data augmentation on the training set samples in S2, expanding the samples to three times their original size, wherein the data augmentation operations include rotation, random cropping, and blurring, wherein the rotation includes 90° clockwise rotation, 90° counterclockwise rotation, and inversion, and the blurring is Gaussian blurring with a blur radius of 3px;

[0012] S4. Integrate the deformable attention mechanism DAttention into the YOLOV8 backbone network;

[0013] S5, integrate the multi-scale sequence feature fusion module SSFF and the triple feature decoding module TFE into the YOLOv8 neck network, and add them to S4 to obtain the YOLO-ADM network model;

[0014] S6. Generate auxiliary bounding boxes based on the scale factor and design a new YOLO-ADM model loss function MioU by minimizing the diagonal distance between the limestone prediction box and the true box. Replace the IoU part in the MPDIoU calculation formula with Inner-IoU to construct a new loss function MioU.

[0015] S7. Train the YOLO-ADM network model using the training set and verify the YOLO-ADM model using the validation set during the training process.

[0016] S8. Test the recognition performance of the YOLO-ADM network model using the test set;

[0017] S9, after pixel-level processing of the limestone object recognition frame predicted by the trained YOLO-ADM model, the size of each limestone particle is obtained;

[0018] Step S1 specifically includes:

[0019] S11, using a high-definition camera installed at each limestone discharge port to take pictures of the limestone raw materials in real time;

[0020] S12, screening the limestone pictures obtained in step S11, selecting pictures under different lighting, different discharge ports, and different time periods, averaging the screened limestone data pictures in terms of category, and eliminating pictures due to blurring;

[0021] S13, annotating the filtered limestone images, framing the target area where the limestone blocks of a set size are located, i.e., the position of the real frame, and then using the annotated limestone images as the initial sample data set;

[0022] The specific operations of step S4 are as follows:

[0023] S41. Obtain the query value by performing a dot product weight matrix of the elements in the input image through a deformable attention mechanism DAttention. Then, dynamically move the key K and value V to the set position according to a set of offsets based on an offset network. The offset network uses GELU (Gaussian Error Linear Unit) as a nonlinear activation function.

[0024] S42, based on the C2f module in the existing YOLOV8 backbone network, the input features first undergo convolution and split operations and are divided into two parts; one part is input to the residual stacking module Bottleneck. The output of each residual module will lead to a branch for subsequent splicing operation, and the other part after the split operation is directly spliced ​​with the output branches of these residual modules; finally, feature extraction is completed through the convolution operation, and the deformable attention module DAttention is inserted into the convolution module. Then, the other part after the split operation is directly spliced ​​with the output branches of these residual modules to obtain the new C2f-DAttention module;

[0025] S43, replacing the C2f module on the upper layer of the multi-scale sequence feature fusion module SPPF module of the backbone network with the C2f-DAttention module obtained in S42;

[0026] The specific operations of step S5 are as follows:

[0027] S51. In the existing YOLOV8 network structure, the backbone network CSPDarkNet53 contains three effective branches P3, P4, and P5. These three effective branches are used as the multi-scale sequence feature fusion module SSFF, as follows:

[0028] The SSFF module is designed based on the P3 feature map. It first resizes P4, P5 to P3 through 1×1 convolution and nearest neighbor interpolation. Then, it uses the unsqueeze method to increase the dimension of each feature layer from a three-dimensional tensor to a four-dimensional tensor. Finally, it stacks them together and inputs them into the 3D convolution channel to achieve multi-scale feature combination.

[0029] In S52 and the backbone network CSPDarkNet53, the P3 and P4 layers are convolved to obtain large-scale feature maps and medium-scale feature maps. The multi-scale sequence feature fusion module SPPF layer is convolved to obtain small-scale feature maps. The large, medium, and small-scale feature maps are input into the TFE module for feature fusion, as follows:

[0030] The triple feature decoding module TFE first performs convolution processing on the large-size feature map, and then uses a hybrid structure of maximum pooling and average pooling for downsampling; for the small-size feature map, the convolution module is used to adjust the number of channels, and then the nearest neighbor interpolation method is used for upsampling. Finally, the three large, medium, and small-size feature maps of the same size are convolved once and concatenated in the channel dimension. The calculation formula is as follows:

[0031] FTEE=Concate(Fl,Fm,Fs)

[0032] Among them, FTEE represents the feature map output by the triple feature decoding module TFE, Fl, Fm, and Fs represent the feature maps of large, medium, and small sizes respectively, and FTEE is obtained by splicing Fl, Fm, and Fs; FTEE has the same resolution as Fm, and the number of channels is three times that of Fm;

[0033] The specific operations of step S7 are as follows:

[0034] S71. Prepare three folders, train, valid, and test, according to the YOLO format to store the training set, validation set, and test set respectively. Each folder contains two folders, images and labels. The images folder stores the limestone image data, and the labels folder stores the label files of the corresponding images.

[0035] S72. Configure network selection, select Windows system and pytorch framework for training;

[0036] S73. Set the network model parameters. According to the computer video memory and memory size, set the YOLO-ADM network model input image size, recognition quantity and name, pre-training weight file, number of training rounds, mosaic enhancement, optimizer type, and patience parameter.

[0037] S74, training until the network model converges, using the validation set for verification during the training process, saving the weight files obtained in each training round and the most weighted files of the trained arguments;

[0038] The specific operations of step S8 are as follows:

[0039] S81. Load the trained optimal network model weight file obtained in S7, input the test set into the trained YOLO-ADM network model, perform feature extraction through the backbone network with DAttention, and fuse the deep and shallow features with the neck network of the multi-scale sequence feature fusion module SSFF and the triple feature decoding module TFE, and obtain all the prediction results of the network belonging to the two feature layers;

[0040] S82, obtaining the coordinates, confidence levels, and class probabilities of all prediction boxes that may contain limestone blocks by setting a confidence threshold; then removing redundant prediction boxes by performing non-maximum suppression to obtain the final detection result;

[0041] The specific operations of step S9 are as follows:

[0042] S91, combining the prediction frame coordinates obtained in S8 with the camera focal length, the straight-line distance between the camera and the ore, and the shooting angle to obtain the particle size of the limestone block corresponding to each prediction frame;

[0043] S92. Obtain the particle size level of each stone according to the set classification threshold.

[0044] Preferably, the calculation process of MPDIoU is:

[0045]

[0046] Among them, (x1, y1), (x2, y2) represent the coordinates of the upper left corner and lower right corner of the prediction box; Indicates the coordinates of the upper left corner and lower right corner of the real box. d1 and d2 are the distances between the upper left corner and lower right corner of the two boxes respectively.

[0047] Preferably, the calculation process of the I-IoU loss function includes:

[0048] Define the coordinates of the upper left and lower right corners of the real box:

[0049]

[0050] Where, is the center point of the ground truth box, w gt , h gt are the width and height of the ground truth box respectively;

[0051] Then, define the top left and bottom right coordinates of the predicted bounding box:

[0052]

[0053] Among them, (x c ,y c) is the center point of the predicted bounding box, w and h are the width and height of the predicted bounding box respectively;

[0054] Next, calculate the intersection and union areas:

[0055]

[0056] Finally, calculate I-IoU:

[0057] LI+I oU =1-S in / S un .

[0058] Preferably, the calculation process of step S9 is:

[0059] S91, the predicted frame coordinates obtained in S8 are combined with the camera focal length f, the straight-line distance d between the camera and the raw ore l Or the vertical distance d between the camera and the raw ore s , the shooting angle θ parameter of the lens to the raw ore, the pixel size d in the image p Convert to actual size d a The specific calculation formula is as follows:

[0060]

[0061] S92. When calculating the particle size of the raw ore, the pixel size in the image is the maximum geometric pixel distance between the coordinates of the diagonal points of the identification frame of the ore object. This is substituted into the calculation formula in S91 to obtain the size of each raw ore particle. The specific calculation formula is as follows:

[0062] Maximum geometric distance d max The calculation is as follows:

[0063]

[0064] Where: d1, d2 - the geometric pixel distance between the diagonal vertices of the mineral identification frame;

[0065] —The pixel coordinates of the upper left corner of the recognition box;

[0066] —The pixel coordinates of the lower right corner of the recognition box;

[0067] —The pixel coordinates of the upper right corner of the recognition box;

[0068] —The pixel coordinates of the lower left corner of the recognition box;

[0069] After conversion, the ore particle size is obtained as follows:

[0070]

[0071] Where: μ is the raw ore material conversion coefficient.

[0072] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0073] The YOLO-ADM-based limestone particle size detection method of the present invention first collects a limestone dataset from the raw material crushing section, then performs data enhancement on the initial sample limestone dataset and divides it into a training set, a validation set, and a test set; the deformable attention mechanism DAttention, the multi-scale sequence fusion module SSFF, and the triple feature decoding module TFE are integrated into the YOLOv8 framework to obtain the YOLO-ADM model; and on this basis, the training loss function is improved, and a loss function MioU based on the auxiliary bounding box and the minimum point distance is designed; the YOLO-ADM model is trained using the training set and the performance of the network model is tested using the test set. The method of the present invention introduces deformable attention DAttention into the YOLOv8 network, and redesigns the YOLOv8 neck network based on the multi-scale sequence fusion module SSFF and the triple feature decoding module TFE. Combined with the designed loss function MioU, it exhibits excellent performance in limestone detection and good generalization performance.

[0074] This paper improves the YOLOv8 framework by introducing a deformable attention mechanism module (DAttention) to optimize the backbone network. This allows the model to flexibly distort the sampling grid and dynamically adjust the features that require attention, enabling the model to better identify dense and stacked raw ore targets. Based on the SSFF module and the TFE module, this paper redesigns a multi-scale feature fusion network and the YOLOv8 neck network to improve the detection performance of dense and stacked targets. On this basis, the feature layers of larger-sized targets are secondary fused for the detection of smaller and denser ore targets. Based on the auxiliary bounding box and the metric of minimizing the diagonal distance between the true box and the predicted box, this paper proposes an improved loss function MIoU (MioU) to address the weak generalization ability of the CIoU loss function in the original YOLOv8 network. At the same time, an algorithm is proposed to generate auxiliary bounding boxes based on the scale factor and directly minimize the diagonal vertex distance between the predicted box and the true box. This more accurately reflects the similarity between the true box and the predicted box, and speeds up the convergence of the bounding box regression, thereby improving the detection performance of the model. This allows the detection results to be better applied to the crushing control process. This has important theoretical and practical significance and promotion value for the application of complex industrial problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0076] Figure 1 is a flow chart of an embodiment of the present invention;

[0077] Figure 2 This is a diagram of the deformable attention DAttention operation structure of an embodiment of the present invention;

[0078] Figure 3 The C2f_DAttention module introduces the attention mechanism in the embodiment of the present invention;

[0079] Figure 4 This is the YOLO-ADM network structure of an embodiment of the present invention;

[0080] Figure 5 It is a multi-scale sequence feature fusion module SSFF according to an embodiment of the present invention;

[0081] Figure 6 This is the triple feature decoding module TFE of an embodiment of the present invention;

[0082] Figure 7 This is a schematic diagram of MPDIoU calculation according to an embodiment of the present invention;

[0083] Figure 8 This is a schematic diagram of I-IoU calculation according to an embodiment of the present invention;

[0084] Figure 9 This is the limestone block recognition effect of the improved YOLO-ADM network model in a normal environment according to an embodiment of the present invention;

[0085] Figure 10 This is the limestone block recognition effect of the improved YOLO-ADM network model in an environment with dust;

[0086] Figure 11 This is the limestone block recognition effect of the improved YOLO-ADM network model in other crushing openings according to the embodiment of the present invention;

[0087] Figure 12 This is a comparison of the performance curves of the improved YOLO-ADM and YOLOv8 according to the embodiment of the present invention;

[0088] Figure 13 Schematic diagram of the conversion between the pixels and the actual size of a limestone image according to an embodiment of the present invention;

[0089] Figure 14 This is a schematic diagram of limestone particle size calculation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0090] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0091] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0092] The terms "first," "second," "third," and "fourth," etc., in the specification, claims, and drawings of this application are used to distinguish between different items, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a statement that a sequence of steps, a process, or a method is included is not limited to the listed steps but may optionally include steps not listed, or may optionally include other steps inherent to the process, method, product, or apparatus.

[0093] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0094] like Figure 1 As shown, the limestone particle size detection method based on YOLO-ADM described in this embodiment is implemented as follows:

[0095] S1. Throughout the day, collect dense images of limestone blocks under different lighting conditions at different limestone discharge ports to ensure the practicality and adequacy of the data. Then, use the collected image data to create an initial sample limestone block dataset.

[0096] The specific method of step S1 is as follows:

[0097] S11, taking pictures of the limestone raw materials in real time using a high-definition camera installed at each limestone discharge port;

[0098] S12, screening the limestone pictures obtained in step S11, selecting pictures under different lighting, different discharge ports, and different time periods, averaging the screened limestone data pictures in terms of category, and eliminating pictures due to blurring;

[0099] S13, the ID of the limestone picture defined picture screened out, then each picture is marked with labelimg software, the target area at the limestone block of a certain size is framed, i.e. the position of the real frame, the txt label file of the yolo format is generated, the label file name is corresponding to the picture ID, and then the limestone picture after marking is used as the initial sample data set, the limestone block of a certain size refers to the stone that can be seen by face and can be sieved out by the screen before entering the crusher;

[0100] S2. Divide the limestone block dataset obtained in S1 into training set, validation set, and test set according to the ratio of 7:2:1, and generate train, valid, and test folders respectively. Each folder corresponds to two folders, images and labels. The images folder stores the collected original images, and the labels folder stores the label files corresponding to the image IDs in the images folder.

[0101] S3, perform data augmentation on the training set samples in S2, expanding the samples to three times their original size, wherein the data augmentation operations include rotation, random cropping, and blurring, wherein the rotation includes 90° clockwise rotation, 90° counterclockwise rotation, and inversion, and the blurring is Gaussian blurring with a blur radius of 3px;

[0102] S4. Integrate the deformable attention mechanism DAttention into the YOLOV8 backbone network;

[0103] The specific operations of step S4 are as follows:

[0104] S41, construct a deformable attention mechanism DAttention Figure 2 As shown, the query volume is obtained by taking the dot product weight matrix of the elements in the input image, and then a set of offsets are generated based on the offset network. Then, the key K and value V are dynamically moved to important positions according to the offsets. The offset network uses GELU (Gaussian Error Linear Unit) as a nonlinear activation function.

[0105] S42, build the improved C2f module C2f-DAttention, such as Figure 3As shown. In the C2f module of the existing YOLOV8 backbone network, the input features first undergo convolution and split operations and are divided into two parts. One part is input to the residual stacking module Bottleneck. The output of each residual module will lead to a branch for subsequent splicing operations, and the other part after the split operation is directly spliced ​​with the output branches of these residual modules. Finally, the feature extraction is completed through the convolution operation, and the deformable attention module DAttention is inserted into the convolution module. Then, the other part after the split operation is directly spliced ​​with the output branches of these residual modules to obtain a new module C2f-DAttention;

[0106] S43, the C2f-DAttention module obtained in S42 is Figure 4 The C2f module on the upper layer of the backbone network SPPF module is replaced as shown;

[0107] S5. According to Figure 4 As shown, the multi-scale sequence feature fusion module SSFF and the triple feature decoding module TFE are integrated into the YOLOv8 neck network, and added to S4 to obtain the YOLO-ADM network model; the specific operations of step S5 are as follows:

[0108] S51, construct a multi-scale sequence feature fusion module SSFF module, such as Figure 5 As shown in the figure, in the existing YOLOV8 network structure, the backbone network CSPDarkNet53 contains three effective branches P3, P4, and P5. These three effective branches are used as the multi-scale sequence feature fusion module SSFF. Specifically, the SSFF module is designed based on the P3 feature map. First, the size of P4 and P5 is adjusted to P3 through 1×1 convolution and nearest neighbor interpolation. Then, the unsqueeze method is used to increase the dimension of each feature layer from a three-dimensional tensor to a four-dimensional tensor (increasing the depth). Finally, they are stacked together and input into the 3D convolution channel to realize the combination of multi-scale features.

[0109] S52, construct a triple feature decoding module, such as Figure 6 As shown in the figure, in the backbone network CSPDarkNet53, the P3 and P4 layers are convolved to obtain large-scale feature maps and medium-scale feature maps, and the SPPF layer is convolved to obtain small-scale feature maps. The large-, medium-, and small-scale feature maps are input into the TFE module for feature fusion, as follows:

[0110] The TFE module first performs convolution on the large-scale feature map, and then downsamples it using a hybrid structure of maximum pooling and average pooling. For the small-scale feature map, the convolution module is used to adjust the number of channels, and then upsamples it using the nearest neighbor interpolation method. Finally, the three large, medium, and small-scale feature maps of the same size are convolved once and concatenated along the channel dimension. The calculation formula is as follows:

[0111] F TEE =Concate(F l , F m , F s )

[0112] Among them, F TEE Represents the feature map output by the TFE module. l , F m , F s Represent the feature maps of large, medium and small sizes respectively. FTEE is composed of F l , F m , F s FTEE has the same m Same resolution, and the number of channels is F m three times;

[0113] S6. Generate auxiliary bounding boxes based on the scale factor and design a new YOLO-ADM model loss function MioU by minimizing the diagonal distance between the limestone prediction box and the true box. The existing YOLOv8 loss function uses CIoU. MPDIoU replaces CIoU, and the IoU part in the MPDIoU calculation formula is replaced with Inner-IoU to construct a new loss function MioU. The specific calculation formula is as follows:

[0114] MIoU=MPDIoU-IoU+I-IoU

[0115] The MPDIoU loss function is a new bounding box similarity comparison metric based on minimum point distance. The specific calculation process is as follows:

[0116]

[0117] Among them, according to Figure 7 As shown, (x1, y1), (x2, y2) represent the coordinates of the upper left corner and lower right corner of the prediction box; Indicates the coordinates of the upper left corner and lower right corner of the real box. d1 and d2 are the distances between the upper left corner and lower right corner of the two boxes respectively.

[0118] The I-IoU loss function is a novel bounding box similarity comparison metric based on minimum point distance. The specific calculation process is as follows: During model training, a scale factor is used to control the generation of auxiliary bounding boxes of different scales for loss calculation. In this way, a smaller auxiliary bounding box loss has a positive effect on the regression of high IoU samples, while the opposite is true for low IoU samples. Its calculation formula is as follows:

[0119] First, define the coordinates of the upper left and lower right corners of the ground truth box:

[0120]

[0121] In the formula, according to Figure 8 As shown, is the center point of the ground truth box, w gt , h gt are the width and height of the ground truth box respectively.

[0122] Then, define the top left and bottom right coordinates of the predicted bounding box:

[0123]

[0124] Among them, (x c ,y c ) is the center point of the predicted bounding box, w and h are the width and height of the predicted bounding box respectively.

[0125] Next, calculate the intersection and union areas:

[0126]

[0127] Finally, calculate I-IoU:

[0128] LI+I o U=1-S in / S un

[0129] From this, we can see that when γ is less than 1, the auxiliary bounding box is smaller than the actual bounding box, and its effective range of regression is smaller than the IoU loss, but its absolute gradient is larger than the gradient obtained by the IoU loss, which can accelerate the convergence of high IoU samples. Conversely, when γ is greater than 1, the larger auxiliary bounding box expands the effective range of regression, which improves the regression of low IoU samples.

[0130] S7. Train the YOLO-ADM network model using the training set and verify the YOLO-ADM model using the validation set during the training process.

[0131] The specific operations of step S7 are as follows:

[0132] S71. Prepare three folders, train, valid, and test, according to the YOLO format to store the training set, validation set, and test set respectively. Each folder contains two folders, images and labels. The images folder stores the limestone image data, and the labels folder stores the label files of the corresponding images.

[0133] S72. Configure network selection, select Windows system and pytorch framework for training;

[0134] S73. Set the network model parameters. According to the computer's video memory and memory size, set the YOLO-ADM network model input image size, recognition quantity and name, pre-trained weight file, number of training rounds, mosaic enhancement, optimizer type, and patience parameter. The user needs to use a graphics card that supports CUDA acceleration.

[0135] S74, training until the network model converges, using the validation set for verification during the training process, saving the weight files obtained in each training round and the most weighted files of the trained arguments;

[0136] S8. Test the recognition performance of the YOLO-ADM network model using the test set;

[0137] The specific operations of step S8 are as follows:

[0138] S81. Load the trained optimal network model weight file obtained in S7, input the test set into the trained YOLO-ADM network model, perform feature extraction through the backbone network with DAttention, and integrate the neck network with SSFF and TFE modules to achieve the fusion of deep and shallow features, and obtain all the prediction results of the network belonging to the two feature layers;

[0139] S82, obtaining the coordinates, confidence levels, and class probabilities of all prediction boxes that may contain limestone blocks by setting a confidence threshold; then removing redundant prediction boxes by performing non-maximum suppression to obtain the final detection result;

[0140] Figure 9 This is the limestone block recognition effect of the improved YOLO-ADM network model in a normal environment; Figure 10 The improved YOLO-ADM network model shows the recognition effect of limestone blocks in a dusty environment. Figure 11 The improved YOLO-ADM network model shows the recognition effect of limestone blocks in other crushing openings;

[0141] S9. Based on the real box position of the test set and the prediction results obtained by the model, calculate the average precision mAP, accuracy, and recall rate of the model and perform a systematic evaluation of the model. Figure 12 As shown in the figure, compared with the YOLOv8 model, the mAP0.5 of YOLO-ADM increased by 1.13% to 84.17%; the mAP0.5:0.95 increased by 1.49%; and the accuracy increased by 1.59%.

[0142] S10, after pixel-level processing of the limestone object recognition frame predicted by the YOLO-ADM model, the size of each limestone particle is obtained;

[0143] The specific operations of step S9 are as follows:

[0144] S91, according to Figure 13 As shown, the predicted frame coordinates obtained in S8 can be combined with the camera focal length f, the straight-line distance d between the camera and the raw ore l Or the vertical distance d between the camera and the raw ore s , the shooting angle θ of the lens to the raw ore and other parameters, the pixel size d in the image p Convert to actual size d a The specific calculation formula is as follows:

[0145]

[0146] S92, according to Figure 14 As shown, when calculating the particle size of raw ore, the pixel size in the image is the maximum geometric pixel distance of the diagonal coordinates of the identification frame of the ore object. Substituting it into the S91 calculation formula can obtain the size of each raw ore particle size. The specific calculation formula is as follows:

[0147] Maximum geometric distance d max The calculation is as follows:

[0148]

[0149] Where: d1, d2 - the geometric pixel distance between the diagonal vertices of the mineral identification frame;

[0150] —The pixel coordinates of the upper left corner of the recognition box;

[0151] —The pixel coordinates of the lower right corner of the recognition box;

[0152] —The pixel coordinates of the upper right corner of the recognition box;

[0153] —The pixel coordinates of the lower left corner of the recognition box;

[0154] Further conversion can be obtained, the ore particle size is:

[0155]

[0156] Where: μ is the raw ore material conversion coefficient.

[0157] In summary, the limestone particle size detection method based on YOLO-ADM in the embodiment of the present invention improves the YOLOv8 framework and introduces a deformable attention mechanism module (DAttention) to optimize the backbone network, so that the model can flexibly distort the sampling grid and dynamically adjust the features that need to be paid attention to, so that the model can better identify dense and stacked raw ore targets; the present invention redesigns a multi-scale feature fusion network based on the SSFF module and the TFE module, and redesigns the YOLOv8 neck network to improve the detection performance of dense and stacked targets; on this basis, the feature layer of the larger size target is secondary fused for smaller and Denser ore target detection; Based on the auxiliary bounding box and the metric of minimizing the diagonal distance between the true box and the predicted box, this paper proposes an improved loss function MIoU to solve the problem of weak generalization ability of the CIoU loss function in the original YOLOv8 network, and proposes an algorithm based on the scale factor to generate auxiliary bounding boxes and directly minimize the diagonal vertex distance between the predicted box and the true box, so as to more accurately reflect the similarity between the true box and the predicted box, and improve the bounding box regression convergence speed, thereby improving the detection performance of the model, so that the detection results can be better applied to the crushing control process, which has important theoretical and practical significance and promotion value for the application of complex industrial problems.

[0158] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0159] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A limestone particle size detection method based on YOLO-ADM, characterized in that: The following steps are included: S1. Collecting a stone block dataset: Collect images of limestone blocks under different lighting conditions at different limestone discharge ports throughout the day to create an initial sample limestone block dataset. S2, divide the limestone block dataset obtained in S1 into training set, validation set, and test set according to the ratio of 7:2:1; S3, perform data augmentation on the training set samples in S2, expanding the samples to three times their original size, wherein the data augmentation operations include rotation, random cropping, and blurring, wherein the rotation includes 90° clockwise rotation, 90° counterclockwise rotation, and inversion, and the blurring is Gaussian blurring with a blur radius of 3px; S4. Integrate the deformable attention mechanism DAttention into the YOLOV8 backbone network; S5, integrate the multi-scale sequence feature fusion module SSFF and the triple feature decoding module TFE into the YOLOv8 neck network, and add them to S4 to obtain the YOLO-ADM network model; S6. Generate auxiliary bounding boxes based on the scale factor and design a new YOLO-ADM model loss function MioU by minimizing the diagonal distance between the limestone prediction box and the true box. Replace the IoU part in the MPDIoU calculation formula with Inner-IoU to construct a new loss function MioU. S7. Train the YOLO-ADM network model using the training set and verify the YOLO-ADM model using the validation set during the training process. S8. Test the recognition performance of the YOLO-ADM network model using the test set; S9, after pixel-level processing of the limestone object recognition frame predicted by the trained YOLO-ADM model, the size of each limestone particle is obtained; Step S1 specifically includes: S11, using a high-definition camera installed at each limestone discharge port to take pictures of the limestone raw materials in real time; S12, screening the limestone pictures obtained in step S11, selecting pictures under different lighting, different discharge ports, and different time periods, averaging the screened limestone data pictures in terms of category, and eliminating pictures due to blurring; S13, annotating the filtered limestone images, framing the target area where the limestone blocks of a set size are located, i.e., the position of the real frame, and then using the annotated limestone images as the initial sample data set; The specific operations of step S4 are as follows: S41. Obtain the query value by performing a dot product weight matrix of the elements in the input image through a deformable attention mechanism DAttention. Then, dynamically move the key K and value V to the set position according to a set of offsets based on an offset network. The offset network uses GELU (Gaussian Error Linear Unit) as a nonlinear activation function. S42, based on the C2f module in the existing YOLOV8 backbone network, the input features first undergo convolution and split operations and are divided into two parts; one part is input to the residual stacking module Bottleneck. The output of each residual module will lead to a branch for subsequent splicing operation, and the other part after the split operation is directly spliced ​​with the output branches of these residual modules; finally, feature extraction is completed through the convolution operation, and the deformable attention module DAttention is inserted into the convolution module. Then, the other part after the split operation is directly spliced ​​with the output branches of these residual modules to obtain the new C2f-DAttention module; S43, replacing the C2f module on the upper layer of the multi-scale sequence feature fusion module SPPF module of the backbone network with the C2f-DAttention module obtained in S42; The specific operations of step S5 are as follows: S51. In the existing YOLOV8 network structure, the backbone network CSPDarkNet53 contains three effective branches P3, P4, and P5. These three effective branches are used as the multi-scale sequence feature fusion module SSFF, as follows: The SSFF module is designed based on the P3 feature map. It first resizes P4, P5 to P3 through 1×1 convolution and nearest neighbor interpolation. Then, it uses the unsqueeze method to increase the dimension of each feature layer from a three-dimensional tensor to a four-dimensional tensor. Finally, it stacks them together and inputs them into the 3D convolution channel to achieve multi-scale feature combination. In S52 and the backbone network CSPDarkNet53, the P3 and P4 layers are convolved to obtain large-scale feature maps and medium-scale feature maps. The multi-scale sequence feature fusion module SPPF layer is convolved to obtain small-scale feature maps. The large, medium, and small-scale feature maps are input into the TFE module for feature fusion, as follows: The triple feature decoding module TFE first performs convolution processing on the large-size feature map, and then uses a hybrid structure of maximum pooling and average pooling for downsampling; for the small-size feature map, the convolution module is used to adjust the number of channels, and then the nearest neighbor interpolation method is used for upsampling. Finally, the three large, medium, and small-size feature maps of the same size are convolved once and concatenated in the channel dimension. The calculation formula is as follows: FTEE=Concate(Fl,Fm,Fs) Among them, FTEE represents the feature map output by the triple feature decoding module TFE, Fl, Fm, and Fs represent the feature maps of large, medium, and small sizes respectively, and FTEE is obtained by splicing Fl, Fm, and Fs; FTEE has the same resolution as Fm, and the number of channels is three times that of Fm; The specific operations of step S7 are as follows: S71. Prepare three folders, train, valid, and test, according to the YOLO format to store the training set, validation set, and test set respectively. Each folder contains two folders, images and labels. The images folder stores the limestone image data, and the labels folder stores the label files of the corresponding images. S72. Configure network selection, select Windows system and pytorch framework for training; S73. Set the network model parameters. According to the computer video memory and memory size, set the YOLO-ADM network model input image size, recognition quantity and name, pre-training weight file, number of training rounds, mosaic enhancement, optimizer type, and patience parameter. S74, training until the network model converges, using the validation set for verification during the training process, saving the weight files obtained in each training round and the most weighted files of the trained arguments; The specific operations of step S8 are as follows: S81. Load the trained optimal network model weight file obtained in S7, input the test set into the trained YOLO-ADM network model, perform feature extraction through the backbone network with DAttention, and fuse the deep and shallow features with the neck network of the multi-scale sequence feature fusion module SSFF and the triple feature decoding module TFE, and obtain all the prediction results of the network belonging to the two feature layers; S82, obtaining the coordinates, confidence levels, and class probabilities of all prediction boxes that may contain limestone blocks by setting a confidence threshold; then removing redundant prediction boxes by performing non-maximum suppression to obtain the final detection result; The specific operations of step S9 are as follows: S91, combining the prediction frame coordinates obtained in S8 with the camera focal length, the straight-line distance between the camera and the ore, and the shooting angle to obtain the particle size of the limestone block corresponding to each prediction frame; S92. Obtain the particle size level of each stone according to the set classification threshold.

2. the limestone particle size detection method based on YOLO-ADM according to claim 1, is characterized in that, The calculation process of MPDIoU is: Among them, (x1, y1), (x2, y2) represent the coordinates of the upper left corner and lower right corner of the prediction box; Indicates the coordinates of the upper left corner and lower right corner of the real box. d1 and d2 are the distances between the upper left corner and lower right corner of the two boxes respectively.

3. the limestone particle size detection method based on YOLO-ADM according to claim 2, is characterized in that, The calculation process of the I-IoU loss function includes: Define the coordinates of the upper left and lower right corners of the real box: Where, is the center point of the ground truth box, w gt , h gt are the width and height of the ground truth box respectively; Then, define the top left and bottom right coordinates of the predicted bounding box: Among them, (x c ,y c ) is the center point of the predicted bounding box, w and h are the width and height of the predicted bounding box respectively; Next, calculate the intersection and union areas: Finally, calculate I-IoU: LI+IoU=1-Sin / Sun.

4. the limestone particle size detection method based on YOLO-ADM according to claim 1, is characterized in that, The calculation process of step S9 is: S91, the predicted frame coordinates obtained in S8 are combined with the camera focal length f, the straight-line distance d between the camera and the raw ore l Or the vertical distance d between the camera and the raw ore s , the shooting angle θ parameter of the lens to the raw ore, the pixel size d in the image p Convert to actual size d a The specific calculation formula is as follows: S92. When calculating the particle size of the raw ore, the pixel size in the image is the maximum geometric pixel distance between the coordinates of the diagonal points of the identification frame of the ore object. This is substituted into the calculation formula in S91 to obtain the size of each raw ore particle. The specific calculation formula is as follows: Maximum geometric distance d max The calculation is as follows: Where: d1, d2 - the geometric pixel distance between the diagonal vertices of the mineral identification frame; —The pixel coordinates of the upper left corner of the recognition box; —The pixel coordinates of the lower right corner of the recognition box; —The pixel coordinates of the upper right corner of the recognition box; —The pixel coordinates of the lower left corner of the recognition box; After conversion, the ore particle size is obtained as follows: Where: μ is the raw ore material conversion coefficient.

Images