A method for identifying particle size of broken rocks based on machine vision

Abstract

The present invention discloses a method for identifying the particle size of crushed rocks based on machine vision, comprising the following steps: S1, obtaining a gravel pile image in a quarry; S2, based on a lightweight edge segmentation model, combining an attention-fused deformable convolution module and an attention residual shrinkage unit, and replacing the maximum pooling layer with a dilated convolution layer, to obtain an improved lightweight edge segmentation model; S3, segmenting the gravel pile image, distinguishing the gravel edges from the background in the gravel pile image, and obtaining a gravel edge map; S4, using a table lookup method to refine the gravel edge map, and combining area filtering with an area filter to refine the edges and eliminate interference edges; S5, extracting the gravel edge contour from the gravel edge map, calculating the area and perimeter of the gravel, and finally obtaining the gravel particle size distribution. The detection method of the present invention significantly reduces the computational complexity while maintaining high accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a method for identifying the particle size of broken rocks based on machine vision. Background Art

[0002] Currently, research on the application of visual technology in gravel particle size segmentation has yielded substantial results. These studies have not only improved the efficiency of gravel processing but also provided strong support for the intelligent development of the gravel particle size detection industry. Scholars have conducted research and exploration in three main areas: first, accurate image algorithms for gravel contour detection; second, reasonable evaluation models that convert two-dimensional gravel information into three-dimensional information and then evaluate particle size distribution; and third, new technologies including neural networks, deep learning, and genetic algorithms. These methods help to understand the distribution of gravel particle size in different scenarios, but most rely too heavily on low-level visual information from image pixels and manual debugging to select the optimal threshold parameters. When images are noisy, complex, and have poor contrast, processing is still cumbersome. Furthermore, the segmentation process is inevitably subject to human influence, resulting in excessive segmentation errors and poor generalization performance of the segmentation model.

[0003] As the application of artificial intelligence (AI) in the gravel field matures, the focus of gravel size distribution detection in gravel pile images is shifting towards deep learning methods. For example, by introducing a residual structure based on DUNet, a new gravel image segmentation model (RDU-Net) was proposed, which significantly improves segmentation accuracy and exhibits better generalization. To extract features of gravel of varying sizes and shapes, residual deformable convolution was introduced into the HED model. This learning method learns image features of varying sizes and shapes, achieving accurate segmentation of gravel and further validating the feasibility of deep learning image processing for gravel size distribution. However, these deep learning methods still suffer from low accuracy and high computational complexity. Summary of the Invention

[0004] In order to solve the problems of low detection accuracy and excessive computational complexity in the existing technology for gravel particle size recognition, the present invention proposes a crushed rock particle size recognition method based on machine vision. A deformable convolution module with fused attention is introduced into the backbone network, and a dilated convolution is used to expand the receptive field. Combined with the noise processing capability of the efficient attention residual shrinkage unit, the accuracy and efficiency of gravel particle size recognition are improved to solve the above problems.

[0005] This application discloses a method for identifying the particle size of broken rocks based on machine vision, comprising the following steps:

[0006] S1. Acquire an image of a gravel pile in a quarry;

[0007] S2. Based on the lightweight edge segmentation model, we combine the attention-fused deformable convolution module and the attention residual shrinkage unit, and use the hole convolution layer to replace the maximum pooling layer to obtain an improved lightweight edge segmentation model;

[0008] S3, using the improved lightweight edge segmentation model obtained in S2 to segment the gravel pile image, distinguishing gravel edges from the background in the gravel pile image, and obtaining a gravel edge map;

[0009] S4, using a table lookup method to refine the edge of the gravel edge map, and combining it with an area filter to perform area filtering to refine the edge and eliminate interference edges;

[0010] S5. Extract the edge contour of the gravel from the gravel edge map, calculate the area and perimeter of the gravel, and finally obtain the particle size distribution of the gravel.

[0011] Preferably, the rubble pile images are from different scenes including construction sites and mines, and are taken under different lighting, angles and backgrounds.

[0012] Preferably, the attention-fused deformable convolution module is composed of a deformable convolution module in parallel with a hybrid attention module.

[0013] Preferably, the attention residual shrinkage unit uses one-dimensional convolution to replace the soft threshold of the residual shrinkage unit, and introduces an attention mechanism with spatial weights at the identity mapping of the residual shrinkage unit.

[0014] Preferably, the improved lightweight edge segmentation model structure is as follows:

[0015] A deformable convolution module with fused attention is added before the second and third modules of the lightweight edge segmentation model, an attention residual shrinkage unit is added before the context-aware fusion block of the lightweight edge segmentation model, and the maximum pooling layer of the lightweight edge segmentation model is replaced by a void convolution layer.

[0016] Preferably, said S3 comprises the following steps:

[0017] S31, performing data annotation, data preprocessing, and data enhancement on the acquired gravel pile image dataset;

[0018] S32, dividing the data set obtained in S31 into a training set and a test set, and using the data of the training set to train the improved lightweight edge segmentation model, and selecting the Dice coefficient loss as the loss function;

[0019] S33. Use the trained model to segment the images in the test set and evaluate the model performance by calculating the difference between the segmentation result and the true mask. The evaluation indicators include pixel accuracy, IoU and Dice coefficient.

[0020] S34. Apply the improved lightweight edge segmentation model to the actual gravel pile monitoring task.

[0021] Preferably, the step of refining the edge of the gravel edge map using a table lookup method comprises the following steps:

[0022] Binarize the gravel edge image to obtain a binary image, and count each edge pixel point and its neighborhood in the binary image;

[0023] Traverse the entire image and divide the edge pixels into skeleton points and boundary points. Keep the skeleton points and delete the boundary points to obtain the edge skeleton map.

[0024] Preferably, the area filtering performed in combination with the area filter comprises the following steps:

[0025] The edge skeleton graph obtained by the table lookup method is inverted, and the area of ​​all connected domains of the inverted image is counted. and the number of connected domains ,in ;

[0026] Setting the threshold , if the area of ​​the connected domain is less than the threshold, that is , then this connected domain is inverted, and finally all edge branches in the edge skeleton are removed.

[0027] Preferably, the S5 comprises the following steps:

[0028] Based on the gravel edge map, the findContours function of OpenCV is used to accurately find the gravel contours in the image, so that each gravel forms an independent connected domain;

[0029] Use the minAreaRect function to obtain the minimum enclosing rectangle parameters of each gravel;

[0030] The area of ​​each connected domain is calculated using the contourArea function, the perimeter of each connected domain is calculated using the arcLength function, and the diameter of the connected domain is calculated using the area and perimeter of the connected domain.

[0031] According to the pixel size of the obtained area, perimeter and diameter, the actual size of the gravel is converted according to the proportional coefficient, and the number of gravels in each particle size is counted to obtain the particle size distribution of the gravel pile.

[0032] Beneficial effects of the present invention:

[0033] (1) Lightweight neural network design: The lightweight neural network used in this invention significantly reduces the computational complexity and resource consumption of the model, making gravel particle size detection more efficient. Compared with traditional heavy networks, the detection method of this invention significantly reduces the computational workload while maintaining high accuracy.

[0034] (2) Image segmentation algorithm optimization: The present invention improves the accuracy of gravel pile image segmentation by optimizing the image segmentation algorithm. Under the same conditions, the segmentation accuracy is significantly improved compared with the existing technology.

[0035] (3) Real-time monitoring capability: The present invention can complete the particle size analysis of the gravel pile image in a short time, while the existing technology takes too long.

[0036] (4) Environmental adaptability: The network model of the present invention has better adaptability to different lighting and weather conditions. In experiments simulating different environmental conditions, the system of the present invention maintained stable performance under various conditions, while the performance of the existing technology decreased significantly under certain conditions.

[0037] (5) Easy to deploy and maintain: Due to the lightweight model, the system of the present invention is easier to deploy in resource-constrained environments and has low maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of a method for identifying particle size of broken rocks based on machine vision according to an embodiment of the present invention;

[0039] Figure 2 Schematic diagram of the deformable convolution module structure according to an embodiment of the present invention;

[0040] Figure 3 Schematic diagram of convolution sampling position according to an embodiment of the present invention;

[0041] Figure 4 Schematic diagram of the structure of a deformable convolution module integrating attention according to an embodiment of the present invention;

[0042] Figure 5 This is a schematic diagram of the residual shrinkage unit structure of an embodiment of the invention;

[0043] Figure 6 This is a schematic diagram of the structure of the attention residual shrinkage unit according to an embodiment of the present invention;

[0044] Figure 7 Schematic diagram of dilated convolution with different expansion coefficients according to an embodiment of the present invention;

[0045] Figure 8 Schematic diagram of the structure of an improved lightweight edge segmentation model according to an embodiment of the present invention;

[0046] Figure 9 This is a flow chart of edge thinning according to an embodiment of the present invention;

[0047] Figure 10 This is a flowchart of area filtering according to an embodiment of the present invention;

[0048] Figure 11 Schematic diagram of edge image post-processing results according to an embodiment of the present invention;

[0049] Figure 12 A schematic diagram of extracting gravel characteristic parameters according to an embodiment of the present invention;

[0050] Figure 13 Schematic diagram of qualitative analysis of example images of a rubble pile dataset using various algorithm models according to an embodiment of the present invention;

[0051] Figure 14 Schematic diagram of qualitative analysis of low-quality images according to an embodiment of the present invention;

[0052] Figure 15 The training loss change curves of the algorithms in the embodiments of the present invention are as follows;

[0053] Figure 16 1 is a curve showing the change in training accuracy of each training algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the objectives, technical solutions and advantages of this application more clear, the application is further described in detail below with reference to the accompanying drawings and examples.

[0055] The embodiment of the present application discloses a method for identifying the particle size of broken rocks based on machine vision, the process of which is as follows: Figure 1 As shown, the following steps are included:

[0056] S1. Obtain images of gravel piles in a quarry. The first task in gravel pile image segmentation is to prepare appropriate training data. Gravel pile images come from various scenarios, including construction sites and mines, and are captured under different lighting, angles, and backgrounds to obtain a diverse dataset.

[0057] S2. Gravel images contain severely stacked and adhered rubble, resulting in varying shapes and sizes, complex and chaotic textures, and poor contrast between the rubble and the image background due to harsh image acquisition environments and uneven lighting. These factors can all lead to suboptimal segmentation results. Directly using a traditional lightweight dense CNN for edge detection (LDC) for edge segmentation in rubble images results in numerous false edges, noise, and loss of detail.

[0058] Based on this, this embodiment proposes an improved lightweight edge segmentation model, which introduces a deformable convolution module with fused attention into the backbone network to accurately learn the detailed features of gravel in the gravel pile. In view of the fact that spatial acuity will be reduced due to the expansion of the receptive field, the conventional pooling layer is replaced by a dilated convolution to achieve the expansion of the receptive field and eliminate the interference of pseudo edges without affecting the resolution of the image. At the same time, before the fusion stage of the four intermediate edge images, the attention residual shrinkage unit is used to solve the noise interference problem and improve the feature recognition ability. Therefore, the improved lightweight edge segmentation model aims to avoid as many bit edges and noise interference as possible under the premise of achieving high-precision judgment of the gravel shape, thereby enhancing the edge segmentation effect of the gravel pile image.

[0059] like Figure 2 As shown, the core components of Deformable Convolution include offset convolution layers, standard convolution layers, batch normalization layers, and activation layers. The offset convolution layers enable it to better adapt to local variations in the input feature map. Standard convolution layers perform convolution operations on resized feature maps, effectively extracting rock features and improving accuracy and robustness. Batch normalization layers reduce internal covariate shift in feature maps, making feature extraction more stable and improving the model's generalization capabilities. This is particularly effective when processing rock images under varying lighting and background conditions. The nonlinear transformations in the activation layers enhance the model's representation of rock features, enabling it to better capture the complex shapes and textures of rock. Because standard convolution kernels have poor adaptability and generalization performance when processing complex gravel shapes and cannot effectively extract detailed features, deformable convolution introduces a learnable orientation offset into the convolution kernel. This offset allows the convolution kernel to have a larger range of variation during training, ensuring that the convolution region closely matches the shape of the target and more accurately extracts target features. Deformable convolution introduces an offset to standard convolution, enabling dynamic adjustment of the convolution kernel. The sampling points of the convolution kernel adaptively change as learning progresses, allowing it to learn information about gravel images of different sizes and shapes.

[0060] like Figure 3 As shown in the figure, the difference between the sampling areas of standard convolution and deformable convolution is shown, taking the convolution kernel of size 7×7 as an example. Figure 3 (a) represents the standard convolution, Figure 3 (b) shows a deformable convolution, and the offset is obtained by applying an offset convolution layer. The offset convolution layer should have the same spatial resolution and fill rate as the standard convolution layer and the same number of channels as the input feature map to extract features in the offset feature map while ensuring that each channel has corresponding offset information. Figure 3The standard convolution in (a) is evenly distributed according to the predefined network pattern. This sampling method works well when processing images with regular structures, but it may not be able to effectively capture these detailed features when processing complex geometric shapes and irregularly distributed features. Figure 3 The position of the deformable convolution in (b) can be dynamically adjusted, and can flexibly adapt to local changes in the input image, so as to better capture the edges, corners and texture features of the rock.

[0061] Since deformable convolution requires additional calculation of offset, it may introduce irrelevant context information, resulting in a less close connection between context information. Figure 4 As shown, in order to overcome this defect, this embodiment introduces a hybrid attention mechanism in parallel in the deformable convolution module, so that the model can not only search for features in a global range, but also focus on key information. The hybrid attention module is a lightweight module that only requires very few parameters to improve model performance. Specifically, the hybrid attention module applies one-dimensional convolution to extract features in the channel area, eliminating the adverse effects of dimensionality reduction on channel learning. In terms of spatial areas, global maximum pooling and average pooling techniques are used. After splicing, dimensionality reduction, normalization and channel output features are used to perform relevant dot products to obtain the overall weight distribution. The application of global maximum pooling and global average pooling helps to extract the most representative features and global features in the rock image, which is crucial for understanding the overall characteristics and distribution patterns of the rock. The use of splicing technology can make the feature vector contain local and global information of the rock image, which helps to more comprehensively describe the characteristics of the rock. After applying dimensionality reduction and normalization operations, the number of model parameters is effectively reduced, and the stability, convergence speed and generalization performance of the model are improved. The attention-fused deformable convolution module not only enhances the adaptability and flexibility of convolution, but also strengthens its ability to extract contextual information. It can be embedded into any existing convolutional neural network without modifying the network structure or training process.

[0062] In the harsh working environment of a quarry, due to the diffuse dust, uneven light distribution, and dense holes on the gravel surface, the gravel pile image is often filled with a large amount of noise and redundant information, which will have a negative impact on edge segmentation. The Residual Shrinkage Building Unit (RSBU) is designed to eliminate the negative impact of noise on the model. Its basic structure is as follows: Figure 5As shown in Figure 2, the core of the residual shrinkage unit lies in soft thresholding. The soft thresholding function is a nonlinear mapping that sets feature values ​​within a certain range to zero, thereby eliminating redundant or noise-related features. Specifically, the convolutional neural network first learns and masters a set of small positive thresholds. The soft thresholding function then sets input data with weights below the threshold to zero, and shrinks them towards zero when they are above the threshold, thus fitting unimportant redundant information to near zero. This method, known as soft thresholding, helps highlight key features in rock images while removing irrelevant information, improving recognition accuracy. Theoretically, since noise contributes less to edge segmentation, the corresponding weights are low and therefore set to zero, reducing noise interference on the model. The channel attention mechanism allows the network to adaptively emphasize or suppress different feature channels, thereby enhancing the most representative features in rock images. By learning the importance weights for each channel, this mechanism enables the network to focus more on features that contribute to rock recognition while ignoring less important features. The residual shrinkage unit mainly includes the soft threshold method and the channel attention mechanism. It first receives the input data, obtains the preliminary feature representation through feature extraction and residual learning, and then refines the features according to the set threshold. Redundant information is extracted through soft threshold processing. The specific expression is:

[0063]

[0064]

[0065]

[0066] in, Indicates the number of channels The output value of Indicates the number of channels The input value of Indicates the number of channels The threshold value, is the scale factor, is the output of the fully connected layer, is the height of the input feature map, is the width of the input feature map.

[0067] By using the residual shrinkage unit, a threshold can be adaptively set for each feature in the feature extraction stage, and the threshold can be continuously optimized through learning. The soft threshold method can set the feature value The features between are set to 0 to achieve the goal of reducing data noise.

[0068] Given that the two fully connected layers involve dimensionality reduction operations when learning soft thresholds and have low propagation efficiency, in order to improve the efficiency of the neural network and reduce the consumption of computing resources, this embodiment makes improvements based on the original residual shrinkage unit structure, such as Figure 6 As shown, one-dimensional convolution is used as an alternative to implement the network learning soft threshold function. Such an improvement can better utilize the interaction between multiple different feature layers and effectively avoid the impact of dimensionality reduction. In addition, based on its function, it can be regarded as a special attention mechanism, but it only focuses on the weight situation in the channel domain and does not consider the weight distribution in the spatial domain, that is, the importance of different spatial positions. This may lead to the loss of some information or interference from noise. Therefore, this embodiment introduces an attention mechanism with spatial weights at the identity mapping of the residual shrinkage unit, so that the improved attention residual shrinkage unit can take into account feature recognition in both the channel domain and the spatial domain, with only a slight increase in parameters and computational complexity. The effect of gravel feature extraction and recognition is significantly improved, key features are enhanced, and higher weights are assigned to key feature areas in the gravel image, so that the network can better capture the significant features of the gravel, and also suppress the interference of irrelevant information on the feature extraction of the gravel and irrelevant background and noise information. At the same time, there are significant improvements in both recognition accuracy and robustness.

[0069] Due to the diverse shapes and complex, chaotic textures of gravel, models that perform well in normal image detection often fail to fully identify gravel edges in gravel pile images, suffering from significant artifacts. Since the max pooling operation only retains the maximum value within each region and alters the positional relationships in the feature map, the model loses valuable feature information and suffers from reduced adaptability to translation invariance. The application of max pooling may have certain negative consequences, misclassifying non-edge points in gravel pile images as edge points, thereby increasing the probability of artifacts. It is worth noting that the primary goal of the pooling layer is dimensionality reduction, and its computational process is largely similar to that of the convolutional layer. Since errors in the pooling layer are directly fed back to the convolutional layer, this may affect the ability of the edge segmentation module to learn features, particularly high-level semantic features.

[0070] As the convolution kernel size and network depth continue to increase, the receptive field will also increase, which helps to extract image features. However, this will lead to a rapid increase in the number of parameters and the amount of calculation, which will bring a heavy burden to the calculation. The dilated convolution can be used to solve this problem. Figure 7 As shown in the figure, the dilated convolution introduces a dilation coefficient to adjust the interval between the internal elements of the convolution kernel, thereby expanding the receptive field without increasing the number of network model parameters. The size of the dilated convolution kernel is expressed as:

[0071]

[0072] in, is the final convolution kernel size, is the kernel size, is the expansion coefficient.

[0073] Therefore, in this embodiment, a dilated convolution layer is used as an alternative to the pooling layer to achieve a dimensionality reduction effect similar to that of the pooling layer. Moreover, by adjusting the dilation rate, the convolution kernel can move on the input image with a larger stride, thereby expanding the receptive field and enabling the network to capture a wider range of contextual information without sacrificing resolution. Compared with the traditional pooling layer, the dilated convolution can extract features without reducing the resolution of the feature map. This is crucial to maintaining the integrity of the detailed features of the gravel, as these detailed features are very important for the accurate recognition and classification of gravel. The dilated convolution can be used in combination with different dilation rates to achieve multi-scale feature extraction. This enables the network to capture both local details and global features of the gravel, improving the accuracy and robustness of recognition. During the training process, by inversely modifying the weight parameters, the network can better learn features under a larger receptive field to improve the detection accuracy of details such as image edges.

[0074] In order to solve the difficulties in the gravel edge segmentation process, this embodiment proposes an improved lightweight edge segmentation model based on the traditional LDC model with faster convergence and higher accuracy. The improved overall structure is as follows: Figure 8 In general, the improvements to the LDC model include the following aspects:

[0075] Add a deformable convolution module with fused attention, denoted by "AD". It is added before the second and third modules and shares information weights with deep convolution through skip connections.

[0076] Add the attention residual shrinkage unit (ARSBU). This unit is introduced before the context-aware fusion block performs feature map fusion to eliminate the effects of noisy or redundant data, allowing the model to obtain a more accurate gravel feature fusion map.

[0077] The dilated convolution layer replaces the maximum pooling layer, denoted by "DC". The pooling layers of the second and third modules in the LDC model are replaced by dilated convolution layers with a stride of 2, a kernel size of 3×3, and dilation factors of 2 and 3 respectively.

[0078] The obtained improved lightweight edge segmentation model network architecture is as follows:

[0079] Input layer: The input layer receives pre-processed image data, usually a normalized color image. The size of the input image is adjusted according to the task requirements.

[0080] Convolutional layer and pooling layer: The network extracts image features through multiple convolutional layers. Each convolution layer is followed by a pooling layer to gradually reduce the spatial dimension of the feature map.

[0081] Densely connected modules: The core feature of the network lies in its densely connected structure. The output of each convolutional layer not only serves as the input to the next convolutional layer but is also connected to the outputs of all previous layers. This effectively prevents information loss and enables the network to fully utilize the features learned by each layer, thereby improving segmentation accuracy. The output of each layer is passed to the downstream module, forming a dense feature fusion.

[0082] Lightweight convolution module: The network uses lightweight techniques such as depthwise separable convolution, splitting the traditional convolution operation into two independent convolution processes (depthwise convolution and pointwise convolution). This can significantly reduce the number of parameters, improve computational efficiency, and enable the network to run on resource-constrained devices.

[0083] Decoder module: The decoder section includes an upsampling operation to restore the low-resolution feature map to a segmentation map of the same size as the input image. This upsampling allows the network to generate accurate pixel-level predictions that can distinguish between rubble and background areas.

[0084] Output layer: The final output is a segmentation mask of the same size as the input image, with each pixel's value indicating whether it belongs to the gravel area or the background area. The output layer uses a softmax activation function to calculate the class probabilities.

[0085] S3. Segment the gravel pile image using the improved lightweight edge segmentation model obtained in S2 to distinguish gravel edges from the background in the gravel pile image and obtain a gravel edge map. Specifically, the following steps are included:

[0086] S31. Data preparation: Data preparation is performed on the acquired gravel pile image dataset, mainly including data labeling, data preprocessing, and data enhancement.

[0087] Data Annotation: Gravel images require precise pixel-level annotation, specifically labeling the specific outlines of the rubble. This is accomplished by generating a binary mask image, where the rubble areas are labeled as 1 and the background areas as 0. This annotation allows the improved lightweight edge segmentation model to learn how to distinguish rubble from other background areas in the image.

[0088] Data preprocessing: To ensure that the improved lightweight edge segmentation model can learn efficiently, all images need to be standardized, including image size unification, normalization, and denoising.

[0089] Data augmentation: Increase the diversity of training data and reduce model overfitting. In this example, the following methods of data augmentation are used:

[0090] Simple spatial geometric transformation: mainly through scaling, translation, rotation and mirroring, etc.

[0091] Color transformation: Use color space-based data enhancement techniques such as gamma transformation or histogram equalization to adjust brightness, enhance contrast, etc., to improve the effect of data enhancement.

[0092] Adding noise: Salt and pepper noise is added to some burst images to improve the generalization ability of the model.

[0093] Random cropping: Randomly crop the gravel pile image to 352×352 pixels.

[0094] S32, model training, divides the data set obtained in S31 into a training set and a test set, and uses the data of the training set to train the improved lightweight edge segmentation model.

[0095] The loss function used is the Dice coefficient loss. The Dice coefficient is a commonly used evaluation metric in image segmentation, with values ​​ranging from 0 to 1, where values ​​closer to 1 indicate better segmentation. By optimizing the overlap of segmentation results, the Dice coefficient loss effectively addresses the issue of uneven background in gravel pile images, especially when the target area is small.

[0096] During the training process, mini-batch gradient descent is used, combined with optimization methods such as momentum and learning rate decay. The loss is calculated for each image in the training dataset through forward propagation, and the network weights are adjusted through backpropagation to minimize the loss function.

[0097] S33, testing and evaluation, uses the trained model to segment the images in the test set. The testing process evaluates the model performance by calculating the difference between the segmentation results and the true mask. The evaluation indicators include:

[0098] Pixel Accuracy: This measures the proportion of pixels that the network correctly predicts. It is applicable when the background and target areas are evenly distributed.

[0099] IoU: IoU represents the ratio of the intersection and union of the predicted area and the true area. The larger the IoU, the better the prediction effect.

[0100] Dice coefficient: The Dice coefficient measures the segmentation accuracy. A higher value indicates that the prediction is closer to the true label.

[0101] S34. Practical Application: The improved lightweight edge segmentation model is applied to the actual gravel pile monitoring task. By accurately segmenting the gravel pile image, automated pile monitoring can be achieved and abnormalities in the pile can be identified in a timely manner.

[0102] S4. Use table lookup method to refine the edge of the gravel edge map, and combine it with area filter to perform area filtering to refine the edge and eliminate interference edges.

[0103] To accurately calculate the characteristic parameters of the gravel, an improved lightweight edge segmentation model is first used to segment the gravel pile image, distinguishing the gravel edges from the background in the image, and ultimately generating a gravel edge map. However, due to the heavy accumulation of crushed gravel and the complex and chaotic texture, the edges in the output of the lightweight edge segmentation model are still relatively coarse, and difficult-to-remove interfering edges are inevitably present in the gravel interior. To address this issue, post-processing of the edge segmentation results is required. This embodiment uses a table lookup method to extract the edge skeleton, combined with an area filter method to refine the edges and eliminate interfering edges.

[0104] The table lookup method is a method used to refine binary images. First, the neighborhood of each edge pixel is counted, and the entire image is traversed to divide it into two different categories: skeleton points and boundary points. Finally, the skeleton points are retained and the boundary points are deleted to achieve the purpose of edge refinement. The specific process is as follows: Figure 9 The gravel edge map is binarized to obtain a binary image. Each edge pixel and its neighborhood in the binary image are counted. The entire image is traversed to divide the edge pixels into skeleton points and boundary points. The skeleton points are retained and the boundary points are deleted to obtain the edge skeleton map.

[0105] For some areas inside the gravel, there are still interference edges. If morphological operations are used, some non-negligible effects may be introduced. Therefore, in order to eliminate the larger interference edges or noise points in the image, this embodiment adopts the area filtering method. The main idea is to set a threshold based on the number of pixels contained in each connected domain in the image, and delete the connected domains that are smaller than the threshold, thereby achieving the purpose of removing irrelevant small areas. The specific process is as follows: Figure 10 The edge skeleton graph obtained by the table lookup method is inverted, and the area of ​​all connected domains of the inverted image is counted. and the number of connected domains ,in ; Set threshold , if the area of ​​the connected domain is less than the threshold, that is , then this connected domain is inverted, the entire image is inverted, and finally all edge branches in the edge skeleton are removed.

[0106] The gravel edge map output by the improved lightweight edge segmentation model was subjected to edge refinement and area filtering. The results are shown in the figure below. Figure 11 As shown, Figure 11 (a) is the edge image, Figure 11 Middle (b) is the image after edge refinement. Figure 11 (c) shows the area-filtered image. Through a series of post-segmentation processing, the rubble pile image successfully extracts the edge skeleton and removes isolated edge branches, achieving more ideal results.

[0107] S5. Extract the edge contour of the gravel from the gravel edge map, calculate the area and perimeter of the gravel, and finally obtain the particle size distribution of the gravel.

[0108] Figure 12 The statistical process of the characteristic parameters of three groups of gravel piles is shown. Figure 12 (a) is a gravel pile image. Figure 12 (b) is the gravel outline. Figure 12 (c) shows the connected domain of the gravel. First, based on the gravel edge map, the OpenCV findContours function is used to precisely locate the gravel contours in the image, so that each gravel forms an independent connected domain. Next, the minAreaRect function is used to obtain the minimum bounding rectangle parameters of each gravel. Finally, the contourArea function is used to calculate the area of ​​each connected domain, and the arcLength function is used to calculate the perimeter of each connected domain. The dimensions of the minimum bounding rectangle reflect the actual size of the gravel, where the length and width of the rectangle correspond to the length and width of the gravel, respectively. The area of ​​the gravel can be obtained by calculating the pixel area within the connected domain, while the perimeter of the gravel is determined by measuring the pixel length of the connected domain boundary contour. The diameter of the gravel can be calculated from the area and perimeter. The pixel size of the above parameters is converted to the actual size of the gravel using a scaling factor. Finally, the number of gravels in each particle size is counted to obtain the particle size distribution of the gravel pile.

[0109] In a specific embodiment, in order to evaluate the improved LDC model, the improved LDC model and other current mainstream image edge segmentation models were trained in a self-built gravel pile dataset, and the corresponding weight parameters were saved. BDCN-B2 in the algorithm involved refers to the first two blocks of BDCN. This embodiment aims to select mainstream lightweight edge segmentation methods for comparative study. Except for HED and DexiNed, the parameter size of all models is controlled within 1MB. In order to ensure the fairness of the comparison results, this embodiment conducts comparative analysis from both qualitative and quantitative aspects after ensuring that all model training and learning processes have achieved good convergence effects.

[0110] Qualitative analysis. Figure 13 The visual comparison of different edge segmentation network models after training on a self-built gravel image dataset is shown. Figure 13 (a) is the image ID, which are Image-1, Image-2 and Image-3 respectively. Figure 13 (b) is the original image. Figure 13 (c) in the middle is the label image, Figure 13 (d) is the image detected by the HED algorithm. Figure 13 (e) is the image detected by the PiDiNet algorithm. Figure 13 (f) in the middle is the image detected by TIN algorithm. Figure 13 (g) is the image detected by the BDCN-B2 algorithm. Figure 13 The image in (h) is detected by the DexiNed algorithm. Figure 13 (i) is the image detected by the LDC algorithm. Figure 13 (j) in the middle is the image detected by the improved LDC algorithm. As can be seen from the figure, the edge positioning detected by the HED, PiDiNet, TIN and BDCN-B2 algorithms is relatively accurate, but they are seriously affected by noise, the edges obtained are discontinuous, and there is a phenomenon of misidentifying the surface texture of gravel as the edge of gravel, and the overall visual effect is relatively stiff; while DexiNed and LDC are accurate in detecting the edges of large pieces of gravel with complex textures and are less affected by noise, but the edge detection effect is poor in areas with low contrast at the edge of the gravel, and the details are not rich enough. In contrast, the improved LDC algorithm proposed in this application can obtain clearer and more continuous edges, has the ability to eliminate noise and accurately locate significant edges, and effectively reduces the generation of pseudo-edges. In areas where gravel seriously overlaps, the algorithm of this embodiment can accurately identify the edge of the gravel, and the predicted edge map is rich in details, has a good visual effect for the human eye, and is closer to the label map of the image.

[0111] Figure 14 The comparative results of the prediction between the LDC algorithm and the improved LDC algorithm proposed in this application on low-quality images with harsh environments, high dust content, and insufficient lighting are shown. Figure 14 (a) is the original image. Figure 14 (b) is the image predicted by the LDC algorithm. Figure 14(c) in the middle is the image predicted by the improved LDC algorithm. Under unfavorable conditions such as insufficient image illumination, low contrast, and blurred edges, the edges detected by traditional LDC have a large number of false detections and missed detections, the edge image is relatively rough, and the edge recognition of low background contrast is incomplete. Its performance is poor under harsh conditions. In contrast, the improved LDC algorithm proposed in this application is more sensitive to small edges than the LDC algorithm model, can detect smaller gravel pieces, and more accurately capture the edge information in the gravel pile image, thereby obtaining clearer and more accurate edges. In general, the improved LDC algorithm proposed in this application performs better in the detection of gravel edges in gravel piles, and can better handle the task of gravel edge segmentation in gravel piles.

[0112] Quantitative comparison. In order to objectively measure the effectiveness of the edge detection results of each model, this embodiment performs the same operation on the gravel pile image when performing specific tests on different models, and uses the same evaluation code to calculate all edge segmentation methods. As shown in Table 1, the edge detection effects of the algorithm proposed in this application and other algorithms are quantitatively compared for Image-1~Image3. The results show that compared with the HED and DexiNed edge detection algorithms with more than 1M parameters, the performance of the improved LDC algorithm proposed in this application exceeds the HED algorithm as a whole, and the AP is improved by 3.1%. The algorithm proposed in this application has reached the maximum value for the three evaluation indicators in the lightweight algorithm model. Compared with the traditional LDC model, only a small number of parameters are added to achieve significant improvement in the performance of key indicators such as ODS, OIS and AP. This shows that the algorithm model proposed in this application effectively improves the accuracy and stability of edge detection while maintaining the model complexity and computational efficiency. The edge pixels detected by the model are highly consistent with the manually labeled edge images.

[0113] Table 1 Objective quantitative comparison results of example images of the rubble pile dataset

[0114]

[0115] In a specific embodiment, to further analyze the effectiveness of different improved modules on edge detection results, ablation experiments were conducted on a self-constructed test set of rubble pile images. The experimental results are shown in Table 2, where "√" indicates the use of the corresponding improved module, and "×" indicates the absence of the corresponding improved module. All algorithms in the table used the same data augmentation and evaluation metric codes. AD represents the attention-integrated deformable convolution module, ARSBU represents the attention residual shrinkage unit, and DC refers to the use of dilated convolution instead of max pooling. Comparing Experiments 4 and 5, the introduction of the attention-integrated deformable convolution module, while sacrificing only 125KB of parameters, resulted in significant performance improvements in the evaluation metrics of ODS and OIS by 2.8% and 3.1%, respectively, and AP by 2.9%. Comparing the results of Experiments 3 and 5, the use of the attention residual shrinkage unit improved the algorithm's performance, with increases of 2.5% in ODS, 2.6% in OIS, and 1.6% in AP. Furthermore, the results of Experiments 2 and 5 show that replacing max pooling with dilated convolution also improves algorithm performance. Overall, the ODS of the algorithm proposed in this application is improved by 3.4%, OIS is improved by 3.6%, and AP is improved by 2.8%.

[0116] like Figure 15 and Figure 16 As shown in the figure, after 20 cycles of training, each algorithm model basically tends to converge. By comparing the loss functions and accuracy change trends of different algorithm models in the ablation test, it can be seen that the algorithm proposed in this application converges faster, and its accuracy at the beginning of training is much higher than other algorithms. The traditional LDC model finally converges to the largest loss value and its accuracy is much lower than other algorithms. AD+DC performs well in training loss, but its accuracy is not as good as the algorithm proposed in this application.

[0117] Table 2 Ablation test results of improved modules

[0118]

[0119] In a specific example, cumulative distribution function and frequency distribution function analysis were used to comprehensively analyze the characteristic parameters of three groups of gravel piles (a, b, and c). The particle sizes of the gravel were calculated, resulting in seven different particle sizes. The particle size distribution data obtained by manual screening and the statistical method proposed in this application were then compared and analyzed to comprehensively evaluate the characteristics of the gravel and gain a deeper understanding of its particle size properties. The comparative results are shown in Tables 3-5.

[0120] Table 3 Comparison of particle size distribution of gravel pile a

[0121]

[0122] Table 4 Comparison of particle size distribution of gravel pile b

[0123]

[0124] Table 5 Comparison of particle size distribution of gravel pile c

[0125]

[0126] As can be seen from Tables 3 to 5, when the method proposed in this application is used to perform particle size statistics, the average error of gravel pile a in each particle size range is 1.10%, and the average errors of gravel piles b and gravel piles c in each particle size range are 2.43% and 2.25%, respectively. At low particle sizes, the detection results of the three groups of gravel piles are slightly larger than the screening results, with a certain degree of over-segmentation, and the errors are relatively large. However, as the particle size increases, the detection results become more and more accurate, and the errors of each particle size range are maintained below 5%. Overall, the capabilities demonstrated by the method proposed in this application in these three groups of experiments are satisfactory. This is not only reflected in the high-precision particle size distribution statistics, but also in the ability to maintain accuracy when processing larger particle sizes, providing reliable data support for practical applications.

[0127] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for identifying the particle size of broken rocks based on machine vision, characterized in that: The following steps are involved: S1. Acquire an image of a gravel pile in a quarry; S2. Based on the lightweight edge segmentation model, we combine the attention-fused deformable convolution module and the attention residual shrinkage unit, and use the hole convolution layer to replace the maximum pooling layer to obtain an improved lightweight edge segmentation model; The attention-fused deformable convolution module is composed of a deformable convolution module in parallel with a hybrid attention module; The attention residual shrinkage unit uses one-dimensional convolution to replace the soft threshold of the residual shrinkage unit, and introduces an attention mechanism with spatial weights at the identity mapping of the residual shrinkage unit; The structure of the improved lightweight edge segmentation model is as follows: a deformable convolution module with fusion attention is added before the second and third modules of the lightweight edge segmentation model, an attention residual shrinkage unit is added before the context-aware fusion block of the lightweight edge segmentation model, and a hole convolution layer is used to replace the maximum pooling layer of the lightweight edge segmentation model; S3, using the improved lightweight edge segmentation model obtained in S2 to segment the gravel pile image, distinguishing gravel edges from the background in the gravel pile image, and obtaining a gravel edge map; S4, using a table lookup method to refine the edge of the gravel edge map, and combining it with an area filter to perform area filtering to refine the edge and eliminate interference edges; S5. Extract the edge contour of the gravel from the gravel edge map, calculate the area and perimeter of the gravel, and finally obtain the particle size distribution of the gravel.

2. The method for identifying the particle size of broken rocks based on machine vision according to claim 1, characterized in that: The rubble pile images come from different scenes including construction sites and mines, and are taken under different lighting, angles and backgrounds.

3. The method for identifying the particle size of broken rocks based on machine vision according to claim 2, characterized in that: The S3 includes the following steps: S31, performing data annotation, data preprocessing, and data enhancement on the acquired gravel pile image dataset; S32, dividing the data set obtained in S31 into a training set and a test set, and using the data of the training set to train the improved lightweight edge segmentation model, and selecting the Dice coefficient loss as the loss function; S33. Use the trained model to segment the images in the test set and evaluate the model performance by calculating the difference between the segmentation result and the true mask. The evaluation indicators include pixel accuracy, IoU and Dice coefficient. S34. Apply the improved lightweight edge segmentation model to the actual gravel pile monitoring task.

4. The method for identifying the particle size of broken rocks based on machine vision according to claim 3, characterized in that: The edge thinning of the gravel edge map using the table lookup method comprises the following steps: Binarize the gravel edge image to obtain a binary image, and count each edge pixel point and its neighborhood in the binary image; Traverse the entire image and divide the edge pixels into skeleton points and boundary points. Keep the skeleton points and delete the boundary points to obtain the edge skeleton map.

5. The method for identifying the particle size of broken rocks based on machine vision according to claim 4, characterized in that: The area filtering performed in combination with the area filter comprises the following steps: The edge skeleton graph obtained by the table lookup method is inverted, and the area of ​​all connected domains of the inverted image is counted. and the number of connected domains ,in ; Setting the threshold , if the area of ​​the connected domain is less than the threshold, that is , then this connected domain is inverted, and finally all edge branches in the edge skeleton are removed.

6. The method for identifying the particle size of broken rocks based on machine vision according to claim 5, characterized in that: The S5 comprises the following steps: Based on the gravel edge map, the findContours function of OpenCV is used to accurately find the gravel contours in the image, so that each gravel forms an independent connected domain; Use the minAreaRect function to obtain the minimum enclosing rectangle parameters of each gravel; The area of ​​each connected domain is calculated by the contourArea function, the perimeter of each connected domain is calculated by the arcLength function, and the diameter of the connected domain is calculated by the area and perimeter of the connected domain; According to the pixel size of the obtained area, perimeter and diameter, the actual size of the gravel is converted according to the proportional coefficient, and the number of gravels in each particle size is counted to obtain the particle size distribution of the gravel pile.

Images