Sand and gravel aggregate type identification method and system
By introducing MHSA and CBAM attention mechanisms into the ResNet18 network, combined with data augmentation and transfer learning, the problem of low accuracy in concrete aggregate recognition was solved, and efficient image recognition results were achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JIANGNAN UNIV
- Filing Date
- 2023-11-14
- Publication Date
- 2026-07-24
AI Technical Summary
Existing technologies have low accuracy in classifying and identifying concrete aggregates, and image processing technology is poorly applicable in production environments with unstable lighting, resulting in high time consumption and increased costs for manual identification.
By combining data augmentation techniques with an improved ResNet18 network model, and adding MHSA and CBAM attention mechanisms, the generalization ability and recognition accuracy of the model are improved through transfer learning.
It significantly improves the recognition accuracy of concrete aggregate images, shortens training time, and reduces the need for manual recognition.
Smart Images
Figure CN117409257B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to a method and system for identifying types of sand and gravel aggregates. Background Technology
[0002] Concrete is a widely used building material, composed of a mixture of various sand and gravel aggregates. In automated concrete production, different sand and gravel aggregates need to be fed into different batching bins via rotary feeders and robotic arms. To ensure that the correct aggregates enter the batching bins, each type of aggregate needs to be identified upon initial entry. Currently, in most concrete production processes, inspectors rely primarily on sensory identification and other methods to determine the type of aggregates entering the bins. However, due to the wide variety of aggregate types, manual identification is time-consuming and increases labor costs for concrete processing companies. Therefore, it is essential to research an automated aggregate type identification method that meets actual production needs.
[0003] Currently, some research institutions and enterprises in China utilize digital processing and machine vision technologies to identify sand and gravel aggregates. On one hand, image processing techniques are used to extract the aspect ratio of coarse concrete aggregates, thus achieving the purpose of coarse concrete aggregate detection. On the other hand, digital image processing techniques are used to segment rock particles and pores, thereby extracting the contour information of rock particles, which has reference value for sand and gravel aggregate identification. However, digital image processing technology is highly sensitive to lighting conditions. In reality, the concrete production process and environment are complex and the lighting is unstable, so simple digital image processing technology cannot be directly applied to actual production and is only suitable for experimental environments. Convolutional Neural Networks (CNNs), due to their weight sharing and local vision characteristics, have strong robustness and are now widely used in image recognition. Existing technologies include training the SqueezeNet network model to achieve the classification of rock thin section images in a short time and achieve good classification results. Furthermore, research has been conducted on the quality problem of sand and gravel aggregates using a large amount of physical property data, proposing a sand and gravel aggregate classification model based on convolutional neural networks.
[0004] Due to the low accuracy of classification and identification of concrete aggregates in existing technologies, no effective solution has yet been proposed.
[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0006] In view of the aforementioned shortcomings and deficiencies of existing technologies, this application provides a method and system for identifying aggregate types. This method combines data augmentation techniques and an improved ResNet18 network model with transfer learning on the ImageNet image dataset, which can accelerate learning speed, improve performance, enhance generalization ability, and better adapt to the characteristics of concrete aggregate images. Experiments demonstrate that this method can significantly improve the recognition accuracy of concrete aggregate images.
[0007] This application mainly includes the following aspects:
[0008] In a first aspect, the present invention provides a method for identifying the type of sand and gravel aggregate, comprising:
[0009] Collect initial data images of different types of sand and gravel aggregates;
[0010] The initial data image is preprocessed to construct the target dataset of the sand and gravel aggregate; the target dataset includes a training dataset and a validation dataset;
[0011] Add MHSA attention mechanism and CBAM attention mechanism to the initial sand and gravel aggregate type identification model to obtain the improved sand and gravel aggregate type identification model;
[0012] The improved sand and gravel aggregate type identification model is trained using the training dataset until the accuracy of sand and gravel aggregate type identification in the training results reaches a preset range, thereby obtaining the target sand and gravel aggregate type identification model.
[0013] The image of the concrete aggregate to be identified is input into the target aggregate type identification model, and the identification result of each type of aggregate in the image is output.
[0014] In one embodiment of the present invention, preprocessing the initial data image to construct the target dataset of the sand and gravel aggregate includes:
[0015] The initial dataset is augmented to obtain an augmented dataset;
[0016] Redundant information in the images of the augmented dataset is removed to obtain the target dataset;
[0017] The target dataset is divided into a training dataset and a validation dataset according to a preset ratio.
[0018] In one embodiment of the present invention, an MHSA attention mechanism and a CBAM attention mechanism are added to the initial sand and gravel aggregate type identification model to obtain an improved sand and gravel aggregate type identification model, including:
[0019] Obtain an initial sand and gravel aggregate type identification model;
[0020] An MHSA attention mechanism is added after the third convolution layer of the initial sand and gravel aggregate type identification model, and a CBAM attention mechanism and an MHSA attention mechanism are added after the fourth convolution layer of the initial sand and gravel aggregate type identification model to obtain an improved sand and gravel aggregate type identification model.
[0021] In one embodiment of the present invention, after obtaining the target sand and gravel aggregate type identification model, the method further includes:
[0022] The target sand and gravel aggregate type identification model was validated using the validation dataset.
[0023] If the accuracy of the verification result reaches the preset range, then the target sand and gravel aggregate type model is output.
[0024] If the accuracy of the verification results does not reach the preset range, the improved sand and gravel aggregate type identification model will continue to be trained until the accuracy of the verification results reaches the preset range.
[0025] In one embodiment of the present invention, when training the improved sand and gravel aggregate type identification model, the output of the preset neurons in the model is set to zero to increase the robustness of the model to the input, reduce overfitting, and improve the accuracy of the model in identifying the results.
[0026] Secondly, embodiments of this application also provide a sand and gravel aggregate type identification system, including a data acquisition module, an image processing module, a model improvement module, a model training module, and an image recognition module, wherein:
[0027] The data acquisition module is used to acquire initial data images of different types of sand and gravel aggregates;
[0028] The image processing module is used to preprocess the initial data image to construct the target dataset of the sand and gravel aggregate; the target dataset includes a training dataset and a validation dataset;
[0029] The model improvement module is used to add MHSA attention mechanism and CBAM attention mechanism to the initial sand and gravel aggregate type identification model to obtain the improved sand and gravel aggregate type identification model.
[0030] The model training module is used to train the improved sand and gravel aggregate type identification model using the training dataset until the sand and gravel aggregate type identification accuracy of the training result reaches a preset range, thereby obtaining the target sand and gravel aggregate type identification model.
[0031] The image recognition module is used to input the image of the concrete aggregate to be identified into the target aggregate type recognition model and output the recognition result of each type of aggregate in the image.
[0032] In one embodiment of the present invention, the image processing module is further configured to:
[0033] The initial dataset is augmented to obtain an augmented dataset;
[0034] Redundant information in the images of the augmented dataset is removed to obtain the target dataset;
[0035] The target dataset is divided into a training dataset and a validation dataset according to a preset ratio.
[0036] In one embodiment of the present invention, the model improvement module is further configured to:
[0037] Obtain an initial sand and gravel aggregate type identification model;
[0038] An MHSA attention mechanism is added after the third convolution layer of the initial sand and gravel aggregate type identification model, and a CBAM attention mechanism and an MHSA attention mechanism are added after the fourth convolution layer of the initial sand and gravel aggregate type identification model to obtain an improved sand and gravel aggregate type identification model.
[0039] In one embodiment of the present invention, a verification module is further included for:
[0040] The target sand and gravel aggregate type identification model was validated using the validation dataset.
[0041] If the accuracy of the verification result reaches the preset range, then the target sand and gravel aggregate type model is output.
[0042] If the accuracy of the verification results does not reach the preset range, the improved sand and gravel aggregate type identification model will continue to be trained until the accuracy of the verification results reaches the preset range.
[0043] In one embodiment of the present invention, the model training module is further configured to set the output of a preset neuron in the model to zero when training the improved sand and gravel aggregate type identification model, thereby increasing the robustness of the model to the input, reducing overfitting, and improving the accuracy of the model in identifying the results.
[0044] The technical solution of the present invention has the following advantages compared with the prior art:
[0045] This application improves traditional image recognition models by adding MHSA and CBAM attention mechanisms, resulting in a gradual increase in the accuracy of the target sand and gravel aggregate type recognition model. Specifically, ResNet18+CBAM improves the classification accuracy by 1.16% compared to ResNet18, indicating that the spatial and channel attention mechanisms added to ResNet18 by CBAM help the model better understand the input image and improve its feature extraction capabilities. ResNet18+CBAM+MHSA (added only in the fourth layer) improves the accuracy by 0.7% compared to adding only the CBAM mechanism, indicating that MHSA... HSA, through its global perspective, allows the model to better grasp the global features and relationships of an image, which helps improve the model's classification performance. Finally, compared with ResNet18+CBAM+MHSA (added only in the fourth layer), CM-ResNet18 shows improvements in accuracy, precision, recall, and F1-score of 0.73%, 0.58%, 0.45%, and 0.59%, respectively. This indicates that the multi-layered MHSA in CM-ResNet18 can gradually expand the receptive field and capture more complex image information. Combined with the CBAM mechanism, it enhances the model's feature extraction and expressive capabilities, thereby improving accuracy. Attached Figure Description
[0046] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:
[0047] Figure 1 A flowchart of a method for identifying sand and gravel aggregate types provided in an embodiment of this application is shown;
[0048] Figure 2 A schematic diagram of the target sand and gravel aggregate type identification model (CM-ResNet18 (CBAM and MHSA ResNet18)) provided in the embodiments of this application is shown.
[0049] Figure 3 A schematic diagram of the CBAM model structure provided in the embodiments of this application is shown;
[0050] Figure 4 A flowchart of the MHSA attention mechanism provided in an embodiment of this application is shown;
[0051] Figure 5 This paper illustrates a functional block diagram of a sand and gravel aggregate type identification system provided in an embodiment of this application. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.
[0053] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0054] In order to enable those skilled in the art to use the content of this application, and in conjunction with the specific application scenario of "efficient recognition of concrete aggregate classification images", the following implementation method is given. For those skilled in the art, the general principles defined herein can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application.
[0055] The methods described in this application can be applied to any scenario requiring efficient recognition of concrete aggregate classification images. This application does not limit specific application scenarios, and any efficient recognition scheme for concrete aggregate classification images provided in this application is within the scope of protection of this application.
[0056] To facilitate understanding of this application, the technical solutions provided in this application will be described in detail below with reference to specific embodiments.
[0057] This application relates to a method and system for identifying aggregate types, proposing a CM-ResNet18 model to achieve efficient identification of concrete aggregate classification images. First, the original dataset of concrete aggregates was significantly expanded. Then, the CM-ResNet18 model was combined with a transfer learning strategy. This allows the model to achieve high performance within a relatively small number of training iterations, significantly shortening the training time and resulting in higher accuracy in identifying aggregate types.
[0058] Figure 1 A flowchart of a method for identifying sand and gravel aggregate types provided in this application embodiment includes:
[0059] S101: Collect initial data images of different types of sand and gravel aggregates.
[0060] For example, taking the various sand and gravel aggregates of concrete before entering the silo as the identification objects, a total of 9 types of concrete sand and gravel aggregates were collected from the concrete production enterprise: 3.95 coarse gravel (the number before the name indicates density), 3.95 mineral powder, 4.3 coarse gravel, 4.3 mineral powder, 4.4 iron concentrate, 5.0 iron concentrate, fine stone, yellow sand, and bluestone.
[0061] S102: Preprocess the initial data image to construct the target dataset of the sand and gravel aggregate; the target dataset includes a training dataset and a validation dataset.
[0062] For example, the preferred experiment in this application uses the Hikvision MV-CA004-10UM / UC advanced area array camera, which employs a Sony IMX287 CMOS chip. This industrial camera has a resolution of 720 pixels × 540 pixels and a maximum frame rate of 526.5 fps. Using Hikvision's industrial camera, images of various types of concrete aggregates were acquired, totaling 7600 images, approximately 850 images per type.
[0063] In some possible implementations, in step S102, the initial data image is preprocessed to construct the target dataset of the sand and gravel aggregate, including:
[0064] The initial dataset is augmented to obtain an augmented dataset;
[0065] Redundant information in the images of the augmented dataset is removed to obtain the target dataset;
[0066] The target dataset is divided into a training dataset and a validation dataset according to a preset ratio.
[0067] For example, after acquiring images of concrete aggregates, the initial dataset needs to be augmented to enrich the distribution of the training dataset, remove redundant image information, and enhance the generalization ability of the model. At the same time, 30% of the target dataset is divided as a test dataset to evaluate the performance of the model.
[0068] Furthermore, the images were augmented using data augmentation techniques. These techniques included randomly flipping images horizontally and vertically, randomly adjusting image brightness, contrast, and saturation, and adding random noise or blurring, among more than ten other methods. Through data augmentation, the total number of training dataset samples was increased to 10,589, with approximately 1,470 images per class. To ensure the recognizability of the concrete aggregate images and to reduce image size to improve training speed and optimization, the image data was uniformly resized to 64 pixels × 64 pixels before being input into the network model.
[0069] S103: Add MHSA attention mechanism and CBAM attention mechanism to the initial sand and gravel aggregate type identification model to obtain the improved sand and gravel aggregate type identification model.
[0070] In some possible implementations, in step S103, the MHSA attention mechanism and the CBAM attention mechanism are added to the initial sand and gravel aggregate type identification model to obtain an improved sand and gravel aggregate type identification model, including:
[0071] Obtain an initial sand and gravel aggregate type identification model;
[0072] An MHSA attention mechanism is added after the third convolution layer of the initial sand and gravel aggregate type identification model, and a CBAM attention mechanism and an MHSA attention mechanism are added after the fourth convolution layer of the initial sand and gravel aggregate type identification model to obtain an improved sand and gravel aggregate type identification model.
[0073] For example, such as Figure 2 As shown, this application proposes a new target sand and gravel aggregate type identification model, namely the CM-ResNet18 (CBAM and MHSA ResNet18) identification model. The improved network model based on ResNet18 integrates the attention mechanisms of CBAM and MHSA. This improvement can make full use of the advantages of CBAM and MHSA, further improve the performance in image classification tasks, and make the network more expressive and generalizable.
[0074] Specifically, adding the MHSA attention mechanism after the third convolutional layer and the CBAM and MHSA attention mechanisms after the fourth convolutional layer can enhance the model's performance. The CBAM (Convolutional Block Attention Module) consists of two sub-modules: the Channel Attention Module (CAM) and the Spatial Attention Module (SAM). The Channel Attention Module focuses on meaningful features in the image, while the Spatial Attention Module focuses on the most informative regions. The CBAM combines channel and spatial attention, simultaneously focusing on both the channel and spatial dimensions of the feature map. Channel attention learns the importance of different channels, while spatial attention learns the importance of different spatial locations. This comprehensive approach allows the network to more accurately understand key information in the image. The CBAM model structure is as follows: Figure 3 As shown:
[0075] The Channel Attention (CAM) module processes the input feature map F(H×W×C) through global max pooling and global average pooling based on width and height, respectively, to obtain two 1×1×C feature maps. These are then fed into a two-layer neural network (MLP). The first layer has C / r neurons (r is the reduction rate) and uses ReLU activation. The second layer has C neurons. This two-layer neural network is shared. Afterward, a sigmoid activation operation is performed to generate the final channel attention feature, i.e., M in formula (1). c (F):
[0076]
[0077] In equation (1), F avg F is the result of global average pooling of the feature map. max This is the result after global max pooling.
[0078] The Spatial Attention (SAM) module uses the feature map F output by the Channel Attention module as the input feature map for this module.
[0079] First, perform channel-based global max pooling and global average pooling to obtain two feature maps of size H×W×1. Then, concatenate these two feature maps based on their channels. Next, perform a 7×7 convolution operation to reduce the dimensionality to H×W×1, and then use a sigmoid function to generate spatial attention features, i.e., M in formula (2). s (F):
[0080]
[0081] In equation (2), f represents the convolution operation, and [] represents the channel concatenation operation. Then, the vector and the input feature map of the module are multiplied to obtain the final generated feature.
[0082] Specifically, Multi-Head Attention (MHSA) is a neural network module that captures relationships and dependencies in an input sequence by computing multiple independent attention heads in parallel. Its core idea is to segment the input vector according to a specified number of heads and perform self-attention computation on each head separately. Each attention head can independently learn different feature representations, thus complementing and cooperating with each other, improving the model's expressive power. For example... Figure 4 The diagram illustrates a multi-head self-attention mechanism, where each head performs attention calculations on the input sequence. Finally, the outputs of each attention head are integrated through linear transformation and concatenation operations to obtain the final representation.
[0083] The key structure of multi-head self-attention is the MHSA self-attention layer. MHSA consists of multiple independent self-attention layers, each head having an independent query Q, key K, and value V vector, and these vectors are mapped to a learnable weight matrix.
[0084]
[0085]
[0086]
[0087] This allows the input to be transformed into different representation spaces. For each head, the input is projected into the spaces of query, key, and value, respectively, and the dot product similarity between vectors is calculated as shown in Equation (6):
[0088]
[0089] By dividing by the learnable parameter To ensure the range of the dot product remains unchanged and to reduce the impact of gradient changes, better backpropagation and optimization are performed. The obtained similarity is normalized using a softmax function to obtain the attention weight distribution for each head, which is used for weighted summation of the value vectors. Finally, the outputs of all self-attention layers are merged and combined with the weight matrix W. o Multiplying these together yields the final MHSA output. MHSA extracts the correlation between features from multiple perspectives, resulting in richer and more comprehensive feature information, as shown in formula (7):
[0090] MultiHead(Q,K,V)=Concat(Z0,Z1,···,Z h W o (7);
[0091] In summary, MHSA maps the input sequence to different subspaces by computing multiple independent attention heads in parallel, and then fuses the information from these different subspaces through attention computation. This allows for a better capture of the relationships and dependencies in the input sequence, improving the model's performance.
[0092] S104: Train the improved sand and gravel aggregate type identification model using the training dataset until the sand and gravel aggregate type identification accuracy of the training result reaches a preset range, and obtain the target sand and gravel aggregate type identification model.
[0093] In some possible implementations, when training the improved sand and gravel aggregate type identification model, the output of a preset neuron in the model is set to zero to increase the model's robustness to the input, reduce overfitting, and improve the model's accuracy in identifying the results.
[0094] For example, adding MHSA attention after the third convolutional layer and CBAM and MHSA attention after the fourth convolutional layer can enhance model performance. Further improvements include the use of the Dropout function: by randomly setting the output of some neurons to zero during training, the model's robustness to input can be increased, making it more adaptable to noise or changes and reducing overfitting. Simultaneously, increasing the learning rate of the last layer during training can accelerate the update of the last layer's weights, allowing them to better adapt to the training data and improve model performance. This speeds up model training and allows more attention to be focused on adjusting the weights of the last layer, thus better adapting to the specific needs of the target task.
[0095] In some possible implementations, after obtaining the target sand and gravel aggregate type identification model in step S104, the method further includes:
[0096] The target sand and gravel aggregate type identification model was validated using the validation dataset.
[0097] If the accuracy of the verification result reaches the preset range, then the target sand and gravel aggregate type model is output.
[0098] If the accuracy of the verification results does not reach the preset range, the improved sand and gravel aggregate type identification model will continue to be trained until the accuracy of the verification results reaches the preset range.
[0099] For example, to verify the effectiveness of the improved target aggregate type model provided in this application, the training results of the CM-ResNet18 network model studied in this application are compared with those of common image classification network models such as ResNet18, ResNet34, AlexNet, VGG19, and EfficientNet on a concrete aggregate dataset. Before training, the parameters of each model were adjusted to be consistent, and the Softmax output categories were adjusted from 1000 categories to 9 categories. Fifty iterations were performed under the same training environment. The performance of each model, including accuracy, precision, recall, F1-score, and recognition time, is shown in Table 1 below.
[0100]
[0101] Table 1
[0102] Table 1 above compares the performance of different network models. Comparing the test accuracy of each network model reveals that the final recognition accuracy of each convolutional network model on the concrete aggregate type dataset is above 88%. The CM-ResNet18 network model has a slightly higher test accuracy than the ResNet34 and ResNet18 network models. After 50 iterations of training, the CM-ResNet18 network model achieves a test set recognition accuracy of 99.09%, which is 2.59% higher than the classic ResNet18 model, 1.62% higher than the deeper ResNet34 network model, and 1.26% higher than the EfficientNet model. The VGG19 model has an accuracy of only 88%, which is 11.09% lower than the CM-ResNet18 network model. The AlexNet model has a final recognition accuracy of only 90.71%, which is 8.38% lower than the CM-ResNet18 network model. Compared with other models, the CM-ResNet18 model performs best in recognizing concrete aggregate types on the dataset.
[0103] This achievement is attributed to the advantages of incorporating CBAM and MHSA mechanisms into ResNet18: the MHSA attention mechanism allows for observation of individual aggregate images from different perspectives, while the CBAM attention mechanism focuses on the importance of different channels and spatial locations within the feature map. By introducing these attention mechanisms, the model can learn and fuse information from multiple angles, improving its understanding of concrete aggregate images. This combination enables the model to consider both local and global information simultaneously, enhancing its image perception capabilities.
[0104] Furthermore, addressing the problem of identifying concrete raw material types, this application proposes a CM-ResNet18 model for efficient identification of concrete aggregate classification images to achieve automatic identification of sand and gravel aggregate types. First, the original dataset of concrete aggregate was significantly expanded. Then, the CM-ResNet18 model was combined with a transfer learning strategy. This allows the model to achieve high performance within a relatively small number of training iterations, thus significantly shortening the training time. This application comprehensively compares the recognition results of the trained CM-ResNet18 model with other widely used classification network models. The results show that the CM-ResNet18 network model achieves a recognition rate of 99.09% for concrete aggregate images, verifying the feasibility of the CM-ResNet18 network model. See Table 2 below.
[0105]
[0106] Table 2
[0107] As shown in Table 2, the target aggregate identification model provided in this application exhibits high recall, F1 score, and specificity. Specifically, the accuracy of this model for identifying nine types of concrete aggregates is above 95%. In terms of specificity, the target aggregate identification model provided in this application achieves over 99.9%. The classification performance ranking based on F1 score is as follows: 4.3 coarse gravel, fine stone, 5.0 iron concentrate, yellow sand, bluestone, 3.95 coarse gravel, 4.4 iron concentrate, 3.95 mineral powder, 3.95 mineral powder. The identification performance for 3.95 mineral powder is slightly weaker. However, overall, the accuracy of the test dataset reaches 99.09%, the precision reaches 99.10%, the recall reaches 98.80%, and the F1 score reaches 98.93%. This demonstrates that the CM-ResNet18 model performs excellently in the concrete aggregate identification task.
[0108] S105: Input the image of the concrete aggregate to be identified into the target aggregate type identification model, and output the identification result of each type of aggregate in the image.
[0109] As can be seen, to address the problem of identifying concrete raw material types and achieve automatic identification of sand and gravel aggregate types, this application proposes a CM-ResNet18 model for efficient recognition of concrete sand and gravel aggregate classification images. First, the original dataset of concrete sand and gravel aggregates was significantly expanded. Then, the CM-ResNet18 model was combined with a transfer learning strategy. This allows the model to achieve high performance within a relatively small number of training iterations, thus significantly shortening the training time. This application comprehensively compares the recognition results of the trained CM-ResNet18 model with other widely used classification network models. The results show that the CM-ResNet18 network model achieves a recognition rate of 99.09% for concrete sand and gravel aggregate images, verifying the feasibility of the CM-ResNet18 network model.
[0110] In summary, the sand and gravel aggregate type identification method provided in this application improves the accuracy of sand and gravel aggregate type identification by adding MHSA attention mechanism and CBAM attention mechanism to the existing ResNet18. Among them, ResNet18+CBAM improved the classification accuracy by 1.16% compared to ResNet18, indicating that the spatial and channel attention mechanisms added to ResNet18 by CBAM can help the model better understand the input image and improve its feature extraction ability. ResNet18+CBAM+MHSA (added only in the fourth layer) improved the accuracy by 0.7% compared to adding the CBAM mechanism only, indicating that MHSA, through the global vision, can help the model better grasp the global features and associations of the image, which helps to improve the model's classification performance. Finally, compared with ResNet18+CBAM+MHSA (added only in the fourth layer), CM-ResNet18 improved the accuracy, precision, recall and F1-score by 0.73%, 0.58%, 0.45% and 0.59% respectively. This shows that the multi-layer MHSA in CM-ResNet18 can gradually expand the receptive field and capture more complex image information. Combined with the CBAM mechanism, it enhances the model's feature extraction and expressive abilities, thus improving the accuracy.
[0111] Based on the same application concept, this application also provides a sand and gravel aggregate type identification system corresponding to the sand and gravel aggregate type identification method provided in the above embodiments. Since the principle of the system in this application to solve the problem is similar to the method in the above embodiments of this application, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be described again.
[0112] Figure 5 This is a functional block diagram of a sand and gravel aggregate type identification system 200 provided in an embodiment of this application. Figure 5As shown, it includes a data acquisition module 210, an image processing module 220, a model improvement module 230, a model training module 240, and an image recognition module 250, wherein:
[0113] The data acquisition module 210 is used to acquire initial data images of different types of sand and gravel aggregates;
[0114] The image processing module 220 is used to preprocess the initial data image to construct the target dataset of the sand and gravel aggregate; the target dataset includes a training dataset and a validation dataset;
[0115] The model improvement module 230 is used to add MHSA attention mechanism and CBAM attention mechanism to the initial sand and gravel aggregate type identification model to obtain an improved sand and gravel aggregate type identification model.
[0116] The model training module 240 is used to train the improved sand and gravel aggregate type identification model using the training dataset until the sand and gravel aggregate type identification accuracy of the training result reaches a preset range, thereby obtaining the target sand and gravel aggregate type identification model.
[0117] The image recognition module 250 is used to input the image of the concrete aggregate to be identified into the target aggregate type recognition model, and output the recognition result of each type of aggregate in the image.
[0118] In some possible implementations, the image processing module 220 is further configured to:
[0119] The initial dataset is augmented to obtain an augmented dataset;
[0120] Redundant information in the images of the augmented dataset is removed to obtain the target dataset;
[0121] The target dataset is divided into a training dataset and a validation dataset according to a preset ratio.
[0122] In some possible implementations, the model improvement module 230 is further configured to:
[0123] Obtain an initial sand and gravel aggregate type identification model;
[0124] An MHSA attention mechanism is added after the third convolution layer of the initial sand and gravel aggregate type identification model, and a CBAM attention mechanism and an MHSA attention mechanism are added after the fourth convolution layer of the initial sand and gravel aggregate type identification model to obtain an improved sand and gravel aggregate type identification model.
[0125] In some possible implementations, a verification module is also included for:
[0126] The target sand and gravel aggregate type identification model was validated using the validation dataset.
[0127] If the accuracy of the verification result reaches the preset range, then the target sand and gravel aggregate type model is output.
[0128] If the accuracy of the verification results does not reach the preset range, the improved sand and gravel aggregate type identification model will continue to be trained until the accuracy of the verification results reaches the preset range.
[0129] In some possible implementations, the model training module 240 is further configured to set the output of a preset neuron in the model to zero when training the improved sand and gravel aggregate type identification model, thereby increasing the robustness of the model to the input, reducing overfitting, and improving the accuracy of the model in identifying the results.
[0130] Those skilled in the art will understand that the embodiments of this application can be provided as methods or systems. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.
[0131] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.
Claims
1. A method for identifying the type of sand and gravel aggregate, characterized in that, include: Collect initial data images of different types of sand and gravel aggregates; The initial data image is preprocessed to construct the target dataset of the sand and gravel aggregate; The target dataset includes a training dataset and a validation dataset; Add MHSA attention mechanism and CBAM attention mechanism to the initial sand and gravel aggregate type identification model to obtain the improved sand and gravel aggregate type identification model; The improved sand and gravel aggregate type identification model is trained using the training dataset until the accuracy of sand and gravel aggregate type identification in the training results reaches a preset range, thereby obtaining the target sand and gravel aggregate type identification model. The image of the concrete aggregate to be identified is input into the target aggregate type identification model, and the identification result of each type of aggregate in the image is output. Adding MHSA attention mechanism and CBAM attention mechanism to the initial sand and gravel aggregate type identification model to obtain an improved sand and gravel aggregate type identification model includes: obtaining the initial sand and gravel aggregate type identification model; adding MHSA attention mechanism after the third convolution layer of the initial sand and gravel aggregate type identification model, and adding CBAM attention mechanism and MHSA attention mechanism after the fourth convolution layer of the initial sand and gravel aggregate type identification model to obtain the improved sand and gravel aggregate type identification model; wherein, the improved model is based on ResNet18.
2. The method for identifying sand and gravel aggregate types according to claim 1, characterized in that, The initial data image is preprocessed to construct the target dataset of the sand and gravel aggregate, including: The initial dataset is augmented to obtain an augmented dataset; Redundant information in the images of the augmented dataset is removed to obtain the target dataset; The target dataset is divided into a training dataset and a validation dataset according to a preset ratio.
3. The method for identifying sand and gravel aggregate types according to claim 1, characterized in that, After obtaining the target sand and gravel aggregate type identification model, it also includes: The target sand and gravel aggregate type identification model was validated using the validation dataset. If the accuracy of the verification result reaches the preset range, then the target sand and gravel aggregate type model is output. If the accuracy of the verification results does not reach the preset range, the improved sand and gravel aggregate type identification model will continue to be trained until the accuracy of the verification results reaches the preset range.
4. The method for identifying sand and gravel aggregate types according to claim 1, characterized in that, When training the improved sand and gravel aggregate type identification model, the output of the preset neurons in the model is set to zero to increase the model's robustness to the input, reduce overfitting, and improve the model's accuracy in identifying the results.
5. A sand and gravel aggregate type identification system, characterized in that, It includes a data acquisition module, an image processing module, a model improvement module, a model training module, and an image recognition module, among which: The data acquisition module is used to acquire initial data images of different types of sand and gravel aggregates; The image processing module is used to preprocess the initial data image to construct the target dataset of the sand and gravel aggregate; the target dataset includes a training dataset and a validation dataset; The model improvement module is used to add MHSA attention mechanism and CBAM attention mechanism to the initial sand and gravel aggregate type identification model to obtain the improved sand and gravel aggregate type identification model. The model training module is used to train the improved sand and gravel aggregate type identification model using the training dataset until the sand and gravel aggregate type identification accuracy of the training result reaches a preset range, thereby obtaining the target sand and gravel aggregate type identification model. The image recognition module is used to input the image of the concrete aggregate to be identified into the target aggregate type recognition model and output the recognition result of each type of aggregate in the image. The model improvement module is also used to: obtain an initial sand and gravel aggregate type identification model; add an MHSA attention mechanism after the third convolution of the initial sand and gravel aggregate type identification model, and add a CBAM attention mechanism and an MHSA attention mechanism after the fourth convolution of the initial sand and gravel aggregate type identification model to obtain an improved sand and gravel aggregate type identification model; wherein, the improved model is based on ResNet18.
6. The sand and gravel aggregate type identification system according to claim 5, characterized in that, The image processing module is also used for: The initial dataset is augmented to obtain an augmented dataset; Redundant information in the images of the augmented dataset is removed to obtain the target dataset; The target dataset is divided into a training dataset and a validation dataset according to a preset ratio.
7. The sand and gravel aggregate type identification system according to claim 5, characterized in that, It also includes a verification module for: The target sand and gravel aggregate type identification model was validated using the validation dataset. If the accuracy of the verification result reaches the preset range, then the target sand and gravel aggregate type model is output. If the accuracy of the verification results does not reach the preset range, the improved sand and gravel aggregate type identification model will continue to be trained until the accuracy of the verification results reaches the preset range.
8. The sand and gravel aggregate type identification system according to claim 5, characterized in that, The model training module is also used to set the output of the preset neurons in the model to zero when training the improved sand and gravel aggregate type identification model, thereby increasing the robustness of the model to the input, reducing overfitting, and improving the accuracy of the model in identifying the results.