A concrete phase segmentation method based on lightweight deep learning and SE attention mechanism

Abstract

The application discloses a concrete phase segmentation method based on lightweight deep learning and SE attention mechanism. The method adopts X-ray tomography to obtain a concrete original image; in the preprocessing, a semi-automatic labeling method is used to accurately label the concrete phase; the data set is divided into a training set, a verification set and a test set; on the basis of the U-Net network architecture, the strategy of reducing the number of convolution layer channels is used to realize network lightweight, and the SE attention mechanism is embedded to construct an SE U-Net model; through model training and optimization, finally, model evaluation is carried out on the test set. The SE U-Net model constructed by the application can realize efficient and accurate concrete phase segmentation in the concrete phase segmentation task, and the mIoU reaches 88.5%, which is 9.7% higher than that of the U-Net model. Meanwhile, through the lightweight design, the model parameter quantity and the calculation resource consumption are reduced.

Description

Technical Field

[0001] This invention belongs to the field of concrete image semantic segmentation technology, and particularly relates to a concrete phase segmentation method based on lightweight deep learning and SE (Squeeze-and-Excitation) attention mechanism. Background Technology

[0002] Concrete, a key material widely used in infrastructure construction, depends heavily on the composition, properties, and distribution of its different phases in terms of quality and performance. Accurate identification and segmentation of these different phases in concrete images provide crucial information for quality assessment, structural analysis, and durability prediction. However, due to the high heterogeneity within concrete, the phases often overlap in image grayscale values, making automatic segmentation extremely challenging.

[0003] Traditional concrete phase segmentation methods mainly rely on image processing techniques such as manual identification, threshold-based algorithms, and edge detection, which typically require setting parameters based on experience and specific rules. These methods are not only time-consuming and labor-intensive, but also often perform poorly when dealing with complex and heterogeneous features, and cannot effectively handle noise and artifacts in the image, resulting in low segmentation accuracy.

[0004] In recent years, deep learning technology, especially the U-Net network architecture, has been gradually applied to fields such as building material image analysis due to its excellent performance in tasks such as medical image segmentation. U-Net, with its symmetrical encoder-decoder structure and skip connections, can learn high-dimensional features through deep networks, achieving good segmentation performance in complex scenes or with small sample data.

[0005] However, U-Net models typically contain a large number of parameters, making them difficult to deploy efficiently in environments with limited computing power. Furthermore, for highly heterogeneous concrete images, relying solely on the U-Net architecture still faces problems such as insufficient detail recognition and unreasonable attention allocation. Therefore, there is an urgent need for a lightweight deep learning method that can improve segmentation accuracy while possessing an efficient attention mechanism. Summary of the Invention

[0006] Purpose of the Invention: To address the above problems, the purpose of this invention is to provide a concrete phase segmentation method based on lightweight deep learning and SE attention mechanism. Based on the U-Net network architecture, a lightweight model design is achieved by reducing the number of channels in each convolutional layer. Furthermore, SE attention modules are embedded in the convolutional layers of the encoder and decoder to effectively enhance the model's weight allocation for key features, thereby achieving high-precision segmentation of the concrete phase.

[0007] Technical solution: The present invention provides a concrete phase segmentation method based on lightweight deep learning and SE attention mechanism, comprising the following steps:

[0008] (1) Data acquisition: High-resolution images of the internal structure of concrete, i.e., the original images, are obtained through X-ray computed tomography (X-CT).

[0009] (2) Data preprocessing: The original images are preprocessed and the concrete phases are labeled using a semi-automatic annotation method to form an initial dataset;

[0010] (3) Data splitting: The initial dataset is divided into a training set, a validation set, and a test set;

[0011] (4) Lightweight model architecture design: Based on the U-Net network architecture, the number of model parameters is reduced by decreasing the number of channels in each convolutional layer, and a lightweight U-Net model is constructed.

[0012] (5) SE attention mechanism integration: The SE (Squeeze-and-Excitation) attention mechanism is embedded in the convolutional layers of the encoder and decoder of the lightweight U-Net model to construct the SE U-Net model. The feature extraction and classification ability of different concrete phases is enhanced by using channel adaptive weighting.

[0013] (6) Model training: The SE U-Net model is trained using the training set obtained in step (3), validated using the validation set obtained in step (3), hyperparameters are tuned based on the validation results, and the optimal model obtained after training is saved.

[0014] (7) Model testing: Input the test set obtained in step (3) into the optimal model for high-precision phase segmentation, output the prediction mask, and give the evaluation index.

[0015] The concrete phase consists of one or more of aggregates, cement paste, or pores.

[0016] In step (2), the preprocessing includes rotation, cropping, and scaling to unify the image size; the semi-automatic annotation method is based on threshold to perform preliminary annotation of concrete phases, supplemented by manual correction, and covers the concrete phases with masks respectively.

[0017] In step (3), the training set, validation set, and test set are all independent of each other; the training set is used for model training, the validation set is used for model tuning, and the test set is used for model evaluation.

[0018] In step (3), the initial dataset is randomly divided into a training set, a validation set, and a test set according to a preset ratio. During the division process, a stratified sampling method is used to ensure that the data categories of the three subsets are evenly distributed. At the same time, a fixed random seed is set to ensure the repeatability of each random division result.

[0019] In step (4), the way to reduce the number of channels in each convolutional layer is to use a smaller number of channels than the preset number of channels in the convolutional modules of the encoder and decoder for feature extraction and fusion, so as to reduce the overall number of model parameters and computational resource consumption.

[0020] In step (5), embedding the SE attention mechanism includes the following steps:

[0021] 1) Squeeze: Spatial compression is performed on each channel of the input feature map through global average pooling, thereby forming global features for each channel. This step can be represented by formula (1):

[0022]

[0023] Where H represents the height of the input feature map, W represents the width of the input feature map, and X represents the height of the input feature map. c,h,w z represents the pixel value of the c-th channel at position (h, w) in the input feature map. c This represents the global feature of the c-th channel after average pooling;

[0024] 2) Excitation: A two-layer fully connected network is used to transform the global features of the channels to learn and generate adjustment weights for each channel. The global features of the channels are dimensionality reduced by the first fully connected network and the ReLU activation function is applied to add non-linear processing. The second fully connected network expands the feature dimension back to the original number of channels and outputs the final channel weights through the Sigmoid activation function. This step can be expressed by formulas (2) to (4):

[0025] ReLU(x) = max(0,x) (2)

[0026]

[0027] s c =σ(W2·ReLU(W1·z) c (4)

[0028] Where x represents the input value, ReLU is the activation function, σ represents the Sigmoid activation function, W1 represents the dimensionality reduction weight matrix, W2 represents the dimensionality increase weight matrix, and s c This represents the weight of the c-th channel;

[0029] 3) Scale (Reweighting): The weights s of each channel are adjusted by element-wise multiplication to adjust the channel intensity of the original input feature map. This step can be expressed by formula (5):

[0030] Y c,h.w =s c×X c,h,w (5)

[0031] Among them, Y c,h.w It is the output feature map after attention weighting, X c,h,w It is the pixel value of the c-th channel in the input feature map at position (h, w).

[0032] In step (6), the model training uses classification cross-entropy as the loss function, which evaluates the difference between the model output probability distribution and the real mask distribution at the pixel level; the Adam optimizer is used to update the parameters, and the early stopping strategy and the best model saving strategy are set based on the loss value of the validation set.

[0033] In step (6), data augmentation techniques are used on the training set data, including horizontal flipping, vertical flipping and random rotation, to improve the model's generalization ability in complex scenarios.

[0034] In step (7), the evaluation metrics include Accuracy, Precision, Recall, F1-score and mIoU, all expressed as percentages. Higher values ​​usually indicate better model performance. The accuracy of the segmentation model is quantitatively evaluated by comprehensively analyzing the evaluation metrics. The segmentation results are then visualized and compared based on the original image, the real mask and the predicted mask.

[0035] This invention discloses a concrete phase segmentation method based on lightweight deep learning and SE attention mechanism, comprising the following steps: acquiring original concrete images using X-ray tomography; preprocessing the original images and accurately labeling the concrete phases using a semi-automatic annotation method; dividing the obtained dataset into training, validation, and test sets; based on the U-Net network architecture, reducing the number of convolutional layer channels to achieve network lightweighting, while embedding an SE attention mechanism in the convolutional layers to adaptively enhance the response of important feature channels, thus constructing an SE U-Net model; training the model using the training set, optimizing the model using the validation set, and finally evaluating the model on the test set. Results show that the SE U-Net model constructed in this invention can accurately classify concrete phases, achieving an mIoU of 88.5%, a significant improvement of 9.7% compared to the U-Net model. Furthermore, the lightweight design reduces the number of model parameters and computational resource consumption, enabling efficient concrete phase segmentation.

[0036] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0037] (1) This invention is based on the U-Net network architecture and is designed to be lightweight. While ensuring the segmentation accuracy, it significantly reduces the number of model parameters and the consumption of computing resources, and has more advantages in terms of hardware resource consumption and running speed.

[0038] (2) The SE U-Net model constructed in this invention, by embedding SE attention modules in the convolutional layers of the encoder and decoder, endows the model with the ability to adaptively enhance key feature channels, significantly improving the segmentation accuracy of the model in complex environments with overlapping image gray levels or large noise interference, and achieving accurate segmentation of concrete phases. (See attached figures)

[0039] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0040] Figure 2 This is a segmentation result image of the SE U-Net model. Detailed Implementation

[0041] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0042] Example 1

[0043] like Figure 1 As shown, the concrete phase segmentation method based on lightweight deep learning and SE attention mechanism includes the following steps:

[0044] (1) Data Acquisition:

[0045] High-resolution images of the internal structure of concrete were obtained using X-ray computed tomography (X-CT). Specifically, the scanning direction was from top to bottom with a scanning interval of 0.05 mm. Images with poor imaging quality at the top and bottom were removed, resulting in a total of 900 scanned images. The images were numbered sequentially and stored in a designated directory. All of these images are original images.

[0046] (2) Data preprocessing:

[0047] From 900 original images, 150 original images are selected using a uniform interval sampling method, that is, one image is selected every 5 images starting from the first original image, to avoid the high similarity between adjacent images affecting the generalization ability of the model; after converting these 150 original images into grayscale images, the image size is unified to 581×581 pixels through operations such as rotation, cropping, and scaling.

[0048] A semi-automated annotation method was used to annotate the concrete phases. Specifically, firstly, the pores and aggregates in the concrete image were initially annotated based on thresholds, with the pore threshold set to 40 and the aggregate threshold set to 130. Then, the initial annotation results were manually corrected in the superpixel mode of the MATLAB image segmenter to optimize abnormal or edge areas and ensure the accuracy of the annotation. Finally, the remaining areas of the image were annotated as cement paste, and corresponding masks were applied to the aggregates, pores, and cement paste, with masks of 0, 1, and 2 respectively, forming mask images.

[0049] In PyCharm, the pixel values ​​of the selected 150 original images are normalized by dividing the pixel values ​​by 255 to scale the pixel range to the [0,1] interval. The original image size is uniformly scaled to 256×256 pixels using bilinear interpolation. The mask image is scaled to 256×256 pixels using nearest neighbor interpolation to avoid class aliasing or blurred boundaries. The mask image and the original image are paired and stored to form the initial dataset.

[0050] (3) Data partitioning:

[0051] The initial dataset was divided into training, validation, and test sets in a 6:2:2 ratio. Stratified sampling was used during the partitioning process to ensure a balanced distribution of data categories across the three subsets. A fixed random seed was also used to ensure the repeatability of each random partition. The training, validation, and test sets were independent of each other; the training set was used for model training, the validation set for model tuning, and the test set for model evaluation.

[0052] (4) Lightweight design of model architecture:

[0053] A U-Net model is constructed, and based on its network architecture, the number of model parameters is reduced by decreasing the number of channels in each convolutional layer, resulting in a lightweight U-Net model. This lightweight design reduces the computational resource consumption of GPUs and CPUs. The reduction in the number of channels in convolutional layers is achieved in each level of the convolutional layers in both the encoder and decoder. The specific architecture of the lightweight U-Net model is as follows:

[0054] The encoder consists of 4 layers of "convolution + max pooling (2×2)" with a kernel size of 3×3 for each layer and activation using the rectified linear function (ReLU). The number of channels increases progressively in each layer, but compared with U-Net, the initial number of channels is reduced from 64 to 32, and then reduced to 64, 128 and 256 in subsequent layers respectively.

[0055] The lowest layer of the network: retains 512 channels to facilitate the capture of complex information about the concrete phase;

[0056] Decoder: Composed of 4 layers of "top-down + convolution", each convolution kernel size is 3×3, the number of channels of each convolution layer is 256, 128, 64 and 32 respectively, and at the end of the network, it is mapped to the specified number of classes 3 by a 1×1 convolution;

[0057] (5) Integration of SE attention mechanisms:

[0058] An SE attention mechanism is embedded in several convolutional layers of the encoder and decoder of a lightweight U-Net model to construct an SE U-Net model. Channel adaptive weighting is used to enhance the model's ability to extract and classify features of different concrete phases. The SE attention mechanism is implemented through the following steps:

[0059] 1) Squeeze: Spatial compression is performed on each channel of the input feature map through global average pooling, thereby forming the global features of each channel. This step can be represented by formula (1):

[0060]

[0061] Where H represents the height of the input feature map, W represents the width of the input feature map, and X represents the height of the input feature map. c,h,w z represents the pixel value of the c-th channel at position (h, w) in the input feature map. c This represents the global feature of the c-th channel after average pooling.

[0062] 2) Excitation: A two-layer fully connected network is used to transform the global features of the channels in order to learn and generate the adjustment weights for each channel. The global features of the channels are reduced in dimensionality by the first layer of the fully connected network and the ReLU activation function is applied to add non-linear processing. The second layer of the fully connected network expands the feature dimension back to the original number of channels and outputs the final channel weights through the Sigmoid activation function. This step can be expressed by formulas (2) to (4):

[0063] ReLU(x) = max(0,x) (2)

[0064]

[0065] s c =σ(W2·ReLU(W1·z) c (4)

[0066] Where x represents the input value, ReLU is the activation function, σ represents the Sigmoid activation function, W1 represents the dimensionality reduction weight matrix, W2 represents the dimensionality increase weight matrix, and s c This represents the weight of the c-th channel.

[0067] 3) Scale: The weights s of each channel are adjusted by element-wise multiplication to adjust the channel intensity of the original input feature map. This step can be expressed by formula (5):

[0068] Y c,h.w =s c ×X c,h,w (5)

[0069] Among them, Y c,h.w It is the output feature map after attention weighting, X c,h,w It is the pixel value of the c-th channel in the input feature map at position (h, w).

[0070] (6) Model training:

[0071] The U-Net model and the SE U-Net model were trained using the training set. During the training process, data augmentation techniques such as horizontal flipping, vertical flipping, and random rotation were used to improve the generalization ability of the model in complex scenes. The classification cross-entropy was used as the loss function, which evaluates the difference between the model output probability distribution and the real mask distribution at the pixel level. Its calculation is shown in Equation (6).

[0072]

[0073] Where L is the loss value, N is the number of samples, M is the total number of classes, and y ic It is an indicator value for whether the i-th sample belongs to category c. It is the probability that sample i belongs to category c as predicted by the model.

[0074] The training parameters for the U-Net and SE U-Net models were set, with the optimizer Adam, BatchSize set to 8, and Epochs set to 100. An early stopping strategy was applied: when the validation set loss value no longer decreased after 20 consecutive training rounds, training was automatically stopped and the current optimal weights were saved to obtain the optimal model. The training process of both the U-Net and SE U-Net models was accelerated using GPUs. The U-Net model could typically complete training within 20 minutes, while the SE U-Net model could typically complete training within 10 minutes.

[0075] (7) Model testing:

[0076] The test set is input into the optimal U-Net and SE U-Net models respectively for high-precision phase segmentation, and the predicted mask is output, along with evaluation metrics. The evaluation metrics include Accuracy, Precision, Recall, F1-score, and mIoU. The calculation of the evaluation metrics is shown in formulas (7) to (11):

[0077]

[0078]

[0079]

[0080]

[0081]

[0082] Where TP stands for True Positive Example, indicating that a positive example is correctly classified as a positive example; FP stands for False Positive Example, indicating that a negative example is incorrectly classified as a positive example; FN stands for False Negative Example, indicating that a positive example is incorrectly classified as a negative example; and TN stands for True Negative Example, indicating that a negative example is correctly classified as a negative example. i This represents the number of pixels correctly predicted as class i; FP i This represents the number of pixels that are actually not of class i but were incorrectly predicted as class i; FN i This represents the number of pixels that are actually of category i but were incorrectly predicted as other categories; n is the total number of segmentation categories.

[0083] All evaluation metrics are expressed as percentages, with higher values ​​generally indicating better model performance. A comprehensive analysis of these metrics is used to quantitatively evaluate the accuracy of the U-Net and SE U-Net models. The segmentation results are then visually compared using the original image, the ground truth mask, and the predicted mask. Table 1 shows the evaluation results of the U-Net model and the SE U-Net model constructed in this invention. A visual comparison of the SE U-Net model's segmentation results is also provided. Figure 2 As shown.

[0084] Table 1 Evaluation results of U-Net and SE U-Net

[0085]

[0086] As the results above show, the SE U-Net model constructed in this invention significantly outperforms the U-Net model in all evaluation metrics for concrete phase segmentation. Specifically, Accuracy, Precision, Recall, and F1-score are improved by 7.3%, 3%, 7.5%, and 5.5% respectively compared to U-Net. Particularly noteworthy is its mIoU, a key metric, which achieves 88.5%, a significant improvement of 9.7% compared to U-Net. These results demonstrate that the SE U-Net model exhibits higher segmentation accuracy in concrete phase segmentation compared to the U-Net model, enabling more precise identification and differentiation of different phase boundaries, and providing more accurate and reliable segmentation results.

Claims

1. A concrete phase segmentation method based on lightweight deep learning and SE attention mechanism, characterized in that, Includes the following steps: (1) Data acquisition: High-resolution images of the internal structure of concrete, i.e., the original images, are obtained through X-ray tomography. (2) Data preprocessing: The original images are preprocessed and the concrete phases are labeled using a semi-automatic annotation method to form an initial dataset; (3) Data splitting: The initial dataset is divided into a training set, a validation set, and a test set; (4) Lightweight design of model architecture: Based on the U-Net network architecture, the number of model parameters is reduced by reducing the number of channels in each convolutional layer, and a lightweight U-Net model is constructed. The way to reduce the number of channels in each convolutional layer is to use a smaller number of channels than the preset number of channels in the convolutional modules of the encoder and decoder for feature extraction and fusion, thereby reducing the overall number of model parameters and computational resource consumption. (5) SE attention mechanism integration: The SE attention mechanism is embedded in the convolutional layers of the encoder and decoder of the lightweight U-Net model to construct the SE U-Net model. The feature extraction and classification ability of different concrete phases is enhanced by using channel adaptive weighting. (6) Model training: The SE U-Net model is trained using the training set obtained in step (3), validated using the validation set obtained in step (3), and the hyperparameters are tuned based on the validation results. The optimal model obtained after training is saved. (7) Model testing: Input the test set obtained in step (3) into the optimal model for high-precision phase segmentation, output the prediction mask, and give the evaluation index.

2. The method according to claim 1, characterized in that, The concrete phase consists of one or more of aggregates, cement paste, or pores.

3. The method according to claim 1, characterized in that, In step (2), the preprocessing includes rotation, cropping, and scaling to unify the image size; the semi-automatic annotation method is based on threshold to perform preliminary annotation of concrete phases, supplemented by manual correction, and masks are applied to the concrete phases respectively.

4. The method according to claim 1, characterized in that, In step (3), the training set, validation set, and test set are all independent of each other; the training set is used for model training, the validation set is used for model tuning, and the test set is used for model evaluation.

5. The method according to claim 1, characterized in that, In step (3), the initial dataset is randomly divided into training set, validation set and test set according to a preset ratio. During the division process, stratified sampling is used to ensure that the data categories of the three subsets are evenly distributed. At the same time, a fixed random seed is set to ensure the repeatability of each random division result.

6. The method according to claim 1, characterized in that, In step (5), embedding the SE attention mechanism includes the following steps: 1) Squeeze: Spatial compression is performed on each channel of the input feature map through global average pooling, thereby forming the global features of each channel. This step can be expressed by formula (1): （1） in, Indicates the height of the input feature map. This indicates the width of the input feature map. This indicates that the c-th channel in the input feature map is at position pixel values, This represents the global feature of the c-th channel after average pooling; 2) Excitation: A two-layer fully connected network is used to transform the global features of the channels in order to learn and generate the adjustment weights for each channel. The global features of the channels are reduced in dimensionality by the first layer of the fully connected network and the ReLU activation function is applied to add non-linear processing. The second layer of the fully connected network expands the feature dimension back to the original number of channels and outputs the final channel weights through the Sigmoid activation function. This step can be represented by formulas (2) to (4): （2） （3） （4） in, Indicates the input value. For activation function, This represents the Sigmoid activation function. Represents the dimensionality reduction weight matrix. This represents the weight matrix for increasing dimensionality. This represents the weight of the c-th channel; 3) Scale: The weights of each channel are adjusted by element-wise multiplication to adjust the channel intensity of the original input feature map. This step can be expressed by formula (5): （5） in, It is the output feature map after attention weighting. It is the position of channel c in the input feature map. The pixel value.

7. The method according to claim 1, characterized in that, In step (6), the model training uses classification cross-entropy as the loss function, which evaluates the difference between the model output probability distribution and the real mask distribution at the pixel level; the Adam optimizer is used to update the parameters, and the early stopping strategy and the best model saving strategy are set based on the loss value of the validation set.

8. The method according to claim 1, characterized in that, In step (6), data augmentation techniques are applied to the training set data, including horizontal flipping, vertical flipping and random rotation, to improve the model's generalization ability in complex scenarios.

9. The method according to claim 1, characterized in that, In step (7), the evaluation metrics include Accuracy, Precision, Recall, F1-score and mIoU, which quantitatively evaluate the accuracy of the segmentation model and visualize the segmentation results for comparison.

Images