A deep learning segmentation method for asphalt mixture CT images

Abstract

A deep learning segmentation method for asphalt mixture CT images relates to the technical field of aggregate image segmentation. CT images of asphalt mixtures are collected and voids are removed. Image equalization is performed using a gamma beam hardening correction algorithm. A filter window is selected, and an adaptive bilateral filtering denoising algorithm based on local image information is used to denoise the image. The U-Net model is improved, with the Inception convolution module replacing the standard convolution operation and residual connections replacing skip-layer connections. A spatial attention mechanism is introduced, and a joint loss function is used. Image samples are selected, aggregates are labeled, and data augmentation is used to increase the sample size to form a training set for model training. After training, this set is used to segment other images. By equalizing and denoising CT images and using the improved U-Net model, the method can achieve accurate segmentation of aggregate-mortar boundary information, avoiding aggregate adhesion issues.

Description

Technical Field

[0001] The present invention relates to the technical field of aggregate image segmentation, and in particular to a deep learning segmentation method for asphalt mixture CT images. Background Art

[0002] Asphalt mixtures are composed of multiple materials and are complex, heterogeneous, multi-phase, and multi-layered particle systems. To understand the internal microstructure of asphalt mixtures, cross-sectional images of asphalt mixtures must be acquired using industrial CT technology. Therefore, accurately segmenting the voids, asphalt mortar, and aggregates in asphalt mixture CT images is crucial for understanding the evolution of the microstructure.

[0003] Currently, image segmentation methods are primarily categorized as traditional image segmentation methods and deep learning-based image segmentation methods. Traditional threshold-based image segmentation methods, such as watershed segmentation, have significant disparities in processing capabilities for different types of asphalt mixture CT images, resulting in poor results. They also severely impact aggregate adhesion during segmentation, and pixel corrosion cannot be used to remove this adhesion. Compared to traditional threshold segmentation algorithms, deep learning algorithms can more efficiently utilize small sample sizes, achieve precise boundary segmentation through skip connections, and simultaneously capture multi-scale feature information, demonstrating excellent generalization capabilities.

[0004] However, in the asphalt mixture CT image segmentation process, due to the close boundary thresholds between the asphalt mortar and the aggregate, deep learning still suffers from aggregate adhesion during the segmentation process. This makes it difficult to accurately analyze aggregate migration patterns in microstructure research. Therefore, it is necessary to improve deep learning methods for asphalt mixture CT images to address the aggregate adhesion problem. This is of great significance for improving aggregate segmentation accuracy and studying the microstructure of asphalt mixtures. Summary of the Invention

[0005] To address the shortcomings of the background technology, the present invention provides a deep learning segmentation method for asphalt mixture CT images. By equalizing and denoising the CT images and adopting an improved U-Net model, it can achieve accurate segmentation of aggregate-mortar boundary information and avoid aggregate adhesion problems.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a deep learning segmentation method for asphalt mixture CT images, comprising the following steps:

[0007] Step 1: Acquire CT images of asphalt mixture

[0008] Industrial CT was used to scan the asphalt mixture specimens to obtain cross-sectional images, which were then binarized and the Otsu method was used to remove the voids in the cross-sectional images.

[0009] Step 2: Image equalization

[0010] The gamma beam hardening correction algorithm is used to improve the grayscale value of the central part of the specimen and complete the equalization processing of the cross-sectional image;

[0011] Step 3: Select the noise reduction algorithm window

[0012] Combining the three indicators of average gradient amplitude, local variance and image visual fidelity (VIF), a 5×5 filter window was selected in the subsequent cross-sectional image processing.

[0013] Step 4: Image noise reduction

[0014] An adaptive bilateral filtering denoising algorithm based on local image information is used to adaptively adjust the spatial standard deviation and range standard deviation according to the local variance of the cross-sectional image. The calculation method is as follows:

[0015]

[0016]

[0017] w((i,j),(x,y))=f s (||(i,j)-(x,y)||,σ s (i,j))·f r (|I(i,j)-I(x,y)|,σ r (i,j))

[0018] W(i,j)=∑ (x,y)∈Ω w((i,j),(x,y))

[0019]

[0020] Where, represents the local variance within the window Ω, μ local (i, j) represents the local mean within the window Ω, Ω represents the window centered at (i, j), |Ω| represents the total number of pixels in the window, σ r (i, j) represents the adaptive spatial standard deviation, σ s (i, j) represents the standard deviation of the adaptive range, σ s·base Indicates the basic spatial standard deviation, with a value of 2, f s represents the spatial domain weight, ||(i,j)-(x,y)|| represents the Euclidean distance between pixels (x,y) and (i,j), and f r represents the range weight, |I(i,j)-I(x,y)| represents the grayscale value difference between pixels (x,y) and (i,j), w((i,j),(x,y)) represents the filter weight, W(i,j) represents the normalization factor, Iout (i, j) represents the output image pixel value;

[0021] Step 5: Convolutional module improvement

[0022] The deep learning structure based on the U-Net model is improved. The Inception convolution module is used to replace the two 3×3 convolution operations and the ReLu activation function in each stage of the U-Net model's downsampling and upsampling process.

[0023] Step 6: Connect module improvements

[0024] Residual connections are introduced in the improved U-Net model to replace skip layer connections. During the downsampling process of the improved U-Net model, a spatial attention mechanism is introduced in the Inception convolution module of each stage to assign different weights to different areas in the cross-sectional image, giving higher weights to key areas in the cross-sectional image. The key area refers to the aggregate-mortar interface area.

[0025] Step 7: Loss function improvement

[0026] The joint loss function is used in the image segmentation task of the improved U-Net model, which is expressed as follows:

[0027]

[0028] L joint =L BCE +λ·L boundary

[0029] Where N represents the number of pixels in the image, y true represents the true label, y pred Represents the model prediction results, represents the sigmoid activation function, λ = 0.3;

[0030] Step 8: Training set image annotation and sample expansion

[0031] For asphalt mixture specimens, multiple cross-sectional images of different layers are selected as samples, the aggregates in the samples are labeled, and the sample size is increased through data enhancement to form a training set;

[0032] Step 9: Model training and image segmentation

[0033] The expanded training set is used to train the improved U-Net model, and the training parameters of the minimum value of the joint loss function are saved. The trained improved U-Net model is used to segment other cross-sectional images.

[0034] Furthermore, in step 2, the calculation method of the gamma beam hardening correction algorithm is as follows:

[0035]

[0036] Where V out Represents the pixel value of each point in the output image, A is a constant with a value of 1, V in Represents the pixel value of each point in the input image, γ represents the gamma parameter, and its value is 0.75.

[0037] Furthermore, in step 5, the Inception convolution module combines multiple convolution operations, including 3×3, 5×5 and 7×7 convolution kernels.

[0038] Furthermore, in step eight, data enhancement refers to rotating the sample in a range of 30° to 270° on a two-dimensional plane.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention collects cross-sectional images of asphalt mixture specimens through industrial CT, and performs equalization and noise reduction on the CT images, and adopts an improved U-Net model. It can effectively reduce noise based on local information in the cross-sectional images, achieve accurate segmentation of aggregate-mortar boundary information, avoid aggregate adhesion problems, and has the advantages of accuracy, strong adaptability, and high efficiency. It is easy to promote and popularize, and is helpful for the study of the microstructure of asphalt mixtures. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a cross-sectional image before and after removal of the gap portion in a specific embodiment;

[0041] Figure 2 is a thermal map of the cross-sectional image before and after equalization processing in a specific embodiment;

[0042] Figure 3 is a graph showing the calculation results of the three indicators under different filter windows in a specific implementation manner;

[0043] Figure 4 is a diagram showing the comparison position and noise reduction effect of a cross-sectional image in a specific embodiment;

[0044] Figure 5 This is a deep learning structure diagram of the traditional U-Net model in a specific implementation;

[0045] Figure 6 is a deep learning structure diagram of the improved U-Net model in a specific implementation;

[0046] Figure 7 is a rotated and annotated image of a training set sample in a specific embodiment;

[0047] Figure 8is a comparison diagram of the cross-sectional image segmentation effect in a specific embodiment;

[0048] Figure 9 It is a diagram of comparison results of evaluation indicators in a specific implementation method. DETAILED DESCRIPTION

[0049] The technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0050] like Figures 1 to 9 As shown, a deep learning segmentation method for asphalt mixture CT images includes the following steps:

[0051] Step 1: Acquire CT images of asphalt mixture

[0052] The asphalt mixture specimen was scanned by industrial CT to obtain the cross-sectional image, and then the cross-sectional image was binarized and the gap part in the cross-sectional image was removed by Otsu method. Figure 1 As shown, cross-sectional images obtained before and after removing the void portion are shown.

[0053] Step 2: Image equalization

[0054] On the basis of removing the cross-sectional image of the void part, the gray value of the central part of the specimen is improved by the gamma beam hardening correction algorithm to complete the equalization processing of the cross-sectional image. Figure 2 As shown in the figure, the heat map of the cross-section image before and after equalization is shown. The calculation method is as follows:

[0055]

[0056] Where V out Represents the pixel value of each point in the output image, A is a constant with a value of 1, V in Represents the pixel value of each point in the input image, γ represents the gamma parameter, and its value is 0.75.

[0057] Step 3: Select the noise reduction algorithm window

[0058] In the process of image denoising, the filter window is an important parameter. The size of different filter windows will directly affect the blur degree of the image. A larger filter window usually leads to a more obvious blur effect, while a smaller filter window can retain more image details. In order to compare the effects of different filter windows on the blur degree of cross-sectional images, the appropriate filter window is selected by combining the three indicators of average gradient amplitude, local variance and image visual fidelity (VIF). Figure 3 As shown in the figure, the calculation results of the three indicators under different filter windows are displayed. It can be seen that when the filter window size is 5×5, the average gradient amplitude indicator drops rapidly, indicating that the cross-sectional image begins to become blurred, while the local variance and image visual fidelity indicators drop significantly, indicating that the cross-sectional image details begin to decrease significantly at this time. Therefore, based on the comprehensive evaluation of the above three indicators, a 5×5 filter window is selected in the subsequent cross-sectional image processing.

[0059] Step 4: Image noise reduction

[0060] In order to meet the noise reduction requirements of different areas of the cross-sectional image, an adaptive bilateral filtering noise reduction algorithm based on local image information is proposed. The spatial standard deviation and range standard deviation are adaptively adjusted according to the local variance of the cross-sectional image. In this way, different smoothing intensities can be applied in high-variability areas (aggregate-mortar interface area) and low-variability areas (aggregate area) to improve the noise reduction effect. The calculation method is as follows:

[0061]

[0062]

[0063] w((i,j),(x,y))=f s (||(i,j)-(x,y)||,σ s (i,j))·f r (|I(i,j)-I(x,y)|,σ r (i,j))

[0064] W(i,j)=∑ (x,y)∈Ω w((i,j),(x,y))

[0065]

[0066] Where (i, j) represents the pixel position of the image, represents the local variance within the window Ω, μ local (i, j) represents the local mean within the window Ω, Ω represents the window centered at (i, j), |Ω| represents the total number of pixels in the window, σ r (i, j) represents the adaptive spatial standard deviation, σ s(i, j) represents the standard deviation of the adaptive range, σ s·base Indicates the basic spatial standard deviation, with a value of 2, f s represents the spatial domain weight, ||(i,j)-(x,y)|| represents the Euclidean distance between pixels (x,y) and (i,j), and f r represents the range weight, |I(i,j)-I(x,y)| represents the grayscale value difference between pixels (x,y) and (i,j), w((i,j),(x,y)) represents the filter weight, W(i,j) represents the normalization factor, I out (i,j) represents the output image pixel value.

[0067] In order to verify the denoising effect of the algorithm, Figure 4 As shown in the figure, a representative contrast location is selected for the cross-sectional image, and the noise reduction effects of the original image at this contrast location, conventional wavelet filtering, and the proposed adaptive bilateral filtering are displayed. In the figure, the area with smooth grayscale pixel values ​​represents the aggregate interface, while the area with sudden grayscale pixel values ​​represents the aggregate-mortar interface. It can be seen that at the aggregate-mortar interface, both the wavelet filtering algorithm and the adaptive bilateral filtering algorithm can effectively preserve the grayscale gradient difference. Moreover, compared with the wavelet filtering algorithm, the grayscale change of the aggregate interface after the adaptive bilateral filtering algorithm is smoother, indicating that the adaptive bilateral filtering algorithm can effectively eliminate aggregate interface noise.

[0068] Step 5: Convolutional module improvement

[0069] The deep learning structure of the traditional U-Net model is mainly divided into downsampling process and upsampling process, combined with Figure 5 As shown in the figure, each stage of the downsampling process includes two 3×3 convolution operations and access to the ReLu activation function, followed by a 2×2 maximum pooling operation. In the upsampling process, each stage first performs a 2×2 deconvolution operation, and then concatenates the upsampled feature map with the corresponding feature map of the encoder through a skip layer connection. The concatenated feature map undergoes two 3×3 convolution operations and accesses the ReLu activation function.

[0070] In order to further improve the feature extraction capability of the U-Net model, the Inception convolution module is used to replace the standard convolution operation in the traditional U-Net model to improve the deep learning structure of the U-Net model. Figure 6 As shown in the figure, the Inception convolution module combines multiple convolution operations, including 3×3, 5×5, and 7×7 convolution kernels, to replace the two 3×3 convolution operations and the connected ReLu activation function in each stage of the downsampling and upsampling process of the traditional U-Net model. Since multiple convolution kernels of different sizes are introduced at the same time, the number of model parameters and computational complexity will inevitably increase, resulting in a decrease in training efficiency.

[0071] Step 6: Connect module improvements

[0072] In order to solve the problem of decreased training efficiency of the improved U-Net model, combined with Figure 6 As shown in Figure 1, residual connections are introduced into the improved network architecture to replace the skip layer connections in the traditional U-Net model.

[0073] In addition, to further enhance the U-Net model's ability to perceive key areas in the image, a spatial attention mechanism is introduced into the Inception convolution module at each stage during the downsampling process of the improved U-Net model. The spatial attention mechanism adaptively assigns different weights to different areas in the image, giving higher weights to key areas in the image, thereby enabling the improved network architecture to more effectively focus on capturing the aggregate-mortar interface area in the image.

[0074] Step 7: Loss function improvement

[0075] Traditional U-Net models typically use a binary cross-entropy loss function for image segmentation tasks, which lacks consideration of pixel boundary differences. This results in poor performance when dealing with fine boundaries or complex geometric shapes. To address this lack of boundary information, we propose a joint loss function that balances classification accuracy and boundary consistency, enabling the improved U-Net model to better fit the target boundary while maintaining overall prediction performance. The joint loss function is expressed as follows:

[0076]

[0077] L joint =L BCE +λ·L boundary

[0078] Where, L joint represents the joint loss function, L BCE represents the binary cross entropy loss function, L boundary Represents the boundary loss function, N represents the number of pixels in the image, y true represents the true label, y pred Represents the model prediction results, Represents the sigmoid activation function, λ represents the weight between the binary cross entropy loss function and the boundary loss function, and takes λ = 0.3.

[0079] Step 8: Training set image annotation and sample expansion

[0080] For asphalt mixture specimens, representative samples need to be selected to train the improved U-Net model. After analysis and comparison, 60 cross-sectional images of different layers were selected as samples for each specimen, which can well represent all images of different positions of the specimen. The aggregates in the samples were labeled using commercial software Label, and the labeled images of the aggregates were obtained. In order to increase the sample size of the training set and enhance the generalization ability of the improved U-Net model, the data enhancement method was adopted to rotate the samples in the range of 30° to 270° on the two-dimensional plane, and the data was combined with the rotation of the sample to obtain the labeled images of the aggregates. Figure 7 As shown, the original image, rotated image and labeled image of a sample are displayed.

[0081] Step 9: Model training and image segmentation

[0082] The expanded training set is used to train the improved U-Net model, and the training parameters of the minimum joint loss function are saved. The trained improved U-Net model is used to segment other cross-sectional images, and the volume correlation error (RVE) and Dice coefficient are selected as evaluation indicators, which are expressed as follows:

[0083]

[0084] Where V seg represents the segmented image, V gt represents the real labeled image.

[0085] The segmentation effects of the cross-sectional images corresponding to the traditional U-Net model and the improved U-Net model were compared. Figure 8 As shown, the comparison results of evaluation indicators are combined with Figure 9 shown.

[0086] By comparing different segmentation effects, it can be found that there are many aggregate adhesion phenomena in the segmentation of the traditional U-Net model, while the aggregates in the adhesion part can be effectively separated after the segmentation processing of the improved U-Net model. At the same time, by analyzing different evaluation indicators, the improved U-Net model has improved segmentation effects in RVE and Dice coefficient compared with the watershed algorithm and the traditional U-Net model, which proves the excellence of the improved U-Net model proposed in this invention.

[0087] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other configurations without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations coming within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0088] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A deep learning segmentation method for asphalt mixture CT images, characterized by: The following steps are involved: Step 1: Acquire CT images of asphalt mixture Industrial CT was used to scan the asphalt mixture specimens to obtain cross-sectional images, which were then binarized and the Otsu method was used to remove the voids in the cross-sectional images. Step 2: Image equalization The gamma beam hardening correction algorithm is used to improve the grayscale value of the central part of the specimen and complete the equalization processing of the cross-sectional image; Step 3: Select the noise reduction algorithm window Combining the three indicators of average gradient amplitude, local variance and image visual fidelity (VIF), a 5×5 filter window was selected in the subsequent cross-sectional image processing. Step 4: Image noise reduction Adaptive bilateral filtering denoising algorithm based on local image information is used to adaptively adjust the spatial standard deviation and range standard deviation according to the local variance of the cross-sectional image. The calculation method is as follows: w((i,j),(x,y))=f s (||(i,j)-(x,y)||,σ s (i,j))·f r (|I(i,j)-I(x,y)|,σ r (i,j)) W(i,j)=∑ (x,y)∈Ω w((i,j),(x,y)) Where, represents the local variance within the window Ω, μ local (i, j) represents the local mean within the window Ω, Ω represents the window centered at (i, j), |Ω| represents the total number of pixels in the window, σ r (i, j) represents the adaptive spatial standard deviation, σ s (i, j) represents the standard deviation of the adaptive range, σ s·base Indicates the basic spatial standard deviation, with a value of 2, f s represents the spatial domain weight, ||(i,j)-(x,y)|| represents the Euclidean distance between pixels (x,y) and (i,j), and f r represents the range weight, |I(i,j)-I(x,y)| represents the grayscale value difference between pixels (x,y) and (i,j), w((i,j),(x,y)) represents the filter weight, W(i,j) represents the normalization factor, I out (i, j) represents the output image pixel value; Step 5: Convolutional module improvement The deep learning structure based on the U-Net model is improved. The Inception convolution module is used to replace the two 3×3 convolution operations and the ReLu activation function in each stage of the U-Net model's downsampling and upsampling process. Step 6: Connect module improvements Residual connections are introduced in the improved U-Net model to replace skip layer connections. During the downsampling process of the improved U-Net model, a spatial attention mechanism is introduced in the Inception convolution module of each stage to assign different weights to different areas in the cross-sectional image, giving higher weights to key areas in the cross-sectional image. The key area refers to the aggregate-mortar interface area. Step 7: Loss function improvement The joint loss function is used in the image segmentation task of the improved U-Net model, which is expressed as follows: L joint =L BCE +λ·L boundary Where N represents the number of pixels in the image, y true represents the true label, y pred Represents the model prediction results, represents the sigmoid activation function, λ = 0.3; Step 8: Training set image annotation and sample expansion For asphalt mixture specimens, multiple cross-sectional images of different layers are selected as samples, the aggregates in the samples are labeled, and the sample size is increased through data enhancement to form a training set; Step 9: Model training and image segmentation The expanded training set is used to train the improved U-Net model, and the training parameters of the minimum value of the joint loss function are saved. The trained improved U-Net model is used to segment other cross-sectional images.

2. The deep learning segmentation method for asphalt mixture CT images according to claim 1, characterized in that: In step 2, the calculation method of the gamma beam hardening correction algorithm is as follows: Where V out Represents the pixel value of each point in the output image, A is a constant with a value of 1, V in Represents the pixel value of each point in the input image, γ represents the gamma parameter, and its value is 0.

75.

3. The deep learning segmentation method for asphalt mixture CT images according to claim 1, characterized in that: In step 5, the Inception convolution module combines multiple convolution operations, including 3×3, 5×5, and 7×7 convolution kernels.

4. The deep learning segmentation method for asphalt mixture CT images according to claim 1, characterized in that: In the step eight, data enhancement refers to rotating the sample in a two-dimensional plane within a range of 30° to 270°.

Images