Deep learning segmentation method for bituminous mixture CT image

Abstract

The invention discloses a deep learning segmentation method for an asphalt mixture CT image, and relates to the technical field of aggregate image segmentation. The method comprises the following steps: acquiring a CT image of an asphalt mixture and removing a gap part, completing equalization processing of the image through a gamma beam hardening correction algorithm, selecting a filtering window, performing noise reduction processing on the image by adopting a self-adaptive bilateral filtering noise reduction algorithm based on local information of the image, improving a U-Net model, and replacing standard convolution operation with an Inception convolution module. Layer skipping connection is replaced by residual connection, a space attention mechanism is introduced, a joint loss function is adopted, image samples are selected, aggregates are labeled, the sample size is increased through data enhancement to form a training set, the training set is used for model training, and the trained training set is used for segmenting other images. By performing equalization and noise reduction processing on the CT image and adopting the improved U-Net model, accurate segmentation of aggregate-mucilage boundary information can be realized, and the problem of aggregate adhesion is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of aggregate image segmentation, and in particular to a deep learning segmentation method for asphalt mixture CT images. Background Art

[0002] Asphalt mixture is composed of a variety of materials and is a non-uniform, multi-phase, multi-layered complex particle system. In order to understand the internal microstructure of asphalt mixture, it is necessary to collect cross-sectional images of asphalt mixture through industrial CT technology. Therefore, accurate segmentation of voids, asphalt mortar and aggregates in asphalt mixture CT images is of great significance to the evolution of microstructure.

[0003] At present, image segmentation methods are mainly divided into traditional image segmentation methods and deep learning-based image segmentation methods. Traditional threshold-based image segmentation methods such as watershed have large differences in processing capabilities for different types of asphalt mixture CT images, and the processing effect is poor. The problem of aggregate adhesion is more serious during segmentation, and it is impossible to remove aggregate adhesion through pixel corrosion. Compared with traditional threshold segmentation algorithms, deep algorithms can efficiently use small sample data, achieve accurate boundary segmentation through jump connections, and capture multi-scale feature information at the same time, and have good generalization capabilities.

[0004] However, in the process of asphalt mixture CT image segmentation, due to the close boundary threshold between asphalt mortar and aggregate, the deep learning still has the problem of aggregate adhesion in the segmentation process, which makes it difficult to accurately analyze the aggregate migration law in the mesostructure study. Therefore, it is necessary to improve the deep learning method of asphalt mixture CT images to solve the problem of aggregate segmentation adhesion, which is of great significance for improving the accuracy of aggregate segmentation and the study of asphalt mixture mesostructure. Summary of the invention

[0005] In order to address the shortcomings of the background technology, the present invention provides a deep learning segmentation method for asphalt mixture CT images, which can achieve accurate segmentation of aggregate-mortar boundary information and avoid aggregate adhesion problems by equalizing and denoising CT images and adopting an improved U-Net model.

[0006] To achieve the above object, the present invention adopts the following technical solution: a deep learning segmentation method for asphalt mixture CT images, comprising the following steps:

[0007] Step 1: Collect CT images of asphalt mixture

[0008] The asphalt mixture specimens were scanned by industrial CT to obtain cross-sectional images, which were then binarized and the voids in the cross-sectional images were removed by the Otsu method.

[0009] Step 2: Image equalization

[0010] The gamma beam hardening correction algorithm is used to improve the gray value of the central part of the specimen and complete the equalization processing of the cross-sectional image;

[0011] Step 3: Select the noise reduction algorithm window

[0012] Combining the three indicators of average gradient amplitude, local variance and image visual fidelity VIF, a 5×5 filter window was selected in the subsequent cross-sectional image processing;

[0013] Step 4: Image noise reduction

[0014] Adaptive bilateral filtering denoising algorithm based on local information of the image is used to adaptively adjust the spatial standard deviation and range standard deviation according to the local variance of the cross-sectional image. The calculation method is as follows:

[0015]

[0016]

[0017] w((i,j),(x,y))=f s (||(i,j)-(x,y)||,σ s (i,j))·f r (|I(i,j)-I(x,y)|,σ r (i,j))

[0018] W(i,j)=∑ (x,y)∈Ω w((i,j),(x,y))

[0019]

[0020] In the formula, represents the local variance within the window Ω, μ local (i, j) represents the local mean within the window Ω, Ω represents the window centered at (i, j), |Ω| represents the total number of pixels in the window, σ r (i,j) represents the adaptive spatial standard deviation, σ s (i,j) represents the standard deviation of the adaptive range, σ s·base represents the basic spatial standard deviation, with a value of 2, f s represents the spatial domain weight, ||(i,j)-(x,y)|| represents the Euclidean distance between pixels (x,y) and (i,j), and f r represents the range weight, |I(i,j)-I(x,y)| represents the grayscale value difference between pixels (x,y) and (i,j), w((i,j),(x,y)) represents the filter weight, W(i,j) represents the normalization factor, Iout (i, j) represents the output image pixel value;

[0021] Step 5: Convolutional module improvement

[0022] The deep learning structure based on the U-Net model is improved by using the Inception convolution module to replace the two 3×3 convolution operations and the connected ReLu activation function in each stage of the downsampling and upsampling process of the U-Net model;

[0023] Step 6: Connect module improvements

[0024] In the improved U-Net model, residual connections are introduced to replace skip layer connections. In the downsampling process of the improved U-Net model, a spatial attention mechanism is introduced in the Inception convolution module of each stage to assign different weights to different areas in the cross-sectional image, giving higher weights to the key areas in the cross-sectional image, which refer to the aggregate-mortar interface area.

[0025] Step 7: Loss function improvement

[0026] The joint loss function is used in the image segmentation task of the improved U-Net model, which is expressed as follows:

[0027]

[0028] L joint =L BCE +λ·L boundary

[0029] Where N represents the number of pixels in the image, y true represents the true label, y pred Represents the model prediction result, represents the sigmoid activation function, λ=0.3;

[0030] Step 8: Training set image annotation and sample expansion

[0031] For asphalt mixture specimens, multiple cross-sectional images of different layers are selected as samples, the aggregates in the samples are labeled, and the sample size is increased through data enhancement to form a training set;

[0032] Step 9: Model training and image segmentation

[0033] The expanded training set is used to train the improved U-Net model, and the training parameters of the minimum value of the joint loss function are saved. The trained improved U-Net model is used to segment other cross-sectional images.

[0034] Furthermore, in step 2, the calculation method of the gamma beam hardening correction algorithm is as follows:

[0035]

[0036] Where V out Represents the pixel value of each point in the output image, A is a constant with a value of 1, V in Represents the pixel value of each point in the input image, γ represents the gamma parameter, and its value is 0.75.

[0037] Furthermore, in step 5, the Inception convolution module combines multiple convolution operations, including 3×3, 5×5 and 7×7 convolution kernels.

[0038] Furthermore, in step eight, data enhancement refers to rotating the sample in a range of 30° to 270° on a two-dimensional plane.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention collects cross-sectional images of asphalt mixture specimens through industrial CT, and performs equalization and noise reduction processing on the CT images, and adopts an improved U-Net model. It can effectively reduce noise according to local information in the cross-sectional images, realize accurate segmentation of aggregate-mortar boundary information, avoid aggregate adhesion problems, and has the advantages of accuracy, strong adaptability, and high efficiency. It is easy to promote and popularize, and is helpful for the microstructure research of asphalt mixtures. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a cross-sectional image before and after the gap portion is removed in a specific embodiment;

[0041] Figure 2 is a thermal map of the cross-sectional image before and after equalization processing in a specific implementation manner;

[0042] Figure 3 is a diagram of calculation results of three indicators under different filter windows in a specific implementation mode;

[0043] Figure 4 is a comparison position and noise reduction effect diagram of a cross-sectional image in a specific implementation manner;

[0044] Figure 5 It is a deep learning structure diagram of the traditional U-Net model in a specific implementation method;

[0045] Figure 6 is a deep learning structure diagram of an improved U-Net model in a specific implementation manner;

[0046] Figure 7 is a rotated and annotated image of a training set sample in a specific implementation manner;

[0047] Figure 8is a comparison diagram of the cross-sectional image segmentation effect in a specific implementation manner;

[0048] Fig. 9 It is a diagram of comparison results of evaluation indicators in a specific implementation method. DETAILED DESCRIPTION

[0049] The technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0050] like Figures 1 to 9 As shown, a deep learning segmentation method for asphalt mixture CT images includes the following steps:

[0051] Step 1: Collect CT images of asphalt mixture

[0052] The asphalt mixture specimen was scanned by industrial CT to obtain the cross-sectional image, and then the cross-sectional image was binarized and the gaps in the cross-sectional image were removed by the Otsu method. Figure 1 As shown, cross-sectional images obtained before and after removing the void portion are shown.

[0053] Step 2: Image equalization

[0054] On the basis of removing the cross-sectional image of the void part, the gray value of the central part of the specimen is improved by the gamma beam hardening correction algorithm to complete the cross-sectional image equalization processing. Figure 2 As shown in the figure, the thermal map of the cross-section image before and after equalization is shown. The calculation method is as follows:

[0055]

[0056] Where V out Represents the pixel value of each point in the output image, A is a constant with a value of 1, V in Represents the pixel value of each point in the input image, γ represents the gamma parameter, and its value is 0.75.

[0057] Step 3: Select the noise reduction algorithm window

[0058] In the process of image denoising, the filter window is an important parameter. The size of different filter windows will directly affect the blur degree of the image. A larger filter window usually leads to a more obvious blur effect, while a smaller filter window can retain more image detail information. In order to compare the effects of different filter windows on the blur degree of cross-sectional images, the appropriate filter window is selected by combining the average gradient amplitude, local variance and image visual fidelity (VIF). Figure 3 As shown in the figure, the calculation results of the three indicators under different filter windows are displayed. It can be seen that when the filter window is 5×5 in size, the average gradient amplitude indicator decreases rapidly, indicating that the cross-sectional image begins to become blurred, while the local variance and image visual fidelity indicators decrease significantly, indicating that the cross-sectional image details begin to decrease significantly at this time. Therefore, the above three indicators are comprehensively evaluated and a 5×5 filter window is selected in the subsequent cross-sectional image processing.

[0059] Step 4: Image noise reduction

[0060] In order to meet the denoising requirements of different areas of the cross-sectional image, an adaptive bilateral filtering denoising algorithm based on local information of the image is proposed. The spatial standard deviation and the range standard deviation are adaptively adjusted according to the local variance of the cross-sectional image. In this way, different smoothing intensities can be applied in high-variation areas (aggregate-mortar interface areas) and low-variation areas (aggregate areas) to improve the denoising effect. The calculation method is as follows:

[0061]

[0062]

[0063] w((i,j),(x,y))=f s (||(i,j)-(x,y)||,σ s (i,j))·f r (|I(i,j)-I(x,y)|,σ r (i,j))

[0064] W(i,j)=∑ (x,y)∈Ω w((i,j),(x,y))

[0065]

[0066] Where (i, j) represents the pixel position of the image, represents the local variance within the window Ω, μ local (i, j) represents the local mean within the window Ω, Ω represents the window centered at (i, j), |Ω| represents the total number of pixels in the window, σ r (i,j) represents the adaptive spatial standard deviation, σ s(i,j) represents the standard deviation of the adaptive range, σ s·base represents the basic spatial standard deviation, with a value of 2, f s represents the spatial domain weight, ||(i,j)-(x,y)|| represents the Euclidean distance between pixels (x,y) and (i,j), and f r represents the range weight, |I(i,j)-I(x,y)| represents the grayscale value difference between pixels (x,y) and (i,j), w((i,j),(x,y)) represents the filter weight, W(i,j) represents the normalization factor, I out (i,j) represents the output image pixel value.

[0067] In order to verify the denoising effect of the algorithm, Figure 4 As shown in the figure, a representative comparison position is selected for the cross-sectional image, and the denoising effects of the original image of the comparison position, conventional wavelet filtering, and the proposed adaptive bilateral filtering are shown. In the figure, the area with a flat grayscale pixel value is the aggregate interface, and the area with a sudden change in grayscale pixel value is the aggregate-mortar interface. It can be seen that at the aggregate-mortar interface, both the wavelet filtering algorithm and the adaptive bilateral filtering algorithm can effectively retain the grayscale gradient difference, and compared with the wavelet filtering algorithm, the grayscale change of the aggregate interface after processing by the adaptive bilateral filtering algorithm is smoother, indicating that the adaptive bilateral filtering algorithm can effectively eliminate the aggregate interface noise.

[0068] Step 5: Convolutional module improvement

[0069] The deep learning structure of the traditional U-Net model is mainly divided into downsampling process and upsampling process. Figure 5 As shown in the figure, each stage of the downsampling process includes two 3×3 convolution operations and access to the ReLu activation function, followed by a 2×2 maximum pooling operation. In the upsampling process, each stage first performs a 2×2 deconvolution operation, and then concatenates the upsampled feature map with the corresponding feature map of the encoder through a skip-layer connection. The concatenated feature map undergoes two 3×3 convolution operations and accesses the ReLu activation function.

[0070] In order to further improve the feature extraction capability of the U-Net model, the Inception convolution module is used to replace the standard convolution operation in the traditional U-Net model to improve the deep learning structure of the U-Net model. Figure 6 As shown in the figure, the Inception convolution module combines multiple convolution operations, including 3×3, 5×5 and 7×7 convolution kernels, to replace the two 3×3 convolution operations and the connected ReLu activation function in each stage of the downsampling and upsampling process of the traditional U-Net model. Due to the introduction of multiple convolution kernels of different sizes at the same time, it will inevitably increase the number of model parameters and computational complexity, resulting in a decrease in training efficiency.

[0071] Step 6: Connect module improvements

[0072] In order to solve the problem of decreased training efficiency of the improved U-Net model, combined with Figure 6 As shown in Figure 1, residual connections are introduced in the improved network architecture to replace the skip layer connections in the traditional U-Net model.

[0073] In addition, in order to further enhance the U-Net model's perception of key areas in the image, a spatial attention mechanism is introduced in the Inception convolution module of each stage during the downsampling process of the improved U-Net model. The spatial attention mechanism adaptively assigns different weights to different areas in the image, giving higher weights to key areas in the image, so that the improved network architecture can more effectively focus on capturing the aggregate-mortar interface area in the image.

[0074] Step 7: Loss function improvement

[0075] Traditional U-Net models usually use binary cross entropy loss functions in image segmentation tasks, which lacks consideration of pixel boundary differences, making them perform poorly when dealing with fine boundaries or complex geometric shapes. To this end, a joint loss function is proposed to address the lack of boundary information differences. By balancing classification accuracy and boundary consistency, the improved U-Net model can better fit the target boundary while ensuring the overall prediction effect. The joint loss function is expressed as follows:

[0076]

[0077] L joint =L BCE +λ·L boundary

[0078] Where, L joint represents the joint loss function, L BCE represents the binary cross entropy loss function, L boundary represents the boundary loss function, N represents the number of pixels in the image, and y true represents the true label, y pred Represents the model prediction result, represents the sigmoid activation function, λ represents the weight between the binary cross entropy loss function and the boundary loss function, and takes λ = 0.3.

[0079] Step 8: Training set image annotation and sample expansion

[0080] For asphalt mixture specimens, representative samples need to be selected to train the improved U-Net model. After analysis and comparison, 60 cross-sectional images of different layers are selected as samples for each specimen, which can well represent all images of different positions of the specimen. The aggregates in the samples are labeled using commercial software Label, and the labeled images of the aggregates are obtained. In order to increase the sample size of the training set and enhance the generalization ability of the improved U-Net model, the data enhancement method is adopted to rotate the samples in the range of 30° to 270° on the two-dimensional plane, and the data is combined with the data from the image processing software. Figure 7 As shown, the original image, rotated image and labeled image of a certain sample are displayed.

[0081] Step 9: Model training and image segmentation

[0082] The expanded training set is used to train the improved U-Net model, and the training parameters of the minimum joint loss function are saved. The trained improved U-Net model is used to segment other cross-sectional images, and the volume correlation error (RVE) and Dice coefficient are selected as evaluation indicators, which are expressed as follows:

[0083]

[0084] Where V seg represents the segmented image, V gt represents the real labeled image.

[0085] The segmentation effects of the cross-sectional images corresponding to the traditional U-Net model and the improved U-Net model are compared. Figure 8 As shown in the figure, the comparison results of evaluation indicators are combined with Fig. 9 shown.

[0086] By comparing different segmentation effects, it can be found that there are many aggregate adhesion phenomena in the segmentation of the traditional U-Net model, while the aggregates in the adhesion part can be effectively separated after the segmentation processing using the improved U-Net model. At the same time, by analyzing different evaluation indicators, the improved U-Net model has improved segmentation effects in RVE and Dice coefficient compared with the watershed algorithm and the traditional U-Net model, which proves the excellence of the improved U-Net model proposed in the present invention.

[0087] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other forms of assembly without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered in all respects as exemplary and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations within the meaning and range of equivalents of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0088] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A deep learning segmentation method for asphalt mixture CT images, characterized in that: The following steps are involved: Step 1: Collect CT images of asphalt mixture The asphalt mixture specimens were scanned by industrial CT to obtain cross-sectional images, which were then binarized and the voids in the cross-sectional images were removed by the Otsu method. Step 2: Image equalization The gamma beam hardening correction algorithm is used to improve the grayscale value of the center of the specimen and complete the equalization processing of the cross-sectional image. Step 3: Select the noise reduction algorithm window Combining the three indicators of average gradient amplitude, local variance and image visual fidelity VIF, a 5×5 filter window was selected in the subsequent cross-sectional image processing; Step 4: Image noise reduction Adaptive bilateral filtering denoising algorithm based on local information of the image is used to adaptively adjust the spatial standard deviation and range standard deviation according to the local variance of the cross-sectional image. The calculation method is as follows: w((i,j),(x,y))=f s (||(i,j)-(x,y)||,σ s (i,j))·f r (|I(i,j)-I(x,y)|,σ r (i,j)) W(i,j)=∑ (x,y)∈Ω w((i,j),(x,y)) In the formula, represents the local variance within the window Ω, μ local (i, j) represents the local mean within the window Ω, Ω represents the window centered at (i, j), |Ω| represents the total number of pixels in the window, σ r (i,j) represents the adaptive spatial standard deviation, σ s (i,j) represents the standard deviation of the adaptive range, σ s·base represents the basic spatial standard deviation, with a value of 2, f s represents the spatial domain weight, ||(i,j)-(x,y)|| represents the Euclidean distance between pixels (x,y) and (i,j), and f r represents the range weight, |I(i,j)-I(x,y)| represents the grayscale value difference between pixels (x,y) and (i,j), w((i,j),(x,y)) represents the filter weight, W(i,j) represents the normalization factor, I out (i, j) represents the output image pixel value; Step 5: Convolutional module improvement The deep learning structure based on the U-Net model is improved by using the Inception convolution module to replace the two 3×3 convolution operations and the connected ReLu activation function in each stage of the downsampling and upsampling process of the U-Net model; Step 6: Connection module improvement In the improved U-Net model, residual connections are introduced to replace skip layer connections. In the downsampling process of the improved U-Net model, a spatial attention mechanism is introduced in the Inception convolution module of each stage to assign different weights to different areas in the cross-sectional image, giving higher weights to the key areas in the cross-sectional image, which refer to the aggregate-mortar interface area. Step 7: Loss function improvement The joint loss function is used in the image segmentation task of the improved U-Net model, which is expressed as follows: L joint =L BCE +λ·L boundary Where N represents the number of pixels in the image, y true represents the true label, y pred Represents the model prediction result, represents the sigmoid activation function, λ=0.3; Step 8: Training set image annotation and sample expansion For asphalt mixture specimens, multiple cross-sectional images of different layers are selected as samples, the aggregates in the samples are labeled, and the sample size is increased through data enhancement to form a training set; Step 9: Model training and image segmentation The expanded training set is used to train the improved U-Net model, and the training parameters of the minimum value of the joint loss function are saved. The trained improved U-Net model is used to segment other cross-sectional images.

2. The deep learning segmentation method for asphalt mixture CT images according to claim 1 is characterized in that: In step 2, the calculation method of the gamma beam hardening correction algorithm is as follows: Where V out Represents the pixel value of each point in the output image, A is a constant with a value of 1, V in Represents the pixel value of each point in the input image, γ represents the gamma parameter, and its value is 0.

75.

3. The deep learning segmentation method for asphalt mixture CT images according to claim 1 is characterized in that: In step 5, the Inception convolution module combines multiple convolution operations, including 3×3, 5×5 and 7×7 convolution kernels.

4. The deep learning segmentation method for asphalt mixture CT images according to claim 1, characterized in that: In the step eight, data enhancement refers to rotating the sample in a two-dimensional plane within a range of 30° to 270°.

Images