A computer vision algorithm driven ECC microstructure digitization method

Abstract

The present invention discloses an ECC microstructure digitization method driven by a computer vision algorithm, comprising the following steps: (1) obtaining a high-precision CT image and an original low-precision CT image, and performing linear downsampling; (2) constructing a planar super-resolution reconstruction model; (3) inputting the original CT image into the constructed planar super-resolution reconstruction model to obtain a super-resolution CT image; (4) preprocessing the super-resolution CT image and the high-precision CT image to obtain a super-resolution CT label image and a high-precision CT label image, and constructing a semantic segmentation model for automatic pixel-level recognition; (5) using a video frame increase model to perform inter-slice super-resolution on the obtained super-resolution CT image to obtain a three-dimensional super-resolution image; (6) inputting the three-dimensional super-resolution image into a semantic segmentation model, automatically recognizing the result image, and performing multi-index verification with the high-precision CT label image to obtain the ECC digitized microstructure. The present invention improves the recognition performance of the ECC microstructure.

Description

Technical Field

[0001] The present invention relates to the field of high-ductility fiber-reinforced cement-based composite materials and microscopic characterization technology thereof, and in particular to an ECC microstructure digitization method driven by a computer vision algorithm. Background Art

[0002] The microstructure determines the cracking behavior of the matrix and the fiber bridging ability. It is a direct manifestation of the micromechanical principles of ECC and is also the focus of ECC material design. Therefore, the accurate digitization of ECC microstructure is of great significance for the theoretical understanding and practical progress in this field. The progress of X-ray Computed Tomography (CT) has enabled the non-destructive mapping of the internal microstructure information of ECC (such as fibers, pores, and aggregates). However, current research still faces two major bottlenecks: 1) Enhancing CT accuracy requires the use of smaller samples, creating a challenging balance between accuracy and sample size; 2) The segmentation of cracks and pores is difficult because the grayscale values ​​of the two are highly similar in CT images. Specifically, the output slice resolution of CT is usually between 1000×1000 and 2500×2500. At this resolution, in order to achieve higher characterization accuracy, that is, smaller voxel size, the sample size of ECC is generally limited to less than (10mm)3. Obviously, compared with the conventional size of ECC (e.g., the dog-bone-shaped specimen used for uniaxial tensile test, whose region of interest is 80mm×30mm×13mm), such specimen size requirements make it difficult for existing CT-based ECC characterization methods to be applied in practice. On the other hand, most existing CT-based characterization methods are often based on grayscale and geometric topological features, and pixel classification is performed by threshold setting. However, due to noise interference and the similarity of grayscale values ​​and coarse-grainedness of each microstructure, it is difficult for traditional image processing methods or off-the-shelf semantic segmentation software (such as Dragonfly) to produce clear segmentation between three-dimensional cracks and other defects in ECC.

[0003] In recent years, with the development of artificial intelligence, deep learning algorithms have provided new ideas for various image processing tasks. At present, various image recognition, super-resolution reconstruction, semantic segmentation and other models based on convolutional neural networks and Transformer architectures have shown excellent performance in accuracy and robustness over traditional methods. However, in the field of materials science, the application of computer vision algorithms is still in a relatively backward stage. Therefore, how to combine the existing advanced achievements in the field of computer vision, address the inherent accuracy-size contradiction of CT characterization and the difficulty in accurately segmenting microstructures, establish a practical ECC microstructure digitization method, and realize the automatic and accurate identification of ECC three-dimensional cracks and defects, has a strong practical significance for promoting the technical application of ECC materials. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a computer vision algorithm driven ECC microstructure digitization method, which solves the contradiction between CT accuracy and ECC specimen size and the problem of insufficient microstructure recognition performance in the prior art.

[0005] Technical solution: The ECC microstructure digitization method driven by a computer vision algorithm described in the present invention comprises the following steps:

[0006] (1) obtaining a high-precision CT image and an original low-precision CT image, and linearly downsampling the original CT image;

[0007] (2) Using the dataset formed by the reference pair {downsampled CT image, original CT image}, a planar super-resolution reconstruction model is constructed;

[0008] (3) Inputting the original CT image into the constructed planar super-resolution reconstruction model to obtain a super-resolution CT image;

[0009] (4) Preprocessing the super-resolution CT images and high-precision CT images to obtain super-resolution CT label images and high-precision CT label images, and using the dataset formed by the reference pair {super-resolution CT image, super-resolution CT label image} to construct a semantic segmentation model for pixel-level automatic recognition;

[0010] (5) The obtained super-resolution CT image is subjected to inter-slice super-resolution using a video frame augmentation model to obtain a three-dimensional super-resolution image;

[0011] (6) The 3D super-resolution image is input into the semantic segmentation model to obtain the automatic recognition result image. After multi-index verification with the high-precision CT label image, the ECC digital microstructure is summarized.

[0012] Furthermore, step (1) is specifically as follows: using high-precision and low-precision CT devices to perform X-ray tomography on the ECC specimen, respectively, to obtain a high-precision CT image and an original low-precision CT image, and linearly down-sampling the original CT image to obtain a down-sampled CT image.

[0013] Furthermore, in step (2), an improved SwinIR is used to build a planar super-resolution reconstruction model, and the improved SwinIR network structure includes: convolution, a residual Swin module consisting of six serially connected Swin Transformers, a pixel reconstruction layer and a convolution layer.

[0014] Furthermore, step (2) includes the following steps:

[0015] (21) Constructing a super-resolution reconstruction loss function;

[0016] (22) The training is performed with the goal of minimizing the super-resolution reconstruction loss function and using gradient back-propagation as a strategy to continuously update the parameters of the current planar super-resolution reconstruction model until all iteration cycles of training are completed.

[0017] Furthermore, the formula of step (21) is as follows:

[0018]

[0019] Where x represents the input downsampled CT image, f(x) represents the output image of the planar super-resolution reconstruction model, y represents the original CT image as the true label reference, m and n represent the pixel length and pixel width of the model output image, respectively, Π(y, f(x)) represents the set of all joint distributions γ(y, f(x)) of the original CT image and the model output image, and inf represents the infimum of all possible joint distributions.

[0020] Furthermore, in step (4), the semantic segmentation model for pixel-level automatic recognition includes: a feature extraction layer, a fully connected layer, a feature fusion and a deconvolution layer, specifically as follows: features at different levels are obtained through multi-level convolution, and maximum pooling is used for downsampling between the levels to gradually change the size of the feature map. After full connection refinement, the deepest layer features are upsampled and fused with the feature output maps obtained by upsampling at the previous levels. Finally, the fused feature map is deconvolved to produce an output of the same size as the input.

[0021] Furthermore, in step (5), the video frame augmentation model is the pre-trained EMA-VFI, and the inter-slice super-resolution multiple of the video frame augmentation model is equal to the plane super-resolution multiple of the plane super-resolution reconstruction model.

[0022] Furthermore, in step (6), the multi-index verification includes the following steps:

[0023] (61) For the selected ECC specimen to be tested, obtain its high-precision CT label image and the corresponding automatic recognition result image obtained by the semantic segmentation model;

[0024] (62) For the automatic recognition result image, the crack width, crack volume, and pore diameter are calculated slice by slice using the R, G, and B pixel information;

[0025] (63) For high-precision CT labeled images, the crack width, crack area, and pore diameter are manually counted slice by slice, and the above results are compared with the results of the three indicators obtained in step (62).

[0026] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: It combines the advanced achievements in super-resolution reconstruction and semantic segmentation in the field of computer vision to achieve high-precision automatic recognition of ECC microstructures. The construction of the super-resolution reconstruction model greatly improves the voxel accuracy of the CT scan of the ECC specimen, breaks the inherent shackles between CT accuracy and ECC specimen size, and ensures subsequent accurate recognition. At the same time, through the semantic segmentation model self-trained for the task, the microstructures that are difficult to distinguish, such as cracks, pores, and aggregates, can be clearly distinguished. Through simulation and experiments, it can be obtained that the method of the present invention effectively expands the available size of the ECC specimen CT scan representation and improves the recognition performance of the ECC microstructure. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flow chart of the present invention;

[0028] Figure 2 The planar super-resolution reconstruction model structure of the present invention;

[0029] Figure 3 is the network structure of the semantic segmentation model of the present invention;

[0030] Figure 4 It is a comparison diagram of the index verification results of the present invention, wherein (a) is the crack width and (b) is the pore diameter;

[0031] Figure 5 It is the statistical result of the indicators of the present invention, wherein (a) is the crack width and (b) is the pore diameter.

[0032] Figure 6 The figures are comparison diagrams of the final results of the digitization of the three-dimensional microstructure of the present invention, wherein (a) is the first viewing angle and (b) is the second viewing angle. DETAILED DESCRIPTION

[0033] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0034] like Figure 1 As shown, an embodiment of the present invention provides an ECC microstructure digitization method driven by a computer vision algorithm, which specifically includes the following steps:

[0035] (1) X-ray tomography was performed on the ECC specimen using high-precision and low-precision CT devices, respectively, to obtain high-precision CT slice images (hereinafter referred to as high-precision CT images, with a size of 1170×1200) and original low-precision CT slice images (hereinafter referred to as original CT images, with a size of 600×616). The original CT images were then linearly downsampled by four times to obtain downsampled CT images (with a size of 150×154).

[0036] (2) Constructing a planar super-resolution reconstruction model, specifically including the following steps:

[0037] (2.1) Build a quadruple planar super-resolution reconstruction model architecture based on the improved SwinIR, such as Figure 2 As shown in the figure, shallow features are extracted through 3×3 convolution, and then deep features are extracted through 6 serial residual Swin modules, where each residual Swin module consists of 6 Swin Transformer layers and 3×3 convolutions. The Swin Transformer layer expands the multi-head self-attention of the original Transformer into a local attention and shift window mechanism. The information interaction between windows is achieved by alternating between conventional window partitioning and sliding window partitioning. The 3×3 convolution in the residual block uses spatial invariance to enhance the transition invariance of the extracted features. The 3×3 convolution after all residual Swin modules introduces the inductive bias of the convolution operation into the Transformer-based network, laying the foundation for the subsequent fusion of shallow and deep features. Finally, super-resolution reconstruction is achieved through a reconstruction module consisting of a pixel reconstruction layer and a convolution layer.

[0038] (2.2) Construct the super-resolution reconstruction loss function, and the specific calculation method is as follows:

[0039]

[0040] Where x represents the input downsampled CT image, f(x) represents the output image of the planar super-resolution reconstruction model, y represents the original CT image as the true label reference, m and n represent the pixel length and pixel width of the model output image, respectively, Π(y, f(x)) represents the set of all joint distributions γ(y, f(x)) of the original CT image and the model output image, and inf represents the infimum of all possible joint distributions.

[0041] (2.3) Using the dataset formed by the reference pair of {downsampled CT image, original CT image}, training is performed with the goal of minimizing the super-resolution reconstruction loss function, and gradient backpropagation is used as the strategy to continuously update the parameters of the current planar super-resolution reconstruction model until the training of all iterative cycles is completed.

[0042] (3) The original CT image (size is 600×616) is input into the constructed four-fold plane super-resolution reconstruction model to obtain a super-resolution CT image (size is 2400×2464);

[0043] (4) Preprocess the super-resolution CT image and the high-precision CT image to obtain the super-resolution CT label image and the high-precision CT label map. Use the data set formed by the reference pair {super-resolution CT image, super-resolution CT label image} to construct a semantic segmentation model that realizes pixel-level automatic recognition of ECC microstructures. The network structure of the semantic segmentation model is as follows: Figure 3 As shown in the figure, the semantic segmentation model for pixel-level automatic recognition includes: feature extraction layer, fully connected layer, feature fusion and deconvolution layer, which are as follows: features at different levels are obtained through multi-level convolution, and maximum pooling is used for down-sampling between layers to gradually change the size of the feature map. After full connection refinement, the deepest layer features are upsampled and fused with the feature output maps obtained by upsampling at the previous levels. Finally, the fused feature map is deconvolved to produce an output of the same size as the input.

[0044] (5) The super-resolution CT image obtained in step 3 is subjected to four-fold inter-slice super-resolution using the pre-trained EMA-VFI video frame augmentation model to obtain a three-dimensional super-resolution image with a consistent multiple of voxel accuracy improvement in all directions.

[0045] (6) Input the 3D super-resolution image into the semantic segmentation model to obtain the automatic recognition result image, and perform multi-index verification with the high-precision CT label image to obtain the ECC digital microstructure. The multi-index verification process specifically includes:

[0046] (6.1) For the selected ECC specimen to be tested, obtain its high-precision CT label image and the corresponding automatic recognition result image obtained by the semantic segmentation model.

[0047] (6.2) For the automatic recognition result image, the R, G, B pixel information is used to calculate the crack width, crack area, and pore diameter slice by slice.

[0048] (6.3) For high-precision CT labeled images, the crack width, crack volume, and pore diameter are manually counted slice by slice, and the above results are compared with the results of the three indicators obtained in step 6.2.

[0049] like Figure 4 The figure shows the comparison of the index verification results of an embodiment of the ECC microstructure digitization method driven by the computer vision algorithm of the present invention, where (a) is the crack width and (b) is the pore diameter. Twenty randomly selected cracks and twenty pores were manually measured at the same position of the slice, and the manually marked high-precision CT label image results were used as the real benchmark. The results are shown in FIG. Figure 5 Analysis Figure 4 and Figure 5It can be seen that the method proposed in the present invention is closest to the true reference value. For the crack width, compared with manual measurement, the average relative error of the original CT image by the threshold method is 19.41%, the high-precision CT image by the threshold method is 17.80%, and the average relative deviation of the method of the present invention is 5.14%, with a maximum of 9.17%. For the pore diameter, compared with manual measurement, the average relative error of the original CT image by the threshold method is 9.84%, the high-precision CT image by the threshold method is 8.59%, and the average relative deviation of the method of the present invention is 1.67%, with a maximum of 4.63%. The reliability of this method is verified by the above example comparison and quantitative analysis.

[0050] like Figure 6 The figure shows a comparison of the final results of the three-dimensional microstructure digitization of an embodiment of the ECC microstructure digitization method driven by the computer vision algorithm of the present invention, wherein (a) is the first viewing angle and (b) is the second viewing angle. The three-dimensional microstructure obtained by the present invention has a clear morphology, and no unconnected pores or other noise are introduced. For very fine cracks (as shown in the yellow box), the method of the present invention can still achieve accurate identification. For the original CT image or the high-precision CT image, the use of the threshold method will result in a large number of misidentified pores and then submerge the fine cracks.

Claims

1. A computer vision algorithm driven ECC microstructure digitization method, characterized in that: The following steps are involved: (1) Obtain high-precision CT images and original low-precision CT images, and perform linear downsampling on the original CT images; (2) Using the dataset formed by the reference pair {downsampled CT image, original CT image}, a planar super-resolution reconstruction model is constructed; The following steps are involved: (21) A quadruple planar super-resolution reconstruction model architecture based on the improved SwinIR was constructed, as follows: shallow features were extracted through 3×3 convolutions, and deep features were extracted through 6 serially connected residual Swin modules, where each residual Swin module consisted of 6 Swin Transformer layers and 3×3 convolutions. The Swin Transformer layer expanded the multi-head self-attention of the original Transformer into a local attention and shift window mechanism. The information interaction between windows was achieved by alternating between conventional window partitioning and sliding window partitioning. The 3×3 convolution within the residual block used spatial invariance to enhance the transition invariance of the extracted features. The 3×3 convolution after all residual Swin modules introduced the inductive bias of the convolution operation into the Transformer-based network, laying the foundation for the subsequent fusion of shallow and deep features. Finally, super-resolution reconstruction was achieved through a reconstruction module consisting of a pixel reconstruction layer and a convolution layer. (22) Construct a super-resolution reconstruction loss function; the formula is as follows: ； In the formula, x represents the input downsampled CT image, f ( x ) represents the output image of the planar super-resolution reconstruction model, y represents the original CT image as the true label reference, m and n Respectively represent the pixel length and pixel width of the model output image, Π ( y , f ( x )) represents all joint distributions of the original CT images and the model output images γ ( y , f ( x ))'s collection, inf represents the infimum among all possible joint distributions; (23) Training is performed with the goal of minimizing the super-resolution reconstruction loss function, using gradient back propagation as a strategy to continuously update the parameters of the current plane super-resolution reconstruction model until all iteration cycles of training are completed; (3) Input the original CT image into the constructed planar super-resolution reconstruction model to obtain a super-resolution CT image; (4) Preprocessing the super-resolution CT images and high-precision CT images to obtain super-resolution CT label images and high-precision CT label maps, and using the dataset formed by the reference pair {super-resolution CT images, super-resolution CT label images} to construct a pixel-level automatic recognition semantic segmentation model; wherein the pixel-level automatic recognition semantic segmentation model includes: feature extraction layer, fully connected layer, feature fusion and deconvolution layer, specifically as follows: obtaining features at different levels through multi-level convolution, using maximum pooling for downsampling between each level, and then after full connection refinement, upsampling the deepest layer features and fusion with the feature output maps obtained by upsampling at the previous levels, and finally deconvolving the fused feature maps to produce an output of the same size as the input; (5) The obtained super-resolution CT image is subjected to inter-slice super-resolution using a video frame augmentation model to obtain a three-dimensional super-resolution image; wherein the video frame augmentation model is a pre-trained EMA-VFI, and the inter-slice super-resolution multiple of the video frame augmentation model is equal to the planar super-resolution multiple of the planar super-resolution reconstruction model; (6) The 3D super-resolution image is input into the semantic segmentation model to obtain the automatic recognition result image, which is then verified with the high-precision CT label image through multiple indicators to obtain the ECC digital microstructure. The multiple indicator verification includes the following steps: (61) For the selected ECC specimen to be tested, obtain its high-precision CT label image and the corresponding automatic recognition result image obtained by the semantic segmentation model; (62) For the automatic recognition result image, the crack width, crack volume, and pore diameter are calculated slice by slice using the R, G, and B pixel information; (63) For high-precision CT label images, the crack width, crack area, and pore diameter are manually counted slice by slice, and the above results are compared with the results of the three indicators obtained in step (62).

2. The method for digitizing ECC microstructures driven by a computer vision algorithm according to claim 1, characterized in that: Step (1) is as follows: Use high-precision and low-precision CT devices to perform X-ray tomography on the ECC specimen to obtain a high-precision CT image and an original low-precision CT image, and linearly downsample the original CT image to obtain a downsampled CT image.

Images