Semantic segmentation-based asphalt concrete CT image segmentation prediction method

Abstract

The invention provides an asphalt concrete CT image segmentation prediction method based on semantic segmentation. According to the method, an asphalt concrete CT image segmentation off-line database is established based on CT scanning and digital image processing technologies, then an AEU-net neural network model is developed based on semantic segmentation, an attention mechanism and a supervised learning theory, and finally the AEU-net neural network model is trained by adopting the off-line database to obtain an AEUnet-DIP proxy model. According to the method, the efficient mapping capability of the neural network model is fully utilized, the aggregate phase morphological information of the asphalt concrete CT image is accurately extracted, and the problems of low efficiency, weak universality, wrong segmentation and the like of an existing digital image processing method are effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of tomographic image processing. Background Art

[0002] CT image set is a sequential collection of multiple homologous two-dimensional CT images that can characterize the internal geometric morphology of an object. It is usually acquired by rotating and processing along the surface contour of the specimen using a computed tomography (CT) device. Asphalt concrete CT images contain explicit morphological information such as the geometric characteristics, density distribution, and material boundaries of heterogeneous materials inside the specimen. After information purification through image segmentation, it can be used for research in microstructural feature analysis, mechanical behavior analysis, and damage self-healing behavior prediction.

[0003] The pixel information of asphalt concrete CT images is represented by 8-bit code "0-255", "0" value pixels are dark, "255" value pixels are white and bright, and "1-254" value pixels display characteristics in a linear distribution according to this rule. Among them, the dark area formed by the "0" value pixel cluster represents the gap, the gray and dark area formed by multiple medium and low value pixel clusters represents asphalt mortar, and the white and bright area formed by multiple medium and high value pixel clusters represents aggregate.

[0004] The essence of CT imaging is imaging of material density differences. Physical limitations lead to inherent defects in CT images. Specifically, the overall brightness of CT images presents the characteristics of "low inside and high around, random distribution of noise", and there is a large range of union of pixel grayscale value intervals of aggregate phase and asphalt mortar phase. This greatly increases the difficulty of image processing.

[0005] Digital Image Processing (DIP) is a technology that processes and analyzes the pixel information features of two-dimensional images through the integration of multiple algorithms to achieve image enhancement, feature extraction, noise reduction and other effects. In the field of asphalt concrete CT image processing, the key goal is to extract the aggregate phase morphology information in CT images.

[0006] The existing asphalt concrete CT image segmentation methods have problems such as low efficiency, weak versatility and incorrect segmentation. In the current technical system, in order to effectively extract the aggregate phase in the asphalt concrete CT image, it is necessary to analyze the characteristics of the inherent defects of the asphalt concrete CT image to be processed in advance, and then match and design a suitable morphological processing algorithm. Secondly, the differences in physical conditions such as specimen quality, environmental factors, and CT power lead to huge differences in the grayscale value distribution of CT images from different sources. The existing morphological processing algorithms cannot meet the requirements for the processing effect of asphalt concrete CT images from different sources. Finally, the essence of digital image processing is to use a fixed algorithm to process the global or artificially defined area of ​​the image. However, the existing asphalt concrete CT image segmentation methods have different sensitivities to aggregates of different shapes and contour features, resulting in the incorrect segmentation of some aggregate phases, which is specifically manifested as incomplete aggregates, missing aggregates, and long strips of aggregates being segmented into multiple linearly distributed fine aggregates. Summary of the invention

[0007] In order to overcome the shortcomings of the prior art, the present invention proposes a CT image segmentation prediction method for asphalt concrete specimens based on semantic segmentation. The method establishes an offline database for asphalt concrete CT image segmentation based on CT scanning and digital image processing technology, and then develops an AEU-net neural network model based on semantic segmentation, attention mechanism and supervised learning theory. Finally, the offline database is used to train U-net to establish an AEUnet-DIP proxy model. The AEUnet-DIP proxy model can be used to achieve efficient and accurate mapping from the CT original image to the image segmentation result. The present invention makes full use of the efficient mapping capability of the neural network model to accurately extract the aggregate phase morphological information of the asphalt concrete CT image, effectively solving the problems of low efficiency, weak versatility and incorrect segmentation of existing digital image processing methods.

[0008] To achieve the above object, the technical solution of the present invention is as follows:

[0009] The asphalt concrete CT image segmentation prediction method based on semantic segmentation includes the following steps:

[0010] Step 1: Initial CT image acquisition: Prepare the asphalt concrete CT scanning specimen, determine the scanning direction and use CT scanning technology to obtain its initial CT tomographic image set;

[0011] Step 2: Establish and expand the offline database for image segmentation;

[0012] Step 3: Optimization and development of neural network model: According to the data structure characteristics of the image segmentation offline database, a U-net neural network model considering the attention mechanism is constructed based on semantic segmentation, attention mechanism and supervised learning theory, namely the AEUnet neural network model;

[0013] Step 4: Neural network model training: Use the image segmentation offline database established in step 2 to train the AEUnet neural network model to obtain the AEUnet-DIP proxy model;

[0014] Step 5: Image segmentation prediction: The AEUnet-DIP proxy model is used to perform aggregate phase segmentation prediction on the asphalt concrete CT images to be tested.

[0015] Preferably, the CT image in step 1 includes an 8-Bit RGB color image and a grayscale image.

[0016] Preferably, each CT image in step 1 contains all pixel information of the complete specimen on the cross section, and redundant pixel positions are filled with "0" value pixels.

[0017] Furthermore, the step 2 includes:

[0018] According to the grayscale distribution characteristics and inherent defect characteristics of the initial CT tomographic image, the DIP processing flow is designed to segment the different components of the image;

[0019] The initial CT original image and the corresponding image segmentation result are taken as a set of data pairs, and a high-quality image segmentation offline database is established based on the initial CT tomographic image set;

[0020] Using the CT scanning computer simulation algorithm, new tomographic images in different directions are extracted from the asphalt concrete CT tomographic image set obtained in step one to expand the number of image segmentation offline databases.

[0021] Furthermore, the image segmentation result in step 2 includes aggregate phase segmentation result and asphalt mortar phase segmentation result.

[0022] Furthermore, the establishment and expansion of the image segmentation offline database in step 2 includes using the DIP technology to improve the quality of the image segmentation offline database and using a CT scanning computer simulation algorithm process to expand the number of image segmentation offline databases.

[0023] Preferably, the DIP processing flow in step 2 includes steps such as image filtering, pixel filling, maximum threshold segmentation, watershed segmentation, and pixel restoration.

[0024] Preferably, the CT scanning computer simulation algorithm in step 2 includes steps such as voxel reconstruction, model coordinate space rotation and voxel model re-slicing.

[0025] Furthermore, the AEUnet neural network model, based on the basic semantic segmentation network U-net, realizes feature fusion optimization under the receptive field of each level of the neural network based on the channel attention mechanism to avoid the interference of irrelevant features on the prediction results.

[0026] Preferably, the training process of the AEUnet neural network model can reserve a transfer learning interface, that is, the existing AEUnet neural network model training results can be superimposed with the CT image scanning results from different sources to further improve the image segmentation prediction accuracy and versatility of the AEUnet neural network model. When using new data for training, the image-label specifications must be consistent with the training data.

[0027] The present invention has the following beneficial effects:

[0028] (1) High segmentation efficiency: The image segmentation efficiency of the AEUnet-DIP proxy model obtained after training is greatly improved, and the efficiency ratio is further improved as the image resolution increases.

[0029] (2) Enhanced versatility: The trained AEUnet-DIP proxy model can achieve efficient and accurate segmentation of asphalt concrete CT images of any image format, shape, resolution, and size.

[0030] (3) Improved accuracy: The AEUnet-DIP proxy model obtained after training has a higher segmentation accuracy for the slender aggregate phase.

[0031] (4) No threshold for use: The trained AEUnet-DIP agent model does not require any prior knowledge or skills (such as morphological processing knowledge and programming skills) for the operator. Anyone can easily achieve high-quality asphalt concrete CT image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a complete technical roadmap of the present invention.

[0033] Figure 2 This is a schematic diagram of the CT scan results.

[0034] Figure 3 Schematic diagram of the inherent defects in the gray value distribution of asphalt concrete CT images.

[0035] Figure 4 This is the DIP processing flow chart.

[0036] Figure 5 Comparison between the original CT image and the corresponding image segmentation results.

[0037] Figure 6 Schematic diagram of offline database data expansion method.

[0038] Figure 7 This is an example of a two-layer AEUnet neural network.

[0039] Figure 8This is the complete AEUnet neural network model architecture.

[0040] Fig. 9 This is the loss function curve of the AEUnet neural network model training process.

[0041] Fig.10 This is an overall comparison of the segmentation effects of CT images from the same source.

[0042] Fig.11 This is a detailed comparison of the segmentation effects of CT images from the same source.

[0043] Fig.12 This is a comparison chart of the segmentation effects of CT images from different sources and with different shapes.

[0044] Fig.13 Comparison of segmentation efficiency of images with different resolutions. DETAILED DESCRIPTION

[0045] The technical solution provided by the present application will be further described below in conjunction with specific embodiments and accompanying drawings. The advantages and features of the present application will become more apparent with the following description.

[0046] The present invention is an asphalt concrete CT image segmentation prediction method based on semantic segmentation, which mainly includes the steps of establishing an asphalt concrete CT image segmentation database, building and training a U-net semantic segmentation neural network model, and the main steps can be completed by computer programming.

[0047] The specific steps of this method are as follows:

[0048] Step 1: Initial CT image acquisition.

[0049] refer to Figure 2 According to the requirements of the "Highway Engineering Asphalt and Asphalt Mixture Test Procedure" and the "Highway Asphalt Pavement Construction Technical Specification", the parameters such as aggregate gradation, void ratio and oil-stone ratio were determined, and a cylindrical asphalt concrete specimen with a height of 75mm and a diameter of 65mm was prepared using a rotary compactor. The German Diondo d2 universal micro-nano focusing CT system equipped with a 270kV and 72μA X-ray source was used for image acquisition, and 750 tomographic images were obtained along the height direction of the specimen to obtain the initial CT tomographic image set. Among them, the section spacing is 0.1mm, the effective resolution of the tomographic image is 914 pixels × 914 pixels, and the resolution is set to 78 microns / pixel.

[0050] Furthermore, the asphalt concrete specimens may include hexahedrons, cylinders, and specimens having other arbitrary shapes and size characteristics.

[0051] Furthermore, the CT image in step 1 includes an 8-Bit RGB color image and a grayscale image.

[0052] Furthermore, each CT image in the step 1 contains all pixel information of the complete specimen on the cross section, and redundant pixel positions are filled with "0" value pixels.

[0053] Step 2: Establish and expand the image segmentation offline database.

[0054] refer to Figure 3 Due to physical limitations, the initial CT original image collected in step 1 has the inherent defects of grayscale value "medium low high frequency and random noise distribution", and the degree of inherent defects of CT images from different sources varies. This leads to a large range of grayscale value intervals of different components in the actual asphalt concrete CT image, which greatly increases the difficulty of CT image segmentation. Specifically, when the segmentation threshold is underestimated, part of the asphalt mortar phase is mistakenly identified as the aggregate phase, and the aggregate phase is severely adhered; when the segmentation threshold is overestimated, part of the aggregate phase is mistakenly identified as the asphalt mortar phase, the aggregate is over-segmented, and a large number of holes appear inside.

[0055] In order to ensure the quality and uniformity of the image segmentation results, the present invention designs a DIP processing flow to segment different components of the image based on the grayscale distribution characteristics and inherent defect characteristics of the initial CT original image. Figure 4 The algorithms involved in the DIP processing flow adopted by the present invention are as follows: a pixel filling algorithm is used to expand the CT image to 3840 pixels × 3840 pixels; a grayscale algorithm is used to uniformly adjust the image grayscale value range to 8-Bit; a local mean filtering algorithm is used to realize image denoising, and the local filtering kernel function selects the four-connected domain operator [0 0.3 0, 0.3 0.3 0.3, 0 0.3 0]; a maximum threshold segmentation algorithm based on annular partitioning is used to evenly divide the CT image from the center into 1 circular area and 4 annular areas according to the distance, and the OTSU algorithm is used to realize maximum threshold segmentation and reorganization; a watershed segmentation algorithm is used to remove the incorrectly connected aggregate phase; a pixel restoration algorithm is used to restore the image to 914 pixels × 914 pixels, and finally a binary image containing the aggregate phase segmentation results and the asphalt mortar phase segmentation results is obtained.

[0056] refer to Figure 5 , the CT original image and image segmentation results are cropped to obtain 512 pixels × 512 pixels of CT original image and image segmentation results. The CT original image and the corresponding image segmentation result (label) are used as a set of data pairs, and an image segmentation offline database is established based on the initial CT tomographic image set (containing a total of 750 sets of original image-label data pairs).

[0057] refer to Figure 6(a) Based on the voxel reconstruction method, the model coordinate space rotation algorithm and the voxel model re-slicing algorithm, the CT scanning process at multiple angles is simulated in the computer memory. The above 750 sets of original image-label data pairs can be regenerated into 650 sets of original image-label data pairs containing "new image segmentation knowledge". Figure 6 (b) For a three-dimensional voxel model, the selectable CT scanning directions are theoretically infinite, but the microscopic structures of the CT scanning results obtained in the scanning directions that are close to the limit are basically the same, and it is impossible to provide more "effective image segmentation knowledge" to the neural network in the subsequent training process. Therefore, this embodiment sets the scanning angle interval to 20°, rotates the model around the x-axis and y-axis in turn, and finally obtains 81 asphalt concrete CT image segmentation data pairs in different directions. At the same time, if the CT scanning layer spacing is too small, the geometric knowledge of adjacent CT images is basically the same, which cannot promote the neural network training effect. Therefore, this embodiment sets the scanning spacing to 1mm. Finally, 5422 groups of asphalt concrete CT original image-label data pairs containing "new image segmentation knowledge" are obtained. The pixels of the 5422 groups of CT original images and corresponding image labels are adjusted to 512 pixels × 512 pixels through pixel expansion or pixel restoration algorithms.

[0058] Step 3: Optimization and development of neural network model.

[0059] refer to Figure 5 , the label essentially provides an image-based description of the aggregate phase contour boundary (concrete features) and position coordinates (abstract features). The semantic segmentation neural network adjusts the size of the receptive field through a continuous convolution and pooling process, and learns the features of the image under different receptive fields. Specifically, the semantic segmentation neural network learns the concrete features of the image from the shallow network (smaller receptive field), learns the abstract features from the deep network (larger receptive field), and finally realizes the organic combination of concrete features and abstract features through feature fusion operations. This unique network architecture learning method is very compatible with the data features before and after asphalt concrete CT image segmentation. However, the unique learning process of the semantic segmentation neural network is inevitably affected by other irrelevant features in the image, such as the original background, noise, grayscale annular distribution characteristics, etc., which have negative effects that increase the difficulty of neural network training convergence and reduce prediction accuracy.

[0060] To this end, this paper proposes an attention-enhanced semantic segmentation neural network (AEUnet), which defines feature weights through an attention mechanism module to highlight important features and suppress irrelevant features. The AEUnet network structure consists of three main parts: encoder, decoder, and feature fusion operation based on channel attention mechanism. Figure 7A simple two-layer AEUnet diagram is shown, where paths a, b, and c represent the encoder, decoder, and feature fusion operations based on the channel attention mechanism, respectively.

[0061] (a) Encoder

[0062] The encoder compresses the input image to increase the receptive field, which is manifested by increasing the image depth (number of image channels) while reducing the image size. Each encoder block consists of several convolutional layers and a maximum pooling layer. Its operation principle is as follows:

[0063] X (l) =ReLU(Conv(X (l-1) ,W (l-1) )) (1)

[0064] X (l+1) =ReLU(Conv(X (l) ,W (l) )) (2)

[0065] Y (l) =MaxPool(X (l+1) ) (3)

[0066] Among them, X (l-1) is the input feature of the encoder; W (l-1) is the convolution kernel of the encoder layer l-1; Y (l) is the output feature of the encoder; Conv() is the convolution operation; ReLU() is the ReLU linear activation operation; MaxPool() is the maximum pooling operation.

[0067] (b) Decoder

[0068] The decoder increases the perception of image details by expanding the image to reduce the receptive field. This is achieved by increasing the image size while reducing the number of image channels. Each decoder consists of a deconvolution layer and several convolution layers. Its operation principle is as follows:

[0069] X (l) =ReLU(TransConv(X (l-1) ,W (l) )) (4)

[0070] X (l+1) =ReLU(Conv(X (l) ,W (l) )) (5)

[0071] Y (l) =ReLU(Conv(X (l+1) ,W (l+1) )) (6)

[0072] Among them, X (l-1) is the input feature of the decoder; W (l-1) is the convolution kernel of the decoder layer l-1; Y (l) is the output feature of the decoder; Conv() is the convolution operation; ReLU() is the ReLU linear activation operation; TransConv() is the deconvolution operation.

[0073] (c) Feature fusion based on channel attention mechanism

[0074] The channel attention mechanism is integrated into the feature fusion stage between the encoder and decoder to achieve effective feature selection. The method for calculating the attention weight is as follows:

[0075]

[0076] Among them, Y enc (l) Y is the output feature of the encoder layer l; dec (l) is the input feature of the decoder layer l; W is the weight matrix; Q is the query vector; K is the key vector; V is the value vector; α ij is the attention weight between positions i and j; d k is the dimension of the key vector; Softmax() is the Softmax function used for normalization. The attention mechanism Attention(Q,K,V) can be calculated by weighted summation of the value vector:

[0077] Attention(Q,K,V)=∑ j α ij V j (11)

[0078] Based on the Concat operation, feature fusion is achieved in the same receptive field:

[0079]

[0080] It has been verified that AEUnet with a 5-level receptive field has the best prediction effect on asphalt concrete CT image segmentation in this embodiment. Its neural network architecture parameters are as follows: Figure 8 As shown, from top to bottom, they are A (A1, A2, A3, A4, A5, A6), B (B1, B2, B3, B4, B5, B6), C (C1, C2, C3, C4, C5, C6, C7), D (D1, D2, D3, D4, D5, D6, D7), and E (E1, E2, E3, E4). The order of forward propagation is:

[0081] Input layer [512×512]: 3 channels by default. If the input is single channel, it is copied to three channels.

[0082] A1 layer [512×512]: obtained by single-channel 1×1 convolution of the input layer, single channel;

[0083] A2 layer [512×512]: obtained by A1 layer through 64-channel 3×3 convolution + padding, 64 channels;

[0084] A3 layer [512×512]: obtained by A2 layer through 64-channel 3×3 convolution + padding, 64 channels;

[0085] B1 layer [256×256]: obtained by channel 2×2 maximum pooling of A3 layer, 64 channels;

[0086] B2 layer [256×256]: obtained by B1 layer through 128-channel 3×3 convolution + padding, 128 channels;

[0087] B3 layer [256×256]: obtained by B2 layer through 128-channel 3×3 convolution + padding, 128 channels;

[0088] C1 layer [128×128]: obtained by channel 2×2 maximum pooling of B3 layer, 128 channels;

[0089] C2 layer [128×128]: obtained by C1 layer through 256-channel 3×3 convolution + padding, 256 channels;

[0090] C3 layer [128×128]: obtained by C2 layer through 256-channel 3×3 convolution + padding, 256 channels;

[0091] C4 layer [128×128]: obtained by C3 layer through 256-channel 3×3 convolution + padding, 256 channels;

[0092] D1 layer [64×64]: obtained by channel 2×2 maximum pooling of C4 layer, 256 channels;

[0093] D2 layer [64×64]: obtained by D1 layer through 512-channel 3×3 convolution + padding, 512 channels;

[0094] D3 layer [64×64]: obtained by D2 layer through 512-channel 3×3 convolution + padding, 512 channels;

[0095] D4 layer [64×64]: obtained by D3 layer through 512-channel 3×3 convolution + padding, 512 channels;

[0096] E1 layer [32×32]: obtained by D4 layer through channel 2×2 maximum pooling, 512 channels;

[0097] E2 layer [32×32]: obtained by E1 layer through 512-channel 3×3 convolution + padding, 512 channels;

[0098] E3 layer [32×32]: obtained by E2 layer through 512-channel 3×3 convolution + padding, 512 channels;

[0099] E4 layer [32×32]: obtained by E3 layer through 512-channel 3×3 convolution + padding, 512 channels;

[0100] D5 layer [64×64]: obtained by fusing the 2×2 deconvolution result of the E4 layer with the features of the D4 layer + attention mechanism, with 1024 channels.

[0101] D6 layer [64×64]: obtained by D5 layer through 512-channel 3×3 convolution + padding, 512 channels;

[0102] D7 layer [64×64]: obtained by D6 layer through 512-channel 3×3 convolution + padding, 512 channels;

[0103] C5 layer [128×128]: obtained by fusion of the 2×2 deconvolution result of the D7 layer and the feature of the E4 layer + attention mechanism, 768 channels.

[0104] C6 layer [128×128]: obtained by C5 layer through 256-channel 3×3 convolution + padding, 256 channels;

[0105] C7 layer [128×128]: obtained by C6 layer through 256-channel 3×3 convolution + padding, 256 channels;

[0106] B4 layer [256×256]: obtained by fusion of the 2×2 deconvolution result of the C7 layer and the features of the B3 layer + attention mechanism, 384 channels.

[0107] B5 layer [256×256]: obtained by B4 layer through 128-channel 3×3 convolution + padding, 128 channels;

[0108] B6 layer [256×256]: obtained by B5 layer through 128-channel 3×3 convolution + padding, 128 channels;

[0109] A4 layer [512×512]: obtained by fusion of the 2×2 deconvolution result of the B6 layer and the features of the A3 layer + attention mechanism, 192 channels.

[0110] A5 layer [256×256]: obtained by A4 layer through 64-channel 3×3 convolution + padding, 64 channels;

[0111] A6 layer [256×256]: obtained by A5 layer through 64-channel 3×3 convolution + padding, 64 channels;

[0112] Output layer [512×512]: obtained by single-channel 3×3 convolution of layer A6, single channel.

[0113] Step 4: Neural network model training.

[0114] The image segmentation offline database established in step 2 is used to train the AEUnet neural network model. The cross entropy loss function is used to judge the learning status during training. After training, the AEUnet-DIP proxy model is obtained.

[0115] In the embodiment, to improve the prediction accuracy, the AEUnet neural network model was trained for 100 rounds using the image segmentation offline database established in step 2, with a total of 5244 sets of image-label data pairs, the learning rate was set to 0.0001, the batch size was set to 32, the training step size was 169, and the total number of training steps was 16900. Fig. 9 ,The cross entropy loss function is used to judge the learning situation, and the loss function during the training process shows a good convergence trend.,The training process is performed by a high-performance Windows 10 computer, whose core configuration is i5-13600KF 3.5GHz CPU, NVIDIAGeForce RTX3080TI 12G GPU, and 32G6000MHz DDR5 RAM.

[0116] Step 5: Image segmentation prediction and verification. The AEUnet-DIP proxy model obtained after training in step 4 can directly realize the fast and accurate mapping of the asphalt concrete CT original image to the segmentation result.

[0117] In order to verify the image segmentation prediction effect of the AEUnet-DIP proxy model, a control experiment was designed with CT image source and shape as independent variables. The verification results are as follows:

[0118] (1) Comparison of segmentation effects of CT images from the same source

[0119] refer to Fig.10, since there is a union of the grayscale value intervals of the asphalt mortar phase and the aggregate phase pixels in the asphalt concrete CT image set obtained in step one, when the grayscale value threshold is overestimated or underestimated, the image segmentation effect is not ideal. The DIP processing flow designed and developed by step two realizes image segmentation very well, which proves the effectiveness and reliability of the image segmentation offline database established by the present invention. The AEUnet-DIP proxy model reproduces the labeling results of the image segmentation offline database very well, proving that the AEUnet architecture designed and developed in step three is reasonable and can effectively "learn" the mapping relationship between the original image and the label.

[0120] refer to Fig.11 , in terms of recovering aggregate contour details, the accuracy of the AEUnet-DIP proxy model exceeds that of the DIP processing flow. For example, randomly distributed slender aggregates are more easily affected by the surrounding asphalt mortar phase pixels during the morphological processing. The essence of the DIP processing flow is to perform morphological processing on the entire image globally or in a predefined area, and it is unable to locally process randomly distributed slender aggregates. Therefore, the DIP processing flow often divides slender aggregates into two or more aggregates distributed along a linear distribution. The essence of the AEUnet-DIP proxy model is to first establish probes for identifying aggregates and asphalt mortar phase pixels through a training process, and then use the probes to distinguish between aggregates and asphalt mortar phase pixels based on the local grayscale distribution pattern in the image. Therefore, the AEUnet-DIP proxy model can accurately process aggregates of any shape.

[0121] (2) Comparison of segmentation effects of CT images of different sources and shapes

[0122] refer to Fig.12 ,Through the results of underestimation and overestimation threshold processing, it can be seen that there are significant differences in the union intervals of pixel grayscale values ​​in the CT image samples from three different sources. Specifically, the range of the union intervals of homologous images (sample I), heterologous images (sample II) and heterologous images (sample III) gradually increases, indicating that the difficulty of image segmentation is gradually increasing.

[0123] The DIP process showed good segmentation results on samples I and II. However, as the pixel grayscale value overlap of the aggregate and asphalt mortar phases increased, the segmentation procedure lost geometric information in the central part of sample III, which was manifested as incomplete and missing aggregates.

[0124] The AEUnet-DIP proxy model performs well in image segmentation results on three different sets of samples. Compared with the CT image segmentation program, the AEUnet-DIP proxy model performs better on sample III and can effectively segment the aggregate phase in the central area of ​​the CT image. This shows that the AEUnet-DIP proxy model has excellent generalization ability on asphalt concrete CT images from different sources and shapes.

[0125] (3) Comparison of segmentation efficiency of images with different resolutions

[0126] refer to Fig.13 , the AEUnet-DIP proxy model can directly process image files of any format, size or color space, avoiding the pre-processing steps such as format adjustment, grayscale conversion and image denoising required by the DIP processing flow, thereby further improving the segmentation efficiency. It has been verified that in the segmentation of AC CT images of different specifications, the AEUnet-DIP proxy model has achieved an exponential improvement in processing efficiency compared to the DIP processing flow.

[0127] Furthermore, the different source CT images in step five include images from different asphalt concrete specimens, different CT scanning equipment and different CT scanning environments.

[0128] Furthermore, the CT images of different shapes in step five include circular, square and images with other arbitrary shapes and size characteristics, and the image resolutions include 1507×914, 1920×1080, 3840×2160 and other size ratio types.

[0129] In addition, when processing CT images of different specifications, the CT image segmentation program needs to be modified according to the image characteristics, which is very difficult for users without image processing and programming skills. However, the powerful versatility of the AEUnet-DIP agent model perfectly solves this problem, and users can easily achieve efficient and accurate segmentation of asphalt concrete CT images of different specifications.

[0130] The above description is only a description of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical contents shall be deemed as equivalent effective embodiments and shall fall within the scope of protection of the technical solution of the present application.

Claims

1. Asphalt concrete CT image segmentation prediction method based on semantic segmentation, characterized in that: The steps include: Step 1: Initial CT image acquisition: Prepare the asphalt concrete CT scanning specimen, determine the scanning direction and use CT scanning technology to obtain its initial CT tomographic image set; Step 2: Establish and expand the offline database for image segmentation; Step 3: Optimization and development of neural network model: According to the data structure characteristics of the image segmentation offline database, a U-net neural network model considering the attention mechanism is constructed based on semantic segmentation, attention mechanism and supervised learning theory, namely the AEUnet neural network model; Step 4: Neural network model training: Use the image segmentation offline database established in step 2 to train the AEUnet neural network model to obtain the AEUnet-DIP proxy model; Step 5: Image segmentation prediction: The AEUnet-DIP proxy model is used to perform aggregate phase segmentation prediction on the asphalt concrete CT images to be tested.

2. The asphalt concrete CT image segmentation prediction method based on semantic segmentation according to claim 1, characterized in that: The CT image in step 1 includes an 8-Bit RGB color image and a grayscale image.

3. The asphalt concrete CT image segmentation prediction method based on semantic segmentation according to claim 1, characterized in that: Each CT image in step 1 contains all pixel information of the complete specimen on the cross section, and redundant pixel positions are filled with "0" value pixels.

4. The asphalt concrete CT image segmentation prediction method based on semantic segmentation according to claim 1, characterized in that: The step 2 comprises: Firstly, according to the grayscale distribution characteristics and inherent defect features of the initial CT tomographic image, a DIP processing flow is designed to segment the different components of the image. Then, the initial CT original image and the corresponding image segmentation result are taken as a set of data pairs, and a high-quality image segmentation offline database is established based on the initial CT tomographic image set; Finally, using the CT scanning computer simulation algorithm, new tomographic images in different directions are extracted from the asphalt concrete CT tomographic image set obtained in step one to expand the number of image segmentation offline databases.

5. The asphalt concrete CT image segmentation prediction method based on semantic segmentation according to claim 4, characterized in that: The image segmentation results in step 2 include aggregate phase segmentation results and asphalt mortar phase segmentation results.

6. The asphalt concrete CT image segmentation prediction method based on semantic segmentation according to claim 4, characterized in that: The DIP processing flow includes pixel filling, image filtering, maximum threshold segmentation, watershed segmentation, and pixel restoration.

7. The asphalt concrete CT image segmentation prediction method based on semantic segmentation according to claim 4, characterized in that: The CT scan computer simulation algorithm includes voxel reconstruction, model coordinate space rotation and voxel model re-slicing.

8. The asphalt concrete CT image segmentation prediction method based on semantic segmentation according to claim 4, characterized in that: The algorithms involved in the DIP processing flow are as follows: the pixel filling algorithm is used to expand the CT image to 3840 pixels × 3840 pixels; the grayscale algorithm is used to uniformly adjust the image grayscale value range to 8-Bit; the local mean filtering algorithm is used to achieve image denoising, and the local filtering kernel function uses the four-connected domain operator [0 0.3 0, 0.3 0.30.3, 0 0.3 0]; the maximum threshold segmentation algorithm based on annular partitioning is used to evenly divide the CT image from the center into 1 circular area and 4 annular areas according to the distance, and the OTSU algorithm is used to achieve maximum threshold segmentation and reorganization; the watershed segmentation algorithm is used to remove the wrongly connected aggregate phase; The pixel restoration algorithm is used to restore the image to 914 pixels × 914 pixels, and finally a binary image including the aggregate phase segmentation results and the asphalt mortar phase segmentation results is obtained; The CT original image and the image segmentation result were cropped to obtain a 512 pixel × 512 pixel CT original image and image segmentation result. The CT original image and the corresponding image segmentation result were used as a set of data pairs. An image segmentation offline database was established based on the initial CT tomographic image set, which contained a total of 750 sets of data pairs. Based on the voxel reconstruction method, the model coordinate space rotation algorithm and the voxel model re-slicing algorithm, the CT scanning process at multiple angles was simulated in the computer memory; the scanning interval was set to 1mm, and finally 5422 sets of data pairs containing "new image segmentation knowledge" were obtained; the pixels of the 5422 sets of CT original images and the corresponding image segmentation results were adjusted to 512 pixels × 512 pixels through pixel expansion or pixel restoration algorithms.

9. The asphalt concrete CT image segmentation prediction method based on semantic segmentation according to claim 1, characterized in that: The AEUnet neural network model: The AEUnet neural network model is based on the basic semantic segmentation network U-net, and realizes the feature fusion optimization under the receptive field of each level of the neural network based on the channel attention mechanism. The feature weights are defined through the attention mechanism module to highlight important features and suppress irrelevant features. The AEUnet neural network structure includes: encoder, decoder and feature fusion operation based on channel attention mechanism, where: The encoder compresses the input image to increase the receptive field, which is specifically manifested by increasing the image depth, i.e., the number of image channels, while reducing the image size; each encoder consists of several convolutional layers and a maximum pooling layer, and its operating principle is as follows: X (l) =ReLU(Conv(X (l-1) ,W (l-1) )) (1) X (l+1) =ReLU(Conv(X (l) ,W (l) )) (2) Y (l) =MaxPool(X (l+1) ) (3) Among them, X (l-1) is the input feature of the encoder; W (l-1) is the convolution kernel of the encoder layer l-1; Y (l) is the output feature of the encoder; Conv() is the convolution operation; ReLU() is the ReLU linear activation operation; MaxPool() is the maximum pooling operation; The decoder enhances the perception of image details by expanding the image to reduce the receptive field; this is achieved by increasing the image size while reducing the number of image channels; each decoder includes a deconvolution layer and several convolution layers, and its operating principle is as follows: X (l) =ReLU(TransConv(X (l-1) ,W (l) )) (4) X (l+1) =ReLU(Conv(X (l) ,W (l) )) (5) Y (l) =ReLU(Conv(X (l+1) ,W (l+1) )) (6) Among them, X (l-1) is the input feature of the decoder; W (l-1) is the convolution kernel of the decoder layer l-1; Y (l) is the output feature of the decoder; Conv() is the convolution operation; ReLU() is the ReLU linear activation operation; TransConv() is the deconvolution operation; The feature fusion operation based on the channel attention mechanism: the channel attention mechanism is integrated into the feature fusion stage between the encoder and the decoder to achieve effective feature selection; the method for calculating the attention weight is as follows: Among them, Y enc (l) Y is the output feature of the encoder layer l; dec (l) is the input feature of the decoder layer l; W is the weight matrix; Q is the query vector; K is the key vector; V is the value vector; α ij is the attention weight between positions i and j; d k is the dimension of the key vector; Softmax() is the Softmax function used for normalization; the attention mechanism Attention(Q,K,V) is calculated by weighted summation of the value vector: Attention(Q,K,V)=∑ j a ij V j (11) Based on the Concat operation, feature fusion is achieved in the same receptive field:

Images