Deep learning-based three-phase structure analysis method for pervious concrete slice scanning image

Abstract

The three-phase structure analysis method of pervious concrete slice scanning image based on deep learning comprises the following steps: based on aggregate particles of different particle sizes, a dyed pervious concrete test piece is prepared; a slice image of the pervious concrete is obtained by cutting and scanning, serving as original image data set for training of a deep learning model; the slice image is labeled based on a deep learning model Mask R-CNN to form an initial image data set of pores, aggregates and cement paste; a data set enhancement algorithm is used to expand the initial image data set to form a standard image data set; and an optimal deep learning model is obtained by training the deep learning model. The slice image of the pervious concrete in a test set is input into the optimal deep learning model, three-phase recognition and segmentation of the pervious concrete are carried out, and evaluation indexes are given. The three-phase structure of aggregates, pores and cement paste in the pervious concrete can be extracted simultaneously, and qualitative and quantitative analysis results of the three-phase structure are obtained, and the precision and efficiency are greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent identification of the three-phase (aggregate, pore and cement paste) structure of pervious concrete, and in particular to a three-phase structure analysis method for slice scanning images of pervious concrete based on deep learning. BACKGROUND

[0002] Pervious concrete is a new type of paving material with multiple environmental benefits such as water permeability, water purification, noise reduction and heat island effect mitigation, and is increasingly widely used. The three-phase structure of aggregate, cement and pore in pervious concrete directly determines its mechanical and water permeability performance, so the efficiency and accuracy of three-phase structure analysis have important practical significance and application value.

[0003] Currently, two common methods for analyzing pores and aggregates in pervious concrete are slice method and computer tomography method, but both methods need to use the threshold setting method in image processing technology to segment the images. The threshold setting is determined by experience, and the entire analysis process relies on manual operation, which is low in efficiency and unstable in accuracy; and only one-phase data, pore or aggregate, can be obtained, and three-phase data cannot be obtained simultaneously. There is no relevant record of simultaneously analyzing three-phase structure for slice scanning images of pervious concrete.

[0004] Therefore, to establish an intelligent method for efficiently and accurately analyzing three-phase structure of pervious concrete simultaneously will provide good technical support for three-phase structure analysis of pervious concrete. SUMMARY

[0005] To solve the problem that the manual threshold method in image processing technology used in the identification and segmentation of pores and aggregates in pervious concrete cannot obtain three-phase analysis results simultaneously and needs manual operation which is low in efficiency and unstable in accuracy, the present application provides a three-phase structure analysis method for slice scanning images of pervious concrete based on deep learning, which greatly improves the accuracy and efficiency compared with the existing manual threshold method; in particular, the three-phase structure (aggregate, pore and cement paste) of slice images of pervious concrete is obtained simultaneously and in real time, and has stronger robustness and generalization ability.

[0006] The technical scheme adopted by the present application is as follows:

[0007] The three-phase structure analysis method for slice scanning images of pervious concrete based on deep learning comprises the following steps:

[0008] Step 1: based on aggregate particles of different sizes, prepare dyed pervious concrete test pieces;

[0009] Step 2: cut and scan to obtain slice images of pervious concrete as original image data sets for training of a deep learning model Mask R-CNN;

[0010] Step 3: Preprocessing the original image dataset;

[0011] Step 4: Based on the image recognition and segmentation technology of the deep learning model Mask R-CNN, the slice image is labeled with pores and aggregates, and the remaining part of the image is cement paste, forming an initial image dataset of pores, aggregates and cement paste;

[0012] Step 5: Expand the initial image dataset to form a standard image dataset using a dataset augmentation algorithm;

[0013] Step 6: Train the deep learning model Mask R-CNN on the standard image dataset to obtain the optimal hyperparameters of the deep learning model Mask R-CNN, and save the optimal deep learning model.

[0014] Step 7: Input the pervious concrete slice image in the test set into the optimal deep learning model to identify and segment the three phases of pervious concrete, output the three-phase overall segmentation image and each single-phase image of aggregate, pore and cement paste, and give the three-phase evaluation index and each single-phase evaluation index.

[0015] The step 1 comprises the following steps:

[0016] Step 1.1: Add ferric oxide powder as a dyeing agent in the mixture containing aggregates, cement and water, and stir uniformly;

[0017] Step 1.2: The prepared pervious concrete test piece is 100mm×100mm×100mm in size, and its preparation process is to weigh and add aggregates, cement and water according to the designed porosity and aggregate-cement ratio, and then stir and vibrate and compact to form, and after demolding, it is placed in a standard curing box for curing.

[0018] The step 2 comprises the following steps:

[0019] Step 2.1: In a plurality of groups of pervious concrete test pieces, randomly select one pervious concrete test piece from each group for cutting. Each pervious concrete test piece is cut equidistantly for 3 times to obtain 6 cutting surfaces;

[0020] Step 2.2: Fill the cutting surface with a certain water content of white viscous powder;

[0021] Step 2.3: Use a color scanner to perform high-precision color scanning on the cutting surface filled with white viscous powder to obtain N pervious concrete slice images.

[0022] In step 3, the images in the original image dataset are uniformly pre-processed, which includes two parts: edge clipping of the image and trapezoidal correction of the image, which is automatically batched by writing a python program.

[0023] Step 4 includes the following steps:

[0024] Step 4.1: Using segmentation technology of large images, one original large image is cropped into 16 local small images, and the local small images form the basic dataset;

[0025] Step 4.2: All images in the basic dataset are labeled in the labeling software Labelme. Before labeling, the images are first enlarged by 2.5 times, and two types are labeled: pores and aggregates. The remaining images after labeling are the cement paste part, and finally the three-phase initial image dataset of pores, aggregates and cement paste is formed.

[0026] In step 5, the data set enhancement techniques include color enhancement, Gaussian noise, horizontal mirror image, image enlargement, image rotation, shear transformation, etc. 20 kinds, which expand the data set by 20 times, totaling 9600 (480*20=9600) as the standard image dataset for model training.

[0027] In step 6, the deep learning model Mask R-CNN has a framework including four components: ① backbone network, ② region candidate network, ③ region of interest alignment network, and ④ functional network.

[0028] Step 6.1: Before model training, the prepared standard image dataset of pervious concrete is divided into training set, validation set and test set according to the proportion 4:4:2.

[0029] Step 6.2: During the training process, the training loss curve on the training set and the validation loss curve on the validation set are drawn and output in real time through the Python program.

[0030] Step 6.3: By observing whether the loss function values of the training set and the validation set respectively reach the convergence state and the accuracy requirement during the model training process, the hyperparameter iteration number Epoch of the deep learning model is set and adjusted; and the optimal deep learning model is obtained.

[0031] In step 6, the training and validation process of the deep learning model Mask R-CNN is as follows:

[0032] The entire training of the deep learning model is divided into two stages:

[0033] First stage: model initialization based on transfer learning:

[0034] First, the network ResNet-101 is selected as the feature extraction backbone network, and the feature experience distribution is learned on the standard COCO dataset through the transfer learning technology to serve as the initialization parameters of the deep learning model Mask R-CNN of the pervious concrete;

[0035] The second stage: training the deep learning model Mask R-CNN on the standard image dataset of the pervious concrete to effectively reduce the calculation amount of the deep learning training;

[0036] The implementation process of the model specific training is divided into two steps: a. freeze the backbone network, and train the random initialization layer which does not use the COCO pre-training weight; b. train the entire deep learning model Mask R-CNN.

[0037] During the model training, the category parameters are set according to the identification category requirements of the target, and in the present application, three categories are included, which are pore, aggregate and background, and the category is set to 3.

[0038] The anchor frame Anchor size is set as 8, 16, 32, 64 and 128 in turn, the alpha weight coefficient is set to 1, the weight attenuation coefficient is set to 0.0005, each anchor frame corresponds to generate three kinds of proportions: 0.5, 1.0 and 2.0, and the momentum factor is set to 0.9.

[0039] The model adopts the nonlinear rectified linear unit ReLU as the activation function, the Adam optimizer is used for optimization, and the Softmax function is used as the loss function, as shown in formula (1):

[0040] The Softmax loss function is as follows:

[0041]

[0042] In formula (1), K represents the total number of categories, i represents a certain category, Z is a set of results output by the full connection layer in the neural network, and i is the subscript number in the set. The larger the loss function, the smaller the classification probability of the classifier on the real label, and the poorer the performance; on the contrary, the smaller the loss function, the larger the classification probability of the classifier on the real label, and the better the performance.

[0043] In step 7, the six indexes of the optimal deep learning model are as follows:

[0044] Absolute error AE; mean absolute percentage error MAPE; intersection over union IoU; mean intersection over union mIoU; F1 score; and differential box counting dimension DBC.

[0045] The AE is the difference between the actual number of pores or aggregates in the image and the number of aggregates or pores identified by the model. The smaller the AE, the better the performance of the model, as shown in formula (2):

[0046]

[0047] wherein: y i is the true value of aggregate or pore number, the model-identified number of aggregate or pore number.

[0048] MAPE is the ratio of the absolute value of all sample errors to the actual value, as shown in equation (3):

[0049]

[0050] wherein: y i is the true value of aggregate or pore number, the model-identified number of aggregate or pore number; n represents the number of samples.

[0051] The intersection over union IoU is a pixel-level evaluation index, and the ratio of the pixel area corresponding to the intersection of the three-phase image identified by the deep learning model and the true value image to the pixel area corresponding to the union, the closer to 1 the higher the accuracy, that is, the intersection of the predicted region and the actual region divided by the union of the predicted region and the actual region, simply referred to as the intersection over union; the proportion of the intersection and the union between two sets is calculated, and in image segmentation, it is the proportion of the true value and the predicted value two sets, as shown in equation (4):

[0052]

[0053] wherein: A is the pixel area corresponding to the intersection of the three-phase image identified by the deep learning model and the true value image; B is the pixel area corresponding to the union of the three-phase image identified by the deep learning model and the true value image.

[0054] mIoU is the average intersection over union obtained by first calculating the intersection over union within each class and then calculating the average value of all classes, as shown in equation (5):

[0055]

[0056] wherein: p ij represents the number of true values for i and predicted for j; k+1 is the number of classes (including empty class); p ii is the true number; p ij , p ji respectively represent the number of false positives and the number of false negatives.

[0057] The F1 score is used to evaluate the overall performance of the model:

[0058] The F1 score is defined as the harmonic mean of precision and recall, and is calculated from precision (P) and recall (R). Precision is the ratio of positive samples in all samples labeled as target in the test set; recall is the proportion of correctly identified target positive samples in all target positive samples in the test set.

[0059] The calculation methods of precision and recall are shown in equations (6) and (7):

[0060]

[0061]

[0062] Where TP refers to actual pores or aggregates, and the model identification is also pores or aggregates; FP refers to actual pores or aggregates, and the model identification is pores or aggregates; FN refers to actual pores or aggregates, and the model identification is not pores or aggregates.

[0063] The calculation method of F1 score is shown in equation (8):

[0064]

[0065] The differential box-counting (DBC) is a measure of the roughness of the surface texture of an image, with high precision and applicability, and can meet the requirements of calculation efficiency and dynamic characteristics. The present application proposes to use differential box-counting to describe the complexity and irregularity of cement paste.

[0066]

[0067] In the formula: D - fractal dimension;

[0068] F - any non-empty bounded subset R n ;

[0069] N δ - the minimum number of squares with the same size as delta covering F.

[0070] The present application is a three-phase structure analysis method for permeable concrete slice scanning images based on deep learning, and the technical effects are as follows:

[0071] 1) Compared with the traditional threshold method currently used, which needs to set a threshold (manually according to experience) for pore segmentation or aggregate segmentation, the present application can only identify two-phase structure and has low efficiency and unstable precision. The three-phase intelligent identification method proposed by the present application can simultaneously extract the aggregate, pore and cement paste three-phase structure in permeable concrete, and obtain qualitative and quantitative analysis results of the three-phase structure, and greatly improve the precision and efficiency.

[0072] 2) The invention also verifies that the deep learning model has strong robustness and generalization ability on other common materials. BRIEF DESCRIPTION OF DRAWINGS

[0073] Figure 1(a) is a flow chart of the three-phase structure analysis of pervious concrete based on deep learning.

[0074] The input pervious concrete slice image is intelligently recognized and segmented by the deep learning model, and the three-phase segmentation result image and quantitative analysis index are output.

[0075] Figure 1(b) is a schematic diagram of three-phase structure identification and segmentation of pervious concrete.

[0076] Figure 2 It is the logical structure diagram of the deep learning model MaskR-CNN framework.

[0077] Figure 3 It is the particle diagram of three different particle sizes of diabase aggregate.

[0078] Figure 4 It is a flow chart of preparing colored pervious concrete test pieces.

[0079] Figure 5 It is a flow chart of cutting test pieces.

[0080] Figure 6 It is the original scanning image of the test sample slice (3 particle sizes).

[0081] Figure 7 It is the scanning image after filling the slice pores with white powder (3 particle sizes).

[0082] Figure 8 It is a flow chart of large image segmentation and labeling.

[0083] Figure 9 It is a segmentation diagram of large images (1 original image is segmented into 16 local images).

[0084] Figure 10 It is a data set labeling diagram.

[0085] Figure 11 It is an iteration curve diagram of model training loss and validation loss.

[0086] Figure 12 It is a three-phase structure segmentation result image based on a deep learning model.

[0087] Figure 13 It is a pore segmentation result image of threshold method and deep learning method.

[0088] Figure 14Pore recognition result of three-phase structure of pervious concrete.

[0089] Figure 15 Aggregate recognition result of three-phase structure of pervious concrete.

[0090] Figure 16 Cement paste recognition result of three-phase structure of pervious concrete

[0091] Figure 17 Flow chart of the present application. DETAILED DESCRIPTION

[0092] The three-phase structure analysis method of the pervious concrete slice scanning image based on deep learning includes the following steps:

[0093] First step: preparation before sample preparation:

[0094] According to the design, prepare raw aggregates of different particle sizes and cement. In order to clearly show the morphology and distribution of the cement paste in the molded test piece, the present application proposes to add ferric oxide powder (reddish) as a dyeing agent to the sample preparation of the pervious concrete, and the required weight is 6% of the mass of the cement.

[0095] Second step: preparation of pervious concrete test sample and curing molding:

[0096] The prepared pervious concrete test sample is a standard cube with a size of 100mmx100mmx100mm. The preparation process weighs the aggregate, cement, water and ferric oxide powder according to the designed porosity, aggregate-cement ratio, and adds them to the mixer for stirring, and then puts them into the standard mold for vibration compaction molding. After 24 hours, the mold is removed and placed in a standard curing box for curing for 28 days.

[0097] Third step: cutting the pervious concrete test sample:

[0098] Randomly select one test piece from each group of prepared and molded pervious concrete test pieces, and use a cutting machine to cut each test piece from the top to the bottom, and cut it 3 times at equal distances. Each test piece obtains 6 cutting surfaces.

[0099] Fourth step: fill the cutting surface with a certain water content white viscous powder:

[0100] In order to clearly show the pore distribution and pore structure in the slice image, the present application proposes to fill the pores in the section of the test piece with a certain water content white viscous powder, and clean other areas of the section: aggregate area and cement paste area, so as to keep them clean. Wait for a few minutes, and when the surface moisture is basically volatilized but not completely dried, use a high-precision color scanner for high-definition scanning. Set the scanner scanning parameters to 1200 dpi, and the image resolution to 4800*4800 pixel. One section image is about 5MB in size, which is a high-resolution high-precision large image.

[0101] Step 5: Data set preprocessing:

[0102] When all the slices of the required pervious concrete are scanned, the original image data set is formed. The images in the original data set are uniformly preprocessed, and the preprocessing includes two parts: image edge clipping and image trapezoidal correction, which are automatically batched by writing a python program.

[0103] Step 6: Data set annotation:

[0104] In order to improve the model training efficiency and recognition and segmentation accuracy, the present application proposes to use large image segmentation technology, i.e. cutting one original large image into 16 local small images, and the local small images constitute the basic data set, aiming at the high resolution and high precision characteristics of the scanned images.

[0105] In order to improve the accuracy of deep learning model training and recognition, when annotating all images in the basic data set in Labelme software, first magnify each image by 2.5 times in Labelme software, then annotate them into two categories: pores and aggregates (pores are 1 class and aggregates are 1 class), and the remaining background part of the image is cement paste after annotation, i.e. the final three-phase image true value of pores, aggregates and cement paste, and is saved as the original image size.

[0106] Step 7: Expand the basic data set to form a standard image data set by using data set augmentation technology:

[0107] In order to solve the overfitting phenomenon in deep learning network training, and improve the generalization and robustness of the network. Expand the data set by using data set augmentation technology on the basic data set, increase the diversity of the data set, which can effectively improve the model precision and generalization, and avoid underfitting and overfitting in model training.

[0108] In the present application, the data set enhancement techniques adopted include color enhancement, adding noise, horizontal or vertical mirroring, image rotation, image flipping, image scaling, shear transformation, brightness adjustment, contrast adjustment, etc. 20 kinds, expand the basic data set 20 times, as the standard data set of model training, effectively expand the scale of data set and the diversity of data set, thereby enhancing the robustness and generalization of deep learning model.

[0109] Eighth step: build a software and hardware experimental platform for deep learning model training

[0110] By deploying Mask R-CNN running environment on Ubuntu server, the software and hardware environment is as follows. Graphics card: NVIDIA T4 GPU, operating system: Ubuntu 18.04, graphics card driver version: NVIDIA-Linux-x86_64-450.80.02, Cuda version: cuda_9.0.176_384.81_linux, Cudnn version: cudnn-9.0-linux-x64-v7.0.5.15, Tensorflow version: TensorFlow 1.14.0+Keras 2.2.5.

[0111] Ninth step: train, verify and test the deep learning model, finally get the optimal model and the optimal hyperparameters of the model, and save the optimal model for subsequent use.

[0112] The deep learning model Mask R-CNN is applied to the intelligent identification and analysis of the three-phase structure of the pervious concrete slice scanning image, which has complete model training, verification and testing, and finally gets the optimal model of the pervious concrete slice scanning image for subsequent use. The whole process first uses the transfer learning technology to set the initial parameters of model training, then gets the optimal hyperparameters and optimal model of the model through training, and saves the optimal model. The deep learning model Mask R-CNN has a framework including four components, which are: (1) backbone network (2) region candidate network (3) region of interest alignment network (4) functional network.

[0113] The standard data set of the prepared pervious concrete is divided into a training set, a validation set and a test set according to a proportion of 4:4:2 before model training. During the training process, the training loss curve on the training set and the validation loss curve on the validation set are drawn and output in real time through a Python program. By observing in real time whether the loss function values corresponding to the training set and the validation set in the model training process reach the convergence state and the accuracy requirement, the number of iterations of the hyperparameters of the deep learning model is set and adjusted, and finally the optimal model is obtained, and the optimal model is stored on the device. The input of the model is the standardized image of the pervious concrete slice image, and the optimal model is used for three-phase intelligent recognition and segmentation, and the three-phase overall segmentation image (including aggregate, pore and cement paste) and each single-phase image are automatically obtained and output, and the overall three-phase evaluation index and each single-phase evaluation index and the quantitative analysis result are given.

[0114] The specific model training and verification process is as follows:

[0115] The Mask R-CNN model is trained under the Tensorflow deep learning framework, and GPU is used for acceleration. First, the network ResNet-101 is selected as the feature extraction backbone network, the pre-trained weight on the COCO data set is used to initialize the network parameters through the transfer learning technology, and then the training and verification are carried out on the training set and the validation set of the data set.

[0116] The whole training of the model is divided into two stages:

[0117] The first stage is model initialization based on transfer learning:

[0118] The image recognition model based on deep learning is composed of a variety of complex neural networks. When the model analyzes the class features of each pixel point in the image, it will continuously compare and analyze the neural network output results of a specific region in the image with the labeled feature results, and then adjust the weight distribution of each neural network layer through back propagation to realize the feature description of the image region. The calculation amount of this process is large, and the weight distribution of the initially established neural network model is random, so the training process of the neural network often needs to consume a lot of time, and the final result may not meet the requirements of image recognition, so the transfer learning technology is used to initialize the model in the present application, which effectively reduces the workload of model training. The specific implementation is to learn the feature experience distribution of the standard COCO data set through the transfer learning technology to obtain the deep learning model initialization parameters of the pervious concrete.

[0119] The second stage is to train the model on the standard image data set of the pervious concrete, effectively reduce the calculation amount of deep learning training, and effectively improve the learning efficiency of the model.

[0120] The implementation process of the model specific training includes two steps: 1. freezing the backbone network, and training the random initialization layer which does not use the COCO pre-training weight; and 2. training the entire Mask R-CNN model.

[0121] During the model training, the category parameters are set according to the recognition category requirements of the target, and in the present application, three categories are included, namely, pore, aggregate and background, and the category setting is 3. The anchor frame Anchor size is set as 8, 16, 32, 64 and 128 in turn. The alpha weight coefficient is set as 1, and the weight attenuation coefficient is set as 0.0005. Each anchor frame corresponds to the generation of three ratios: 0.5, 1.0 and 2.0. The momentum factor is set as 0.9. The model adopts the nonlinear correction linear unit ReLU as the activation function, the Adam optimizer is used for optimization, and the Softmax function is used as the loss function, as shown in formula (1). The initial learning rate of the network weight parameter is set as 0.005, a total of 500 iterations Epoch are set for training, the image data in the training set are trained for 1 time operation, and at the same time, the verification set is verified for 1 time operation, which is called 1 Epoch, and the learning rate is reduced to 0.0001 when the 20th Epoch is set. During the entire training process, the training loss curve on the training set and the verification loss curve on the verification set are output in real time by the Python program.

[0122] In the Mask R-CNN, the Softmax loss function is adopted, and formula (1) is as follows:

[0123]

[0124] In formula (1), K represents the total number of categories, i represents a certain category, Z is a group of results output by the full connection layer in the neural network, and i is the subscript number in the group. The larger the loss function, the smaller the classification probability of the classifier on the real label, and the poorer the performance; on the contrary, the smaller the loss function, the larger the classification probability of the classifier on the real label, and the better the performance.

[0125] Step 10: evaluating the deep learning model:

[0126] The slice images in the test set are input into the optimal deep learning model for three-phase intelligent recognition and segmentation, and the three-phase overall segmentation images and single-phase images of aggregate, pore and cement paste are output, and the overall three-phase evaluation index and single-phase evaluation index are given. The evaluation indexes used in the present application include absolute error AE, mean absolute percentage error MAPE, intersection over union IoU, mean intersection over union mIoU, F1 score (including precision P and recall R), and differential box dimension DBC.

[0127] The specific calculation methods of the evaluation indexes are as follows:

[0128] 1) Absolute Error (AE)

[0129] AE is the difference between the true value and the model identification result. In the present application, AE is the difference between the actual number of pores or aggregates in the image and the number of aggregates or pores identified by the model. The smaller the AE, the better the performance of the model, as shown in equation (2).

[0130]

[0131] where y i is the true value of the aggregate or pore number, the model-identified number of aggregate or pore number.

[0132] 2) Mean Absolute Percent Error (MAPE)

[0133] MAPE is the ratio of the absolute value of all sample errors to the actual value, as shown in equation (3).

[0134]

[0135] where y i is the true value of the aggregate or pore number, the model-identified number of aggregate or pore number.

[0136] 3) Intersection over Union (IoU)

[0137] The intersection over union (IoU) is a pixel-level evaluation index, which is the ratio of the pixel area corresponding to the intersection of the three-phase image identified by the deep learning model and the true value image to the pixel area corresponding to the union. The closer to 1, the higher the accuracy, i.e. the intersection of the predicted region and the actual region divided by the union of the predicted region and the actual region, simply referred to as the intersection over union. Calculate the proportion of the intersection and union between two sets. In image segmentation, it is the proportion of the true value and the predicted value of the two sets, as shown in equation (4).

[0138]

[0139] where A is the pixel area corresponding to the intersection of the three-phase image identified by the deep learning model and the true value image, and B is the pixel area corresponding to the union of the three-phase image identified by the deep learning model and the true value image.

[0140] 4) Mean Intersection over Union (mIoU)

[0141] mIoU is the average intersection over union, which is calculated by first calculating the intersection over union for each class and then calculating the average of all classes, as shown in equation (5).

[0142]

[0143] where k is the number of classes, and p is the pixel value.

[0144] 5) F1 score: F1 score is used to evaluate the overall performance of the model.

[0145] F1 score is defined as the harmonic mean of precision and recall, which is calculated from precision and recall. Precision is the ratio of positive samples in all samples labeled as target in the test set; recall is the proportion of correctly identified target samples in all target samples in the test set.

[0146] The calculation method of precision and recall is shown in equations (6) and (7):

[0147]

[0148]

[0149] where TP refers to actual pores or aggregates, and the model also identifies pores or aggregates;

[0150] FP refers to actual pores or aggregates, and the model identifies pores or aggregates;

[0151] FN refers to actual pores or aggregates, and the model does not identify pores or aggregates.

[0152] The calculation method of F1 score is shown in equation (8):

[0153]

[0154] 6) Differential Box-counting (DBC)

[0155] Differential Box-counting (DBC) is a measure of the roughness of the surface texture of the image, with high precision and applicability, and can meet the requirements of calculation efficiency and dynamic characteristics. This method effectively improves the calculation efficiency and precision of the fractal dimension calculation method. The present application proposes to use differential box-counting to describe the complexity and irregularity of cement paste.

[0156] Eleventh step: practical application

[0157] The slice image of the pervious concrete sample is randomly selected and input into the deep learning optimal model to perform intelligent identification and segmentation of three phases, and the overall identification image of three-phase segmentation and each phase image and quantitative analysis results are output.

[0158] Embodiments:

[0159] The application provides an intelligent identification and segmentation method for three-phase structures of a pervious concrete slice image based on deep learning, Figure 1 (a) is a three-phase structure analysis flowchart of the pervious concrete based on deep learning, and Figure 1 (b) is a three-phase structure identification and segmentation schematic diagram of the pervious concrete, Figure 2 The deep learning framework MaskR-CNN logical structure diagram used in the application.

[0160] The specific steps of the embodiment include:

[0161] S1: Before sample preparation, 6% of the mass of the cement in the mixture containing aggregate, cement and water is added with ferric oxide powder (red) and stirred uniformly.

[0162] The original aggregate used in the embodiment is common diabase gravel, and three different particle sizes are selected, and the particle sizes are 2.36-4.75 mm, 4.7-9.5 mm and 9.5-15 mm, as shown in the figure. Figure 3 In order to clearly show the shape and distribution of the cement paste in the test piece, ferric oxide powder is added as a dye for dyeing.

[0163] S2: Prepare the pervious concrete sample and cure the molding.

[0164] The prepared pervious concrete sample is 100mm*100mm*100mm in size, and the preparation process is to add aggregate, cement and water according to the designed porosity and aggregate ratio, and to stir and vibrate and compact the molding, and after 24 hours, the sample is removed from the mold and placed in a standard curing box for curing for 28 days, and the preparation process is as shown in the figure. Figure 4

[0165] S3: Cutting of the pervious concrete sample.

[0166] In the 18 groups (6 samples in each group) of samples, one sample is randomly selected from each group for cutting. The cutting machine is used to cut each test block at equal distances for 3 times to obtain 6 cutting surfaces, and the cutting process is as shown in the figure. Figure 5 The color scanner is used to scan the cutting surface, the scanner scanning parameter is set to 1200dpi, the image resolution is about 4800*4800, and the size of one cutting surface image is about 5MB, and the scanning image of the cutting surface is as shown in the figure. Figure 6

[0167] S4: The pore of the cutting surface is filled with white viscous powder with a certain water content. ​​

[0168] In order to clearly show the pore distribution and structure, 30 sections of specimens with different porosities and particle sizes were randomly selected, the sections were filled with flour, and the cement paste and aggregate parts of the sections were kept clean.

[0169] S5: Obtain the scanning image of the specimen section to form an initial data set.

[0170] For the section filled with white sticky powder, wait for several minutes, and when the surface moisture is basically volatilized but not completely dried, use a color scanner to complete the high-definition scanning. The scanning parameters of the color scanner are set to 1200 dpi, and the image resolution is about 4800*4800, and the size of one section image is about 5MB. After all the required sections are scanned, 30 original data set images are formed, as shown in Figure 7 .

[0171] S6: Data set preprocessing.

[0172] The images in the original data set are uniformly preprocessed, and the preprocessing includes image edge trimming and image trapezoidal correction, which is automatically realized by writing a python program for batch processing.

[0173] S7: Data set annotation of two classes (pore and aggregate) to form three-phase true values.

[0174] In order to improve the model recognition and segmentation accuracy, the large image segmentation technology is introduced for the high-resolution large images obtained by scanning, and one original large image is cropped into 16 local small images, so that 480 (30*16=480) local images are obtained to form the basic data set. All images in the basic data set are annotated in the labelme software. In order to improve the accuracy of deep learning model training and recognition, when annotating the same image, first magnify the image by 2.5 times, then divide it into two classes: one class for pores and one class for aggregates, and then annotate them, respectively named h1, h2, h3, … and s1, s2, s3, …), and the remaining background part in the image is the cement paste, i.e. the true value of the three-phase image of pores, aggregates and cement paste is finally formed. The large image segmentation and annotation flow chart is shown in Figure 8 , the segmentation result is shown in Figure 9 , and the true value formed by annotation is shown in Figure 10 .

[0175] S8: Use data set augmentation technology to expand the basic data set to form a standard data set.

[0176] The dataset expansion is carried out on the basis dataset by using the dataset expansion technology, which is beneficial to improve the precision and generalization of the model, and meanwhile avoids the underfitting and overfitting of the model. In the application, the dataset expansion technology includes color enhancement, Gaussian noise, horizontal mirror image, image enlargement, image rotation, shear transformation and the like, and the dataset is expanded by 20 times, and a total of 9600 (480*20=9600) is used as the standard dataset for model training.

[0177] S9: training, verification and testing of the deep learning model are carried out, and the optimal deep learning model and the optimal hyperparameter are obtained, and the optimal model is saved.

[0178] The 9600 standard datasets are divided into a training set, a verification set and a test set according to a ratio of 4:4:2. The model is trained on the training set, the model is verified on the verification set, and the model is applied on the test set. During the whole training process, the training loss curve on the training set and the verification loss curve on the verification set are drawn in real time by a Python program. In this embodiment, the loss iteration graph tends to be flat after the 40th Epoch, and the curve tends to converge when the iteration reaches the 500th Epoch, as shown in Figure 11 .

[0179] S10: deep learning model evaluation and application.

[0180] Three images with different aggregate particle sizes are randomly selected from the test set of the standard dataset, input into the optimal model, and the three-phase intelligent recognition and segmentation are carried out, and the three-phase overall image and the evaluation index value are output, and the evaluation index value of each single phase: aggregate, pore and cement paste: image and each phase evaluation index value.

[0181] (1) Model overall three-phase analysis:

[0182] Three images are randomly selected from the test set of the dataset, input into the optimal deep learning model, and the three-phase recognition and segmentation of aggregate, pore and cement paste are carried out. The overall segmentation result of the three-phase structure based on the deep learning model is shown in Figure 12 , and the overall quantitative evaluation index data is shown in Table 2.

[0183] Table 1: Overall evaluation index and data of three-phase segmentation of deep learning model

[0184]

[0185] Table 1_1: Pore recognition index and data of three-phase structure of pervious concrete

[0186] Pore identification Number of true values Number of identifications AE MAPE Precision Recall F1 IoU Random image 65 64 1 1.54％ 0.998 0.956 0.981 0.985

[0187] Table 1_2: Aggregate recognition index and data of three-phase structure of pervious concrete

[0188]

[0189]

[0190] Table 1_3 Pervious concrete three-phase structure Cement paste identification index and data

[0191]

[0192] Table 2 Evaluation index of deep learning model and three-phase overall evaluation index

[0193]

[0194] (2) Model pore evaluation, and compared with traditional threshold method:

[0195] In order to verify the accuracy and efficiency of the three-phase structure intelligent identification and segmentation based on the deep learning model proposed in the application, the common threshold method (which can only segment two phases) is compared and analyzed, and the accuracy and efficiency of the model in application are evaluated.

[0196] Threshold method is only applicable to binary image, and the original slice color image needs to be converted into binary image (white part represents pore part, and black part represents non-pore part) in image processing software. The identification process is as follows: first, improve the contrast of the image in the image processing software, so that the pore structure is clear and easy to identify; Then manually set the threshold and color tolerance value for segmentation in the software. This method needs to adjust the setting value according to experience to make it as close to the true value as possible, which is low in efficiency and unstable in accuracy. In three-phase structure analysis, aggregate and cement paste cannot be effectively segmented by threshold method, so only pore segmentation accuracy can be compared and analyzed in comparative analysis, and the segmentation result is shown in Figure 13 , and the evaluation index is shown in Table 3.

[0197] Table 3 Overall evaluation index IoU of threshold method and deep learning method

[0198] Original image Pore (true value) Pore (threshold method) Pore (intelligent segmentation method of deep learning model) Evaluation index IoU 1 0.51 0.96

[0199] The results show that the IoU of the threshold method for segmenting the pores is only 0.51, the segmentation accuracy is low, and there are obvious over-segmentation or under-segmentation phenomena in pore segmentation. The segmentation effect of the deep learning model intelligent segmentation method is good, the accuracy is high, and the pore IoU reaches 0.96, close to the true value.

[0200] (3) Three-phase structure single-phase analysis:

[0201] Three images with different aggregate particle sizes are selected in the test set of the data set. The deep learning model is used to identify, segment and statistically analyze each phase in the three-phase structure of the image.

[0202] ①Pore recognition image and quantitative analysis:

[0203] The deep learning model is used to identify and segment the pores in the image, and the segmentation result image of the pores is obtained, as shown in Figure 14 The analysis indicators are: pore number, AE, MAPE, precision, recall, F1, and IoU, for quantitative analysis, as shown in Table 4.

[0204] Table 4 Pervious concrete three-phase structure_pore recognition index

[0205]

[0206] ②Aggregate recognition image and quantitative analysis:

[0207] The deep learning model is used to identify and segment the aggregate in the image, and the segmentation result image of the aggregate is obtained, as shown in Figure 15 The analysis indicators are: particle number, AE, MAPE, precision, recall, F1, and IoU, for quantitative analysis, as shown in Table 5.

[0208] Table 5 Pervious concrete three-phase structure_aggregate recognition index

[0209]

[0210] ③Cement paste recognition image and quantitative analysis:

[0211] The trained deep learning model is used to identify and segment the cement paste in the image, and the segmentation result image of the cement paste is obtained, as shown in Figure 16 The analysis indicators are: precision, recall, F1, IoU, mix ratio, and difference box dimension, for quantitative statistical analysis, as shown in Table 6.

[0212] Table 6 Pervious concrete three-phase structure_cement paste recognition index

[0213]

[0214] S11: Practical application:

[0215] In practical application, any selected slice image of pervious concrete sample is input into the optimal deep learning model for three-phase intelligent recognition and segmentation, and the result image of three-phase recognition and segmentation and the result of quantitative analysis indicators are output in real time.

[0216] The embodiment 1 of the present application provides a new method for analyzing and researching the three-phase structure of aggregate, cement and pores, and is characterized in that, compared with the traditional manual threshold determination method, the precision and efficiency are greatly improved, the key is that the three-phase structure (aggregate, pore and cement paste) of the pervious concrete section image is obtained at the same time and in real time, and has stronger robustness and generalization ability.

Claims

1. A method for analyzing the three-phase structure of a pervious concrete slice scanning image based on deep learning, characterized by The method comprises the following steps: Step 1: preparing dyed pervious concrete test pieces based on aggregate particles of different sizes; Step 2: cutting and scanning to obtain pervious concrete slice images as original image data sets for training of a deep learning model; Step 3: preprocessing the original image data sets; Step 4: based on image recognition and segmentation technology of the deep learning model, labeling the slice images for pores and aggregate, and the remaining part of the image being cement paste, to form an initial image data set of pores, aggregate and cement paste; Step 5: expanding the initial image data set to form a standard image data set by using a data set expansion algorithm; Step 6: training the deep learning model on the standard image data set to obtain optimal hyperparameters of the deep learning model, and saving the optimal deep learning model; Step 7: inputting the pervious concrete slice images in the test set into the optimal deep learning model to recognize and segment the three phases of the pervious concrete, outputting the three-phase overall segmentation images and single-phase images of aggregate, pores and cement paste, and simultaneously giving three-phase evaluation indexes and single-phase evaluation indexes; The step 4 comprises the following steps: Step 4.1: using a large image segmentation technology to cut one original large image into 16 local small images to form a basic data set; Step 4.2: labeling all images in the basic data set in a labeling software Labelme; before labeling, the images are first enlarged by 2.5 times, and two types are labeled: pores as one type and aggregate as one type; the remaining images after labeling are the cement paste part, and finally the three-phase initial image data set of pores, aggregate and cement paste is formed; In the step 6, the training and verification process of the deep learning model is as follows: The whole training of the deep learning model is divided into two stages: First stage: model initialization based on transfer learning: First, the network ResNet-101 is selected as the feature extraction backbone network, and the feature experience distribution is learned on the standard COCO data set by the transfer learning technology to serve as the initialization parameters of the deep learning model Mask R-CNN of the pervious concrete; Second stage: training the deep learning model Mask R-CNN on the standard image data set of the pervious concrete to effectively reduce the calculation amount of the deep learning training; The specific training process of the model is divided into two steps: a. freezing the backbone network to train the randomly initialized layers not using the COCO pre-training weight; b. training the whole deep learning model Mask R-CNN; When training the model, the class parameters are set according to the recognition class requirements of the target, the class parameters contain three classes, namely pores, aggregate and background, and the class setting is 3; The model uses the nonlinear rectified linear unit ReLU as the activation function, the Adam optimizer for optimization, and the Softmax function as the loss function; In the step 7, the six indexes for evaluating the optimal deep learning model are as follows: The step 1 comprises the following steps: Absolute error AE; mean absolute percentage error MAPE; intersection over union ; mean intersection over union ; F1 score; differential box counting dimension DBC. 2.The method of claim 1, wherein the method is characterized by: Step 1.1: adding ferric oxide powder as a dyeing agent in the mixture containing aggregate, cement and water, and stirring uniformly; ​ Step 1.2: The prepared pervious concrete test pieces are all 100mm x 100mm x 100mm in size, and are prepared according to the designed porosity, aggregate-cement ratio, and aggregate, cement and water are weighed and added in sequence, and are stirred and vibrated and compacted to form, and after demolding, are placed in a standard curing box for curing. 3.The method of claim 1, wherein the method further comprises: The step 2 comprises the following steps: Step 2.1: In a plurality of groups of pervious concrete test pieces, one pervious concrete test piece is randomly selected from each group for cutting; each pervious concrete test piece is cut at equal intervals 3 times to obtain 6 cutting surfaces; Step 2.2: The cutting surfaces are filled with white viscous powder of a certain water content; Step 2.3: The cutting surfaces filled with white viscous powder are high-precision color scanned using a color scanner to obtain N pervious concrete slice images. 4.The method of claim 1, wherein the method further comprises: In the step 3, the images in the original image dataset are uniformly preprocessed, and the preprocessing comprises two parts: edge clipping of the images and trapezoidal correction of the images, and is automatically batch-processed by writing a python program. 5.The method of claim 1, wherein the method further comprises: In the step 6, the deep learning model Mask R-CNN has a framework comprising 4 components: ① backbone network, ② region candidate network, ③ region of interest alignment network, and ④ functional network; Step 6.1: Before model training, the prepared standard image dataset of pervious concrete is divided into a training set, a validation set and a test set according to a ratio of 4:4:2; Step 6.2: During the training process, the training loss curve on the training set and the validation loss curve on the validation set are drawn and output in real time by a Python program; Step 6.3: The number of iterations of the hyperparameters of the deep learning model is set and adjusted by observing in real time whether the loss function values of the training set and the validation set respectively reach a convergence state and accuracy requirements during the model training process, and an optimal deep learning model is finally obtained. 6.The method of claim 1, wherein the method further comprises: The absolute error in step 7 AE is the difference between the actual number of pores or aggregates in the image and the number of aggregates or pores identified by the model; AE The smaller the absolute error, the better the performance of the model, as shown in equation (2): （2）； wherein: is the true value of the number of aggregates or pores, is the model identified number of aggregates or pores; mean absolute percentage error MAPE is the ratio of the absolute value of the error of all samples to the actual value, as shown in equation (3): （3）； wherein, is the true value of the number of aggregates or pores, is the model identified number of aggregates or pores; n denotes the number of samples; intersection over union is an evaluation index at the pixel level, the ratio of the pixel area corresponding to the intersection of the three-phase image identified by the deep learning model and the true value image to the pixel area corresponding to the union, the closer to 1, the higher the accuracy, that is, the intersection of the predicted area and the actual area divided by the union of the predicted area and the actual area, simply referred to as intersection over union; the proportion of the intersection and the union between two sets, in image segmentation, it is the proportion of the true value and the predicted value two sets, as shown in formula (4): （4）； wherein, is the pixel area corresponding to the intersection of the three-phase image identified by the deep learning model and the true value image; is the pixel area corresponding to the union of the three-phase image identified by the deep learning model and the true value image. average intersection over union The average intersection over union is obtained by first calculating the intersection over union for each class and then calculating the average over all classes, as shown in equation (5): （5）； wherein, k +1 is the number of classes; p ii is the true number; p ij , p ji then respectively represent the number of false positives and the number of false negatives; The model was evaluated using F 1 The overall performance of the model was evaluated using F 1The F-score is defined as the harmonic mean of precision and recall, calculated from precision (Precision) and recall Recall ; Precision is the ratio of positive samples among all samples labeled as target in the test set; Recall is the proportion of correctly identified target positive samples among all target positive samples in the test set. Precision and Recall are calculated as shown in equations (6) and (7): （6）； （7）； wherein, TP actual pore or aggregate, model identifies as pore or aggregate; FP actual not a pore or aggregate, model identifies as pore or aggregate; FN actual pore or aggregate, model identifies as not a pore or aggregate; F 1The score is calculated as shown in equation (8): (8)； Box dimension using differential counting DBC To describe the complexity and irregularity of cement slurry; （9）； In the formula: D is the fractal dimension; F - any non-empty bounded subset R n ; N 𝛿 - the minimum number of squares of size equal to d and covering F.

Images