An Automatic Identification Method for ECC Surface Cracks Based on Deep Learning

Abstract

The present invention provides an automatic recognition method for ECC surface cracks based on deep learning, including the following steps: preprocessing the ECC surface image; designing a loss function based on a dual strategy according to the ECC surface characteristics; building an automatic recognition model for ECC surface cracks based on the improved semantic segmentation network FCN; and using the automatic recognition model for ECC surface cracks to perform pixel-level classification on the ECC surface image to be measured. In the loss function of the automatic recognition model for ECC surface cracks constructed by the present invention, a customized design for ECC materials is added, enabling the model to pay more attention to cracks as few-shot categories and hard sample categories, eliminating the performance deficiency problems caused by category imbalance and large differences in difficulty, effectively improving the recognition performance of ECC surface cracks, and obtaining recognition results with high accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of high-ductility fiber-reinforced cementitious composites, and more particularly but not limited to an automatic recognition method for ECC surface cracks based on deep learning. Background Art

[0002] ECC (Engineered Cementitious Composite) is a randomly distributed fiber-reinforced cementitious composite. The number and width of cracks in ECC are closely related to its high ductility and durability, and are also quantitative characterization indexes for the multi-crack cracking process of ECC. The crack width control ability of ECC restricts the durability of ECC structures, and its permeability completely depends on the crack width. In addition, the self-healing performance of cracked ECC is also closely related to the crack width. Therefore, crack assessment must be carried out to provide accurate and comprehensive information on the number and width of crack development in ECC.

[0003] In recent years, with the development of the field of artificial intelligence, deep learning algorithms have excellent applications in all walks of life. Among them, the convolutional neural network has the advantages of multi-core extraction, local perception, parameter sharing, etc. Without the participation of artificial and prior knowledge, it can accurately and efficiently achieve various goals, becoming a milestone network in the field of computer vision. With the proposed structure of the fully convolutional neural network FCN (Fully Convolutional Network), the convolutional neural network has been successfully applied to the field of semantic segmentation as the backbone network, realizing the leap from image-level classification to pixel-level classification.

[0004] However, in the field of materials science, the application and development of deep learning are still in a relatively backward stage. Most of the automatic recognition methods for ECC surface cracks based on deep learning lack research and design for the material properties of ECC, resulting in poor recognition effects. Therefore, how to establish an automatic recognition model for ECC surface cracks according to the background, crack characteristics and property differences of the ECC surface, and achieve accurate and efficient recognition of ECC surface cracks has strong practical significance for promoting the technical application of ECC materials.

[0005] In view of this, a new method is needed to solve at least some of the above problems. Summary of the Invention

[0006] Aiming at one or more problems in the prior art, the present invention proposes an automatic recognition method for ECC surface cracks based on deep learning, which solves the problems of lack of structural design for the material properties of ECC and insufficient recognition performance in the prior art.

[0007] The technical solution for achieving the object of the present invention is as follows:

[0008] A method for automatic identification of ECC surface cracks based on deep learning, comprising:

[0009] Step 1: Obtain the original ECC surface image and preprocess the original ECC surface image;

[0010] Step 2: Design the loss function of the automatic identification model for ECC surface cracks based on the dual strategies of few-shot classes and hard samples;

[0011] Step 3: Combine the loss function, construct an automatic identification model for ECC surface cracks based on a semantic segmentation network, and use the constructed model to perform pixel-level classification on the ECC surface image to be measured.

[0012] Furthermore, for the method for automatic identification of ECC surface cracks based on deep learning of the present invention, the specific steps of the preprocessing in Step 1 include:

[0013] Step 1-1: Cut the collected ECC surface image to obtain several ECC surface images with uniform pixel sizes;

[0014] Step 1-2: Use a deep learning annotation tool to classify and label all pixels in all the ECC surface images obtained in Step 1-1 according to cracks and backgrounds to form an ECC surface image dataset;

[0015] Step 1-3: Divide the above ECC surface images into a training set, a validation set, and a test set according to the quantity ratio of 8:1:1.

[0016] Furthermore, for the method for automatic identification of ECC surface cracks based on deep learning of the present invention, the specific steps of designing the loss function in Step 2 include:

[0017] Step 2-1: Calculate the adaptive class weights of the ECC surface image:

[0018]

[0019] In the formula, c represents the true class, where c = 0 represents the background, c = 1 represents the crack, m represents the number of classes, n represents the total number of pixels in the dataset, and α c represents the number of pixels of the c-th class;

[0020] Step 2-2: Use the softmax function as the activation function to calculate the probability of being predicted as a crack:

[0021]

[0022] In the formula, z1 represents the input of the neuron corresponding to the crack;

[0023] Step 2-3: Calculate the loss function of the model:

[0024]

[0025] In the formula, γ is the sample difficulty balance coefficient, and γ > 1.

[0026] Furthermore, for the automatic ECC surface crack recognition method based on deep learning of the present invention, the construction of the automatic ECC surface crack recognition model in step 3 specifically includes:

[0027] Step 3-1: Feed the training set into the semantic segmentation network and train it with the goal of minimizing the loss function. After one iteration cycle of training, obtain the automatic ECC surface crack recognition model and lock the current model;

[0028] Step 3-2: Feed the validation set into the above-locked automatic ECC surface crack recognition model and calculate the evaluation metrics of the model;

[0029] Step 3-3: If the current evaluation metric is higher than that of the previous iteration cycle, retain the automatic ECC surface crack recognition model of the current iteration cycle; otherwise, discard the automatic ECC surface crack recognition model corresponding to the current iteration cycle; until the training of all iteration cycles is completed, and obtain the optimal automatic ECC surface crack recognition model.

[0030] Furthermore, for the automatic ECC surface crack recognition method based on deep learning of the present invention, the calculation formula of the evaluation metric of the model is:

[0031]

[0032] In the formula, N i,j represents the number of pixels with the true label of i but predicted as j.

[0033] The present invention adopts the above technical solutions and has the following technical effects compared with the prior art:

[0034] For the automatic ECC surface crack recognition method based on deep learning of the present invention, when designing the loss function, an adaptive class weight and a sample difficulty balance coefficient designed for ECC materials are added, so that the obtained automatic recognition model pays more attention to cracks as few-sample categories and difficult-sample categories, eliminating the performance deficiency problems caused by class imbalance and large differences in difficulty. Through simulation and experiments, it can be obtained that using the method of the present invention effectively improves the recognition performance of ECC surface cracks. Description of the Drawings

[0035] The drawings are used to provide a further understanding of the present invention, and are used together with the description to explain the embodiments of the present invention, and do not constitute a limitation to the present invention. In the drawings:

[0036] Figure 1 The flowchart of the deep learning-based automatic ECC surface crack identification method of the present invention is shown.

[0037] Figure 2 The flowchart of the preprocessing of the ECC surface image in the deep learning-based automatic ECC surface crack identification method of the present invention is shown.

[0038] Figure 3 The identification result graph of an embodiment of the deep learning-based automatic ECC surface crack identification method of the present invention is shown.

[0039] Figure 4 The comparison graph of the identification performance results between the method of the present invention and the method of the prior art is shown, where (a) is the comparison result of the crack identification accuracy, and (b) is the comparison result of the background identification accuracy. Detailed implementation manners

[0040] To further understand the present invention, the preferred implementation manners of the present invention will be described below in conjunction with embodiments. However, it should be understood that these descriptions are only for further explaining the features and advantages of the present invention, rather than limiting the claims of the present invention.

[0041] The description of this part only focuses on typical embodiments, and the present invention is not limited to the scope described in the embodiments. Combinations of different embodiments, mutual replacement of some technical features in different embodiments, and mutual replacement of the same or similar prior art means and some technical features in the embodiments are also within the scope of description and protection of the present invention.

[0042] A deep learning-based ECC surface crack identification method, as Figure 1 shown, includes the following steps:

[0043] (1) Preprocess the ECC surface image. Taking the collected 4016×6016 RGB raw image as an example, the preprocessing process of the dataset image is as Figure 2 shown. The specific steps of the preprocessing include:

[0044] (1.1) Eliminate invalid images.

[0045] (1.2) Perform image cutting to make the pixel size of the ECC surface image uniformly 480×352.

[0046] (1.3) Use Labelme (a deep learning annotation tool) to label all pixels in the image into two categories: cracks and background, and store the labeled image in PNG format. In the preparation of the dataset, artificial annotation of cracks and background is adopted, and the annotation is carried out according to the on-site marks on the physical object.

[0047] (1.4) Divide all the processed images (a total of 604 images) into a training set, a validation set, and a test set according to a ratio of 8:1:1, with 480 images, 60 images, and 64 images respectively.

[0048] (2) Design a loss function based on a dual strategy. The dual strategy refers to the strategy for ECC surface cracks belonging to few-shot and hard-shot categories. This step specifically includes:

[0049] (2.1) Calculate the adaptive class weight w, with the formula:

[0050]

[0051] In the formula, c is the true class, m represents the number of classes, n represents the total number of pixels in the dataset, and α c represents the total number of pixels in class c. In this embodiment, m = 2, c = 1 represents cracks, and c = 0 represents the background. After statistical calculation, it is obtained that: w1 = 14.3324; w0 = 0.5181;

[0052] (2.2) Use the Softmax function as the activation function to calculate the probability that the sample is predicted by the model as a crack, with the formula:

[0053]

[0054] In the formula, z1 represents the input of the neuron corresponding to the crack;

[0055] (2.3) Calculate the loss function L, with the formula:

[0056]

[0057] In the formula, γ is the sample difficulty balance coefficient, and γ > 1. In this embodiment, γ = 2.

[0058] (3) Build an automatic recognition model for ECC surface cracks. The network structure of the built model follows FCN (semantic segmentation network), as shown in Table 1.

[0059] Table 1 Network structure of the automatic recognition model for ECC surface cracks

[0060]

[0061] As can be seen from Table 1, the network structure includes five convolutional blocks (L1-L5). Each convolutional block contains several convolutions with a 3×3 convolutional kernel, followed by 2×2 pooling respectively. To achieve pixel-level end-to-end semantic segmentation, the commonly used fully connected layer is replaced by two convolutions with a 1×1 convolutional kernel (L6 and L7). For the original 480×352 RGB image, after 5 times of pooling, the length and width of the output image become 1 / 32 of the original image (i.e., 15×11), and 4096 feature maps of 15×11 are obtained in the output after L7. Finally, these feature maps are restored to the size of the original image through 32-fold upsampling.

[0062] The steps to build the ECC surface crack automatic recognition model specifically include:

[0063] (3.1) Send the dataset into the FCN (semantic segmentation network). After one epoch of training is completed, lock the current model;

[0064] (3.2) Send the validation set into the currently locked ECC surface crack automatic recognition model, and calculate the current evaluation index FWIoU of the model:

[0065]

[0066] where N i,j represents the number of pixels with the true label of i but predicted as j.

[0067] (3.3) Model solidification: If the current FWIoU is higher than the FWIoU after the end of the previous epoch, retain the model corresponding to the current epoch; otherwise, discard the model corresponding to the current epoch.

[0068] As Figure 3 shown in the recognition result diagram of an embodiment of the deep learning-based ECC surface crack automatic recognition method of the present invention, where Figure 3 (a) is the image to be recognized, and (b) is the recognition result diagram. Through the test experiments on all the images in the test set, the recognition accuracies of the deep learning-based ECC surface crack automatic recognition method of the present invention for the background and cracks can reach 99.74% and 99.87% respectively.

[0069] As Figure 4 shown in the comparison diagram of the recognition performance results between the method of the present invention and the method of the prior art (i.e., the method before improvement), it can be seen from Figure 3 that the method of the present invention has better recognition effects on cracks (see Figure 4 (a)) and the background (see Figure 4 (b)) than the prior art, which proves the feasibility of the method in engineering practice.

[0070] The description and application of the present invention herein are illustrative and are not intended to limit the scope of the present invention to the above embodiments. The relevant descriptions of effects or advantages involved in the specification may not be reflected in actual experimental examples due to uncertainties in specific condition parameters or other factors, and the relevant descriptions of effects or advantages are not used to limit the scope of the invention. Variations and changes to the disclosed embodiments are possible, and various substitutions and equivalent components of the embodiments are known to those of ordinary skill in the art. Those skilled in the art should clearly understand that the present invention can be implemented in other forms, structures, arrangements, proportions, and with other components, materials, and parts without departing from the spirit or essential characteristics of the present invention. Other variations and changes can be made to the disclosed embodiments without departing from the scope and spirit of the present invention.

Claims

1. An automatic recognition method for ECC surface cracks based on deep learning, characterized in that, Including: Step 1: Obtain the original ECC surface image and preprocess the original ECC surface image; Step 2: Design the loss function of the ECC surface crack automatic recognition model based on the dual strategies of few-shot categories and hard sample categories; The specific steps include: Step 2-1: Calculate the adaptive class weights of the ECC surface image: In the formula, c represents the true category, where c = 0 represents the background, c = 1 represents the crack, m represents the number of categories, n represents the total number of pixels in the dataset, and α c represents the number of pixels in the c-th category; Step 2-2: Use the softmax function as the activation function to calculate the probability predicted as a crack: In the formula, z1 represents the input of the neuron corresponding to the crack; Step 2-3: Calculate the loss function of the model: In the formula, γ is the sample difficulty balance coefficient, γ > 1; Step 3: Combine the loss function and construct an ECC surface crack automatic recognition model based on the semantic segmentation network, and use the constructed model to perform pixel-level classification on the ECC surface image to be measured.

2. The automatic ECC surface crack recognition method based on deep learning according to claim 1, wherein, The specific steps of the preprocessing in Step 1 include: Step 1-1: Cut the collected ECC surface image to obtain several ECC surface images with the same pixel size; Step 1-2: Use a deep learning annotation tool to classify and label all pixels in all the ECC surface images obtained in Step 1-1 according to cracks and backgrounds to form an ECC surface image dataset; Step 1-3: Divide the above ECC surface image dataset into a training set, a validation set, and a test set according to the quantity ratio of 8:1:

1.

3. The automatic ECC surface crack recognition method based on deep learning according to claim 1, characterized in that The construction of the ECC surface crack automatic recognition model in Step 3 specifically includes: Step 3-1: Send the training set into the semantic segmentation network and train it with the goal of minimizing the loss function. After one iteration cycle of training, obtain the ECC surface crack automatic recognition model and lock the current model; Step 3-2: Send the validation set into the above-locked ECC surface crack automatic recognition model and calculate the evaluation index of the model; Step 3-3: If the current evaluation index is higher than the evaluation index of the previous iteration cycle, retain the ECC surface crack automatic recognition model of the current iteration cycle; otherwise, discard the ECC surface crack automatic recognition model corresponding to the current iteration cycle; until the training of all iteration cycles is completed to obtain the optimal ECC surface crack automatic recognition model.

4. The automatic ECC surface crack recognition method based on deep learning according to claim 3, characterized in that The calculation formula of the evaluation index of the model is: where N ij represents the number of pixels with true label i but predicted as j.

Images