A method for inversion and reproduction of multi-scale fracture three-dimensional morphology

Abstract

The present invention discloses a method for inverting and reproducing the three-dimensional morphology of multi-scale cracks, comprising the following steps: 1) obtaining two-dimensional images of cracks of several concrete structures; 2) extracting two-dimensional feature information of the crack surface of the two-dimensional image based on deeplabv3+convolutional neural network; 3) establishing a mapping relationship between the two-dimensional feature information of the crack surface based on deeplabv3+convolutional neural network and its corresponding acoustic wave scattering signal; 4) according to the relationship between the two-dimensional feature information of the crack surface and the acoustic wave scattering information established in step 3), the two-dimensional feature information obtained in step 2) is used to obtain the corresponding acoustic wave scattering signal mapping relationship, and the three-dimensional morphology of the crack is obtained by using the full waveform inversion model. The present invention can reconstruct the three-dimensional morphology of the crack, realize a comprehensive evaluation of the overall parameters of the crack, and thus provide more comprehensive information for the safety evaluation of the concrete structure.

Description

Technical Field

[0001] The invention relates to the field of concrete building safety assessment, and in particular to a multi-scale crack three-dimensional morphology inversion and reproduction method. Background Art

[0002] As an important material in construction engineering, concrete is widely used in infrastructure fields such as dams, bridges and tunnels due to its superior mechanical properties and durability. Among the many types of diseases that affect the performance of concrete buildings, cracks are the most important and most common disease characteristics. Accurately obtaining various parameter information of cracks is the basis for evaluating the health status of concrete structures and subsequent maintenance.

[0003] In the early days, manual methods were the main means of detecting cracks, but there are many defects in manual crack detection methods, such as: low detection efficiency, complex terrain at the construction sites of bridges, dams and tunnels, and most of the external auxiliary equipment such as scaffolding is required for detection. The installation and disassembly process is time-consuming and labor-intensive, and concrete buildings are generally large in area, so manual detection efficiency will be very low; low detection accuracy, when personnel actively detect cracks, their eyes are easily fatigued, cracks will be missed and misdetected, and human eyes will be affected by their own subjective influence when observing cracks; high risk factor, personnel need to go near concrete buildings for detection, and their safety cannot be guaranteed; high cost, there are many concrete buildings, and if they are to be timely and comprehensively detected, a large amount of manpower and material resources will be used, which will bring huge cost consumption; low degree of informatization, when manual detection records data, it is mainly implemented on paper, and some data are not even recorded, and paper materials are difficult to find, which is not conducive to the timely repair of dangerous buildings. With the development of photographic equipment and digital image technology, crack recognition technology based on digital image processing has also been rapidly developed. In particular, with the rapid development of deep learning in the field of target detection, concrete crack recognition technology based on deep learning has also been widely used, and has greatly improved the accuracy and speed of crack recognition. However, due to the influence of geographical environment and natural conditions, the surface cracks of concrete structures have different scale diversity and morphological diversity due to their different formation mechanisms, which leads to an increase in crack feature information, making it difficult to extract crack features in concrete structures, and further affecting the accuracy of crack feature extraction. Large-area stains, pits, honeycomb holes, etc. on the surface of concrete structures have a certain degree of influence on crack recognition. In addition, the edges of Zen seams (template imprints, etc.), pipes, cables, brackets, holes, decorations, etc. on the surface of concrete structures are slender after imaging, which are easily confused with cracks after imaging, and the rust on the surface of steel structures will also interfere with crack recognition. At the same time, the differences in image acquisition equipment and acquisition distance, angle, lighting, etc., also further affect the accuracy of crack recognition. Moreover, with the continuous introduction of information technology into the field of intelligent structural health monitoring, higher requirements are also put forward for the recognition of multi-scale cracks in concrete structures. In recent years, concrete crack recognition technology based on computer vision has developed rapidly. However, this technology can only extract crack surface information, such as crack length and width, surface two-dimensional morphology, and crack direction, but it cannot effectively extract three-dimensional information such as crack depth. However, three-dimensional information such as crack depth is key information for determining the degree of impact of cracks on structural safety. With the continuous development of information technology and the improvement of structural health monitoring requirements, simple crack identification or depth detection has become increasingly unable to meet the requirements of structural health monitoring. It is necessary to comprehensively evaluate the overall parameters of the crack-cavity (including length, width, depth, morphology, etc.) to provide more comprehensive information for structural safety assessment. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a multi-scale crack three-dimensional morphology inversion and reproduction method, which can reconstruct the three-dimensional morphology of the cracks and realize a comprehensive evaluation of the overall parameters of the cracks, such as length, width, depth, morphology, etc., thereby providing more comprehensive information for the safety assessment of concrete structures.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows: a method for inversion and reproduction of multi-scale fracture three-dimensional morphology, characterized in that it comprises the following steps:

[0006] 1) Obtain two-dimensional images of cracks in several concrete structures;

[0007] 2) Extract the two-dimensional feature information of the crack surface of the two-dimensional image based on deeplabv3+convolutional neural network;

[0008] 3) Establish the mapping relationship between the two-dimensional feature information of the crack surface and its corresponding acoustic wave scattering signal based on deeplabv3+convolutional neural network;

[0009] 4) According to the relationship between the two-dimensional characteristic information of the crack surface and the acoustic scattering information established in step 3), the two-dimensional characteristic information obtained in step 2) is used to obtain the corresponding acoustic scattering signal mapping relationship, and the three-dimensional morphology of the crack is obtained using the full waveform inversion model.

[0010] Furthermore, in step 2), the process of extracting the two-dimensional feature information of the crack surface is as follows:

[0011] a. Dataset preprocessing: Use annotation tools to annotate the crack 2D images to form a json file, and convert the json file into a label image; flip, rotate, crop and / or scale the original crack 2D images and their corresponding label images to enhance the crack 2D images and label image data, and finally form a crack 2D image dataset;

[0012] b. Network training: Input the crack 2D image dataset into the deeplabv3+ convolutional neural network of the ECA module with an efficient channel attention mechanism, and train it until the network converges to obtain the final crack semantic segmentation network;

[0013] c. Network prediction: Input the two-dimensional crack image into the trained crack semantic segmentation network to obtain the semantic segmentation image of the crack, that is, the two-dimensional feature information of the crack surface.

[0014] Furthermore, in the deeplabv3+ convolutional neural network, the ECA module structure performs global average pooling on the input data, then learns through 1D convolution, and finally uses the Sigmoid function to fix the value between 0 and 1 to obtain the weight of each channel of the input feature layer, and then multiply the weight by the original input feature layer to obtain the ECA module output data.

[0015] Furthermore, in step 3), the process of acquiring the mapping relationship between the two-dimensional feature information and its corresponding acoustic wave scattering signal is as follows:

[0016] a. Using ultrasonic equipment to detect the acoustic wave scattering information of the concrete entity corresponding to the two-dimensional image of the finite tension crack;

[0017] b. Fusing the two-dimensional feature information of the crack surface of the above-mentioned finite crack two-dimensional image and the acoustic wave scattering information into the input feature data of the neural network, and inputting them into the multi-channel deep neural network; wherein the multi-channel deep neural network has a total of L+1 layers, wherein the 0th layer is the multi-channel input layer, the 1st to L-1 layers are the model hidden layers, the Lth layer is the output layer, and adjacent layers are connected by a feedforward weight matrix;

[0018] c. Set different L values ​​to construct the network hidden layer, and train the models separately to obtain the optimal L value to determine the final network structure;

[0019] d. Input the two-dimensional feature information of other crack two-dimensional images into the neural network to obtain the acoustic wave scattering information corresponding to the two-dimensional feature information.

[0020] Furthermore, in step 4), the process of obtaining the three-dimensional morphology of the crack is as follows:

[0021] (1) Initialize the three-dimensional morphology of the crack;

[0022] (2) Calculate the acoustic wave scattering signal of the current three-dimensional morphology of the crack based on the wave equation;

[0023] (3) obtaining an objective function based on the difference between the acoustic wave scattering signal of the current three-dimensional crack model and the actual acoustic wave scattering signal;

[0024] (4) Determine whether the objective function converges. If not, the L-BFGS algorithm is used to update the current three-dimensional morphology of the crack, and repeat steps (2), (3), and (4) until the objective function converges.

[0025] (5) Output the three-dimensional shape of the crack.

[0026] Compared with the prior art, the present invention has the following advantages: the method extracts the crack surface information through the deeplabv3 neural network, and then combines the acoustic wave scattering technology and the multi-channel neural network to establish the corresponding relationship between the two-dimensional characteristics of the crack surface and its acoustic wave scattering information, thereby obtaining the mapping relationship between the two-dimensional information of the crack and its acoustic wave scattering signal, and then inverts the acoustic wave scattering signal corresponding to the two-dimensional characteristics of the crack surface to obtain the three-dimensional morphology of the crack; thereby, the three-dimensional morphology of the crack can be reconstructed, and the overall parameters of the crack, such as length, width, depth, morphology, etc., can be comprehensively evaluated, thereby providing more comprehensive information for the safety assessment of concrete structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a principle block diagram of the present invention.

[0028] Figure 2 Schematic diagram of the deeplabv3+ neural network structure.

[0029] Figure 3 Schematic diagram of the ECA module structure.

[0030] Figure 4 Flowchart for two-dimensional feature extraction of crack surface.

[0031] Figure 5 This is a diagram of the multi-channel neural network structure.

[0032] Figure 6 This is the flow chart of full waveform inversion. DETAILED DESCRIPTION

[0033] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0034] Example: See Figure 1-Figure 6 A method for inverting and reproducing multi-scale fracture three-dimensional morphology includes the following steps:

[0035] 1) Obtain 2D images of cracks in several concrete structures.

[0036] 2) Extract the two-dimensional feature information of the crack surface of the two-dimensional image based on deeplabv3+convolutional neural network; the extraction process is as follows:

[0037] a. Dataset preprocessing: Use annotation tools to annotate the crack 2D images to form a json file, and convert the json file into a label image; flip, rotate, crop and / or scale the original crack 2D images and their corresponding label images to enhance the crack 2D images and label image data, and finally form a crack 2D image dataset;

[0038] b. Network training: Input the crack 2D image dataset into the deeplabv3+ convolutional neural network of the ECA module with an efficient channel attention mechanism, and train it until the network converges to obtain the final crack semantic segmentation network;

[0039] c. Network prediction: Input the two-dimensional crack image into the trained crack semantic segmentation network to obtain the semantic segmentation image of the crack, that is, the two-dimensional feature information of the crack surface.

[0040] Among them, in the deeplabv3+ convolutional neural network, the ECA module structure performs global average pooling on the input data, then learns through 1D convolution, and finally uses the Sigmoid function to fix the value between 0 and 1 to obtain the weight of each channel of the input feature layer, and then multiply the weight by the original input feature layer to obtain the ECA module output data. The ECA module can highlight useful channels, suppress redundant channel information, and strengthen channel information learning in a targeted manner according to the contribution of each channel to crack segmentation, further improving the model's crack segmentation effect.

[0041] In this solution, deeplabv3+ uses the deeplabv3 network model as the encoder, and cascades a simple and efficient decoder at its back end. The spatial pyramid pooling module in the encoder part samples in parallel with dilated convolutions at different sampling rates, then splices the results together, and then uses 1x1 convolution to reduce the number of channels to the expected value to achieve the context of capturing images at multiple scales. After passing through the ASPP module, the data contains rich channel information. The ECA module performs global average pooling on the input data, then learns through 1D convolution, and finally uses the Sigmoid function to fix the value between 0-1. At this time, the weight of each channel of the input feature layer is obtained. This weight multiplied by the original input feature layer is the output data of the ECA module, which can highlight the useful channels and suppress redundant channel information according to the information carried by each channel. Strengthen channel information learning in a targeted manner, and further improve the model's effect on crack segmentation.

[0042] Crack surface two-dimensional feature extraction process based on deeplabv3+convolutional neural network Figure 4 As shown in the figure. The two-dimensional crack image is annotated using annotation tools, and the json file formed by the annotation is converted into a label image using code. The original two-dimensional crack image and its corresponding label image are enhanced by data, including flipping, rotating, cropping, etc., to finally form a two-dimensional crack image dataset. The crack image dataset is input into the deeplabv3+ network with an efficient channel attention mechanism, and the crack semantic segmentation network is trained to finally reach the convergence of this network and build a two-dimensional information extraction mechanism for the crack surface.

[0043] 3) Establish the mapping relationship between the two-dimensional feature information of the crack surface and its corresponding acoustic scattering signal based on deeplabv3+convolutional neural network; the specific acquisition process is as follows:

[0044] a. Use ultrasonic equipment to detect the sound wave scattering information of the concrete entity corresponding to the two-dimensional image of the crack.

[0045] b. The two-dimensional feature information of the crack surface and the acoustic wave scattering information are fused and input into the input feature data of the neural network, and then input into the multi-channel deep neural network; wherein, the multi-channel deep neural network has a total of L+1 layers, wherein the 0th layer is the multi-channel input layer, the 1st to L-1 layers are the model hidden layers, the Lth layer is the output layer, and adjacent layers are connected by a feedforward weight matrix.

[0046] c. Set different L values ​​to construct the network hidden layer, and train the models separately to obtain the optimal L value to determine the final network structure.

[0047] d. Input the two-dimensional image of the crack into the neural network to obtain the acoustic wave scattering information of the crack.

[0048] In this scheme, the structure of the Multi-Channel Deep Neural Network (MCDNN) is as follows Figure 5 As shown. The ultrasonic equipment is used to measure the acoustic wave scattering information of the concrete entity corresponding to the two-dimensional image of the crack, and the acoustic wave scattering information and the two-dimensional features of the crack surface are spliced ​​into the input feature data of the neural network through standardized mapping, and input into the multi-channel neural network. The neural network finds the relationship between the two-dimensional features of the crack surface and its acoustic wave scattering information. When the neural network training converges, the acoustic wave scattering information corresponding to the crack can be obtained from the input two-dimensional information of the crack surface. MCDNN has a total of L+1 layers, of which the 0th layer is the multi-channel input layer, the 1st to L-1 layers are the model hidden layers, the Lth layer is the output layer, and adjacent layers are connected by a feedforward weight matrix. It is generally believed that the number of hidden layers affects the accuracy of the model.

[0049] The more hidden layers there are, the higher the accuracy is, but it also makes the network more complicated, which increases the training time of the network and the tendency of overfitting. In this method, different L values ​​are set to construct the network hidden layer, and the models are trained separately. The accuracy and computational efficiency of different models are compared, and the optimal L value is selected to determine the final network structure.

[0050] According to the multi-channel deep neural network structure diagram and combined with the neural network calculation rules, we can get:

[0051] z l =w l zl-1 +b l ,l=1,2,…,L (1);

[0052] h l =f l (z l ) (2);

[0053] in is the weight matrix from layer 1 to layer l-1, is the bias vector of the first layer. l It is the characteristic data layer unit. l is the output vector of each layer, f l is the activation function of the lth layer. This project intends to use the ReLU function as the activation function of each layer, in the following form:

[0054]

[0055] Where λ is the sample convergence threshold. In summary, the feature x of the training sample first enters the input layer and then propagates forward along the multi-channel deep neural network. For convenience, the parameters of MCDNN are recorded as θ = {W l ,b l |l=1,2,…,L}, the characteristics of the training sample set are recorded as The corresponding label is ζ={r i |i=1,2,…,S}. Then the loss function on the training set is

[0056]

[0057] E(θ,x i ,r i ) is the training sample x i The corresponding mean square error loss function, the purpose of neural network training is to minimize the mean square error function. Therefore, in order to ensure the accuracy and computational efficiency of network training, this project intends to use mini-batch gradient descent (MBGD) to solve the minimization problem of the above error loss function. In each iteration, MBGD randomly selects a small number of samples from the training samples to form a mini-batch, calculates the gradients corresponding to these small samples, and uses these gradient information to update the model parameters. Compared with batch gradient descent (BGD) and stochastic gradient descent (SGD), MBGD has a faster convergence speed, thereby improving the training speed of MCDNN to a certain extent.

[0058] 4) According to the relationship between the two-dimensional characteristic information of the crack surface and the acoustic scattering information established in step 3), the two-dimensional characteristic information obtained in step 2) is used to obtain the corresponding acoustic scattering signal mapping relationship, and the three-dimensional morphology of the crack is obtained using the full waveform inversion model; the specific acquisition process is as follows:

[0059] (1) Construct a three-dimensional model and initialize the three-dimensional shape of the crack;

[0060] (2) calculating the acoustic wave scattering signal of the current three-dimensional shape of the crack according to the wave equation, that is, generating the acoustic wave scattering signal of the current three-dimensional shape of the crack through the prediction model;

[0061] (3) obtaining an objective function based on the difference between the acoustic wave scattering signal of the current three-dimensional crack model and the actual acoustic wave scattering signal;

[0062] (4) Determine whether the objective function converges. If not, the L-BFGS algorithm is used to update the current three-dimensional shape of the crack, and repeat steps (2), (3), and (4) until the objective function converges (the acoustic wave scattering signal of the three-dimensional crack model is the same as the actual acoustic wave scattering signal);

[0063] (5) Output the FWI inversion model, that is, the final output is the three-dimensional shape of the fracture.

[0064] Specifically, step 2 can obtain the two-dimensional information of the crack surface, and step 3 can obtain the ultrasonic scattering information of the corresponding crack from the surface information of the crack. Through the obtained ultrasonic scattering information, the three-dimensional morphology of the crack can be obtained by using full waveform inversion (FWI). The task of the full waveform inversion algorithm is to gradually reduce the mismatch function value between the observed data and the calculated data through an iterative optimization method.

[0065] The propagation of a pressure wave caused by an applied sound wave in a 2D or 3D medium over a time span of T seconds is modeled by the acoustic wave equation, as follows:

[0066]

[0067] Where u represents the sound field, v p represents the speed of sound, f s Represents a point source ultrasonic pulse.

[0068] Define the objective function:

[0069]

[0070] Where Ω is the effective simulation area, m is the iterative model, T is a time step, For x i The ultrasonic information at time T at the location, u(m;x,t) is the iterative model at x i Ultrasonic information within time T at .

[0071] The L-BFGS optimization method is used to solve the objective function J(m). The model parameter m is updated by the following formula:

[0072]

[0073] In it, α is the step size and δm is the perturbation (or model parameter increment).

[0074] BFGS formula:

[0075]

[0076] Where s = m k+1 -m k is the model parameter vector increment, s = m k+1 -m k y=g(m k+1 )-g(m k ) is the gradient vector increment. Therefore, the inverse Hessian matrix is:

[0077]

[0078] In order to reduce the number of s and y, Nocedal developed the L-BFGS method, defining l as the storage limit, and letting, in the optimization process, in order to stabilize the calculation process, the Wolfe condition needs to be satisfied:

[0079] J(m k +λ k d k )≤J(m k )+μλ k g * d k (10);

[0080] |g(m k +λ k d k ) * d k |≥v|g(m k ) * d k | (11).

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. A method for inversion and reproduction of multi-scale fracture three-dimensional morphology, characterized in that: The steps include: 1) Obtain two-dimensional images of cracks in several concrete structures; 2) Extract the two-dimensional feature information of the crack surface of the two-dimensional image based on deeplabv3+convolutional neural network; 3) Establish the mapping relationship between the two-dimensional feature information of the crack surface and its corresponding acoustic wave scattering signal based on deeplabv3+convolutional neural network; The process of obtaining the mapping relationship between the two-dimensional feature information and its corresponding acoustic scattering signal is as follows: a. Using ultrasonic equipment to detect the acoustic wave scattering information of the concrete entity corresponding to the two-dimensional image of the finite tension crack; b. Fusing the two-dimensional feature information of the crack surface of the above-mentioned finite crack two-dimensional image and the acoustic wave scattering information into the input feature data of the neural network, and inputting them into the multi-channel deep neural network; wherein the multi-channel deep neural network has a total of L+1 layers, wherein the 0th layer is the multi-channel input layer, the 1st to L-1 layers are the model hidden layers, the Lth layer is the output layer, and adjacent layers are connected by a feedforward weight matrix; c. Set different L values ​​to construct the network hidden layer, and train the models separately to obtain the optimal L value to determine the final network structure; d. inputting the two-dimensional feature information of other crack two-dimensional images into the neural network to obtain the acoustic wave scattering information corresponding to the two-dimensional feature information; 4) according to the relationship between the two-dimensional characteristic information of the crack surface and the acoustic scattering information established in step 3), the two-dimensional characteristic information obtained in step 2) is used to obtain the corresponding acoustic scattering signal mapping relationship, and the three-dimensional morphology of the crack is obtained by using the full waveform inversion model; The process of obtaining the three-dimensional morphology of the crack is as follows: (1) Initialize the three-dimensional morphology of the crack; (2) Calculate the acoustic wave scattering signal of the current three-dimensional morphology of the crack based on the wave equation; (3) obtaining an objective function based on the difference between the acoustic wave scattering signal of the current three-dimensional crack model and the actual acoustic wave scattering signal; (4) Determine whether the objective function converges. If not, the L-BFGS algorithm is used to update the current three-dimensional morphology of the crack, and repeat steps (2), (3), and (4) until the objective function converges. (5) Output the three-dimensional shape of the crack.

2. A method for inversion and reproduction of multi-scale fracture three-dimensional morphology according to claim 1, characterized in that: In step 2), the process of extracting the two-dimensional feature information of the crack surface is as follows: a. Dataset preprocessing: Use annotation tools to annotate the crack 2D images to form a json file, and convert the json file into a label image; flip, rotate, crop and / or scale the original crack 2D images and their corresponding label images to enhance the crack 2D images and label image data, and finally form a crack 2D image dataset; b. Network training: Input the crack 2D image dataset into the deeplabv3+ convolutional neural network of the ECA module with an efficient channel attention mechanism, and train it until the network converges to obtain the final crack semantic segmentation network; c. Network prediction: Input the two-dimensional crack image into the trained crack semantic segmentation network to obtain the semantic segmentation image of the crack, that is, the two-dimensional feature information of the crack surface.

3. A method for inversion and reproduction of multi-scale fracture three-dimensional morphology according to claim 2, characterized in that: In the deeplabv3+ convolutional neural network, the ECA module structure performs global average pooling on the input data, then learns through 1D convolution, and finally uses the Sigmoid function to fix the value between 0 and 1 to obtain the weight of each channel of the input feature layer, and then multiply the weight by the original input feature layer to obtain the ECA module output data.

Images