U-Net network construction method based on dual input and image separable convolution

Abstract

The invention discloses a U-Net network construction method based on dual input and image separable convolution, and the method achieves the data imaging of a ground penetrating radar (GPR) based on the dual input Unet (DIWSUnet) improved by the image separable convolution, and obtains an underground real object through the inversion. The method comprises the steps that firstly, a B-Scan scanning image is generated based on GPR data obtained by a ground penetrating radar, then frequency-wave shift (FK) processing is conducted through the GPR data, and dual-input inversion of the B-Scan scanning image and an FK offset image is achieved to obtain a final real object image; different real objects are distinguished by adopting double-color input (6 channels) and color output (3 channels), and the types of the objects are judged while the underground positions of the real objects are inverted; according to the method, inspiration is obtained based on a depth separable convolution model, an original input module is improved into an image separable convolution (IWS) module, and the performance is greatly improved only by using a small amount of parameter increase.

Description

Technical Field

[0001] The present invention belongs to the field of GPR inversion imaging, and specifically is a U-Net network construction method based on dual-input and image separable convolution. Background Art

[0002] Ground Penetrating Radar (GPR) uses high-frequency electromagnetic waves to detect underground structures. It can obtain high-resolution underground images in real time and is widely used in archaeology, geological exploration, environmental monitoring, infrastructure assessment and other fields. With the acceleration of urbanization and aging of infrastructure, the detection and maintenance of underground facilities have become increasingly important.

[0003] Ground Penetrating Radar (GPR) is widely used in detecting underground objects and imaging underground structures in geophysical and civil engineering applications, which can help us better understand the structural properties of underground objects. In the GPR inversion problem, the properties of the underground scene can be reconstructed from the electromagnetic (EM) information in the recorded GPR data.

[0004] The underground image inversion technology processes and analyzes the data obtained by GPR to reconstruct the true characteristics of underground objects. This technology can help researchers and engineers accurately identify underground obstacles, soil characteristics and underground physical objects, thereby providing a reliable basis for engineering design, construction and safety assessment, especially the identification and detection of underground cavities. In addition, by improving the inversion algorithm, the imaging accuracy can be improved, further promoting the application of GPR technology in complex environments.

[0005] In the field of GPR inversion imaging, there are some traditional methods and deep learning-based methods. Among the traditional methods, there are several classic methods that can reconstruct underground structure images from the acquired GPRB-scan images, including migration algorithms, tomography methods, and full-wave inversion (FWI) technology. In addition, there are some migration algorithms, such as Reverse Time Migration (RTM) and Kirchhoff Migration, which are used to convert unfocused space-time GPR images into focused GPR images to show the position and size of objects. However, these methods do not show the parameters such as the structural shape of the original object very well, and there will be certain position deviations.

[0006] Each iteration of the FWI algorithm requires a forward modeling, which consumes a lot of computational cost. In summary, the traditional method has two main defects. The first is that the positioning of structural parameters is not accurate enough. The second is that the dielectric constant map is iteratively constructed, and the mapping process requires a lot of computational cost. The third is that in complex scenarios, the constructed dielectric constant map is not accurate enough and is affected by factors such as noise.

[0007] In this case, methods based on deep learning came into being. At present, there are many deep learning methods to model Bscan images obtained from GPR data, such as using Unet. Unet is a deep learning model widely used in image semantic segmentation or image style transfer. Compared with traditional methods, deep learning methods have obvious efficiency and accuracy. Most of them are single-input and single-output, inputting a Bscan image and outputting the predicted structural position of the physical object. Most methods also stay at the single-channel method and do not realize the output of the color channel to achieve multi-target and multi-category prediction, thereby realizing the reconstruction of the dielectric constant map. In addition, most of the simulated real images are in the case of large real objects. The invention simulates the situation where the real object (radius 2 to 5 cm) is much smaller than the overall scanned area (2.5*0.5m), and achieves good results. Summary of the invention

[0008] In order to solve the above problems, the present invention proposes a U-Net network construction method based on dual input and image separable convolution. The present invention first uses two inputs, and uses the B-scan image and the image after FK offset processing as input to improve the accuracy of the model; the color image is used as input to predict the color image of the physical object, and the color mapping relationship from the original physical object image to the target physical object image obtained by Paraview modeling is used to reconstruct the actual dielectric constant map, and accurately classify the physical object. In addition, the input of Unet is improved, and the image separable convolution module is improved by inspiration from the deep separable module. The model is screened using IoU as an indicator to screen the optimal model and obtain the output inversion image, so that the actual position, radius size, and category of the object can be comprehensively analyzed.

[0009] To achieve the above object, the technical solution adopted by the present invention is:

[0010] The U-Net network construction method based on dual input and image separable convolution is characterized by comprising the following steps:

[0011] S1, GPR data simulation:

[0012] Use three different common physical objects including metal, plastic and air to generate GPRmax input in files, and then use GPRmax to simulate and generate GPR data;

[0013] S2, GPR data preprocessing and dataset construction:

[0014] For the GPR data obtained in S1, denoising preprocessing is first performed. Specifically, the geological information without physical objects is also scanned to obtain background GPR data, which is then subtracted to obtain denoised GPR data. Next, the denoised GPR data is used to draw a B-Scan imaging image to obtain one of the input data sets; the denoised GPR data is then FK-migrated, and the imaging after FK migration is output to obtain the second part of the input data set. Finally, the VTI file obtained in the simulation process is input into the Paraview software for physical image visualization, and then denoising and color mapping are performed to obtain the target image data set. The images in the data set are divided into training sets and test sets according to a certain ratio.

[0015] S3, trained using a dual-input Unet based on an improved image separable module:

[0016] Use the improved network for training, perform deep learning on the training set, select appropriate learning rate planning and measurement criteria, and set the loss function to BCEloss;

[0017] S4, select the optimal model and output the inversion image:

[0018] The optimal model is selected based on IoU, and the inverted physical image is output. The category of the output physical object is determined by analyzing the color of the physical object. In addition, the radius and location of the underground physical object can be directly determined based on the output physical image.

[0019] As a further improvement of the present invention, the specific steps of step S2 are as follows:

[0020] S21, denoising the target object image:

[0021] After using Paraview to obtain the preliminary physical image I, first use the background VTI file without the physical object to visualize the physical image using Paraview to obtain the background B, and then subtract the two IB to obtain the denoised physical geological image I1;

[0022] S22, color mapping is performed on the physical image:

[0023] At this time, the color of the physical object in I1 cannot be directly used as the target image. The colors need to be mapped. The three different physical objects are mapped to three colors (0, 255, 255), (255, 0, 255), and (255, 255, 0), and the background is set to white (255, 255, 255) to obtain I2. This makes it convenient to use the Unet network for binary classification on the same channel.

[0024] As a further improvement of the present invention, the specific steps of step S3 are as follows:

[0025] S31, combining depth-wise separable convolution and ordinary convolution:

[0026] Inspired by the deep separable convolutional network, we designed a separable convolution in the image dimension. The original deep separable convolution only performs convolution at the depth level, and then performs a 1*1 convolution on the obtained feature map to enlarge the feature dimension to 64 dimensions.

[0027] S32, input channel transformation:

[0028] At the time of input, the original input channel is changed to 6 channels, and two 3-channel RGB images, namely the B-Scan image and the FK offset image, are received at the same time, and split into two three-channel images. Each of them passes through a convolution layer containing 32 filters, and each obtains a 32-dimensional feature matrix. After BatchNorm and ReLU, the two feature matrices are connected in the feature dimension and spliced ​​into 64 channels. Finally, they pass through 1*1 convolution of 64 filters to keep the feature dimension at 64 dimensions.

[0029] S33, the number of parameters is only slightly increased:

[0030] Assume that the input channel is C, the image is H×W, the number of feature maps in the first layer is M, and the convolution kernel size is K;

[0031] Because the first layer has two convolutional layers and BatchNorm parameters, the original parameters of the first layer are:

[0032] Num param =(K×K×C×M+2×M)+(K×K×M×M+2×M)

[0033] Now we use the IWS module instead. Assuming that the size of the inner channel for each image is S, the parameter amount is:

[0034] Num′ param1 =K×K×C×S+4×S

[0035] Num′ param2 =2×S×M+2×M

[0036] Num′ param3 =K×K×M×M+2×M

[0037] Num′ param =Num′ param1 +Num′ param2 +Num′ param3

[0038] Only for the left input module of the first layer, substituting the actual parameters used, we can get Num param is 40576;

[0039] For the IWS module, the number of parameters is 42944. For the small modules in the first layer, there are only 2368 more parameters. For the entire model, it is a symmetrical structure and the number of parameters below increases exponentially. The total number of parameters of the model is about 40 million, and the number of parameters increased is less than one ten-thousandth.

[0040] S34, set the loss function to BCELoss:

[0041] BCELoss is often used as a loss function in the traditional image segmentation field or grayscale image inversion field;

[0042]

[0043] On the basis of the original, first calculate the loss in the channel dimension, and then find the average:

[0044]

[0045] As a further improvement of the present invention, the specific steps of step S4 are as follows:

[0046] S41, binarize the predicted output image:

[0047] Because the prediction output is three channels, each channel is classified into two categories to determine whether it is a physical object or background. The first channel is the channel to determine whether it is a plastic object (0, 255, 255), the second channel is to determine whether it is air, and the third channel is to determine whether it is metal. Before inputting the image, the image has been divided by 255 and constrained between 0 and 1. Therefore, before judging IoU, a threshold must be set to binarize the output prediction result. If the output value is less than the threshold, it is a physical object and it is converted to 1, otherwise it is 0, thus converting it to a binary mask of 0 and 1.

[0048] S42, calculate the intersection-and-combination ratio of each channel:

[0049] For each channel of the binary mask, calculate the number of pixel elements with an output value of 1, and also calculate the number of pixel elements of the physical object in the target physical image, and you can get the intersection INtersection and union Union of the two.

[0050]

[0051] Beneficial effects:

[0052] The present invention is based on a U-Net network construction method of dual input and image separable convolution, and an improved dual input Unet (DIWSUnet) based on image separable convolution to realize imaging with ground-penetating radar (GPR) data and invert underground physical objects. First, a B-Scan scan image is generated based on the GPR data obtained by the ground-penetating radar, and then the GPR data is processed by frequency-wavenumber shift (FK) to realize the double input inversion of the B-Scan scan image plus the FK shift image to obtain the final physical image; dual color input (6 channels) and color output (3 channels) are used to distinguish different physical objects, and the underground position of the physical object is inverted while the category of the object is judged; inspired by the deep separable convolution model, the original input module is improved to an image separable convolution module IWS (Image Wise Seperable Convolution) module, and the performance is greatly improved by only a small amount of parameter increase. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is the overall process of using the improved network to perform ground penetrating radar GPR inversion imaging;

[0054] Figure 2 This is the structure diagram of the image separable convolution module;

[0055] Figure 3 This is the corresponding DIWSUnet network structure diagram;

[0056] Figure 4 This is a comparison chart of the recognition results of inversion imaging. DETAILED DESCRIPTION

[0057] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0058] The specific overall inversion identification process of the present invention is as follows: Figure 1 shown.

[0059] Based on the improved dual-input Unet (DIWSUnet) of the image separable module, the Bscan image obtained by GPR data and the FK migration image obtained by FK migration are used as input to invert the real object image. Specifically, the following steps are included:

[0060] S1, GPR data simulation:

[0061] Three different common physical objects (metal, plastic, air) are used to generate GPRmax input in files, and then GPRmax is used to simulate and generate GPR data.

[0062] S2, GPR data preprocessing and dataset construction:

[0063] For the GPR data obtained in S1, denoising preprocessing is first performed. The specific method is to scan the geological information without physical objects to obtain background GPR data, and then subtract them to obtain denoised GPR data. Next, use the denoised GPR data to draw the B-Scan imaging image to obtain one of the input data sets; then perform FK offset on the denoised GPR data, output the imaging after FK offset, and obtain the second part of the input data set. Finally, the VTI file obtained in the simulation process is input into the Paraview software for physical image visualization, and then denoising and color mapping are performed to obtain the target image data set. The images in the data set are divided into training sets and test sets according to a certain ratio.

[0064] S3, trained using a dual-input Unet based on an improved image separable module:

[0065] Use the improved network (DIWSUnet) for training, perform deep learning on the training set, select appropriate learning rate planning and measurement criteria, and set the loss function to BCEloss.

[0066] S4, select the optimal model and output the inversion image:

[0067] The optimal model is selected based on IoU, and the inverted physical image is output. The output physical object category is determined by analyzing the color of the physical object. In addition, the radius and location of the underground physical object can be directly determined based on the output physical image.

[0068] The improved dual-input Unet (DIWSUnet) based on the image separable module uses the Bscan image obtained by the ground penetrating radar GPR data and the FK migration image obtained by the FK migration as input, and inverts the physical object image. It is characterized by inputting the geological file into Paraview to obtain the physical image visualization, and then performing denoising and color mapping to obtain the target data set. The specific steps are as follows:

[0069] S21, denoising the target object image:

[0070] After using Paraview to obtain the preliminary physical image I, first use the background VTI file without the physical object to visualize the physical image using Paraview to obtain the background B, and then subtract the two IB to obtain the denoised physical geological image I1.

[0071] S22, color mapping is performed on the physical image:

[0072] The color of the physical object in I1 cannot be used directly as the target image, and the color must be mapped. Map the three different physical objects to three colors (0,255,255), (255,0,255), and (255,255,0), and set the background to white (255,255,255) to obtain I2. This makes it easier to use the Unet network for binary classification on the same channel.

[0073] The improved dual-input Unet based on the image separable module (DIWSUnet) uses the Bscan image obtained by the ground penetrating radar GPR data and the FK migration image obtained by the FK migration as input, and inverts the image of the real object. It is characterized by using the improved dual-input Unet based on the image separable module.

[0074] S31, combining depth-wise separable convolution and ordinary convolution:

[0075] Inspired by the deep separable convolutional network, we designed a separable convolution in the image dimension. The original deep separable convolution only performs convolution at the depth level, and then performs 1*1 convolution on the obtained feature map to enlarge the feature dimension to 64 dimensions. The original intention was to reduce the number of parameters used and the amount of calculation. Here, we combine deep separable convolution with ordinary convolution, only perform separable convolution at the image level, and then after convolution, BatchNorm and ReLU, connect the originally separated and extracted features in the channel dimension, and then enlarge them to 64 dimensions through 1*1 convolution.

[0076] S32, input channel transformation:

[0077] At the time of input, the original input channels are changed to 6 channels, and two 3-channel RGB images (B-Scan image and FK offset image) can be received at the same time, which are split into two three-channel images. Each of them passes through a convolution layer containing 32 filters, and each obtains a 32-dimensional feature matrix. After BatchNorm and ReLU, the two feature matrices are connected in the feature dimension and spliced ​​into 64 channels. Finally, they pass through 1*1 convolution of 64 filters to keep the feature dimension at 64 dimensions.

[0078] S33, the number of parameters is only slightly increased:

[0079] We assume that the input channel is C, the image is H×W, the number of feature maps in the first layer is M, and the convolution kernel size is K

[0080] Because the first layer has two convolutional layers and BatchNorm parameters, the original parameters of the first layer are:

[0081] Num param=(K×K×C×M+2×M)+(K×K×M×M+2×M)

[0082] Now we use the IWS module instead. Assuming that the size of the inner channel for each image is S, the parameter amount is:

[0083] Num′ param1 =K×K×C×S+4×S

[0084] Num′ param2 =2×S×M+2×M

[0085] Num′ param3 =K×K×M×M+2×M

[0086] Num′ param =Num′ param1 +Num′ param2 +Num′ param3

[0087] Only for the left input module of the first layer, substituting the actual parameters used, we can get Num param is 40576

[0088] For the IWS module, the number of parameters is 42944. For the small modules in the first layer, there are only 2368 more parameters. For the entire model, it is a symmetrical structure and the number of parameters below increases exponentially. The total number of parameters in the model is about 40 million, and the increase in parameters is less than one ten-thousandth.

[0089] The specific module structure diagram and DIWSUnet network structure diagram are as follows Figure 2 and Figure 3 Shown

[0090] S34, set the loss function to BCELoss:

[0091] BCELoss is often used as a loss function in traditional image segmentation or grayscale image inversion fields.

[0092]

[0093] Here, based on the original method, we first calculate the loss in the channel dimension and then find the average:

[0094]

[0095] The dual-input Unet (DIWSUnet) improved based on the image separable module uses the Bscan image obtained by ground penetrating radar GPR data and the FK migration image obtained by FK migration as input to invert the real object image. It is characterized by selecting the optimal model according to IoU and outputting the inverted image. The specific steps are as follows:

[0096] S41, binarize the predicted output image:

[0097] Because the prediction output is three channels, each channel is classified into two categories to determine whether it is a physical object or background. The first channel is the channel for determining whether it is a plastic object (0, 255, 255), the second channel is for determining whether it is air, and the third channel is for determining whether it is metal. Before inputting the image, the image has been divided by 255 to constrain it between 0 and 1. Therefore, before determining IoU, a threshold must be set to binarize the output prediction result. If the output value is less than the threshold, it is a physical object, and it is converted to 1, otherwise it is 0, thus converting it to a binary mask of 0 and 1.

[0098] S42, calculate the intersection-and-combination ratio of each channel:

[0099] For each channel of the binary mask, calculate the number of pixel elements with an output value of 1, and also calculate the number of pixel elements of the physical object in the target physical image, and you can get the intersection and union of the two.

[0100]

[0101] Then we average each channel to get the predicted IoU value.

[0102] The specific recognition result comparison chart is as follows Figure 4 shown.

[0103] The above description is only a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent change made based on the technical essence of the present invention still falls within the scope of protection required by the present invention.

Claims

1. A U-Net network construction method based on dual input and image separable convolution, characterized in that: The following steps are involved: S1, GPR data simulation: Use three different common physical objects including metal, plastic and air to generate GPRmax input in files, and then use GPRmax to simulate and generate GPR data; S2, GPR data preprocessing and dataset construction: For the GPR data obtained in S1, denoising preprocessing is first performed. Specifically, the geological information without physical objects is also scanned to obtain background GPR data, which is then subtracted to obtain denoised GPR data. Next, the denoised GPR data is used to draw a B-Scan imaging image to obtain one of the input data sets; the denoised GPR data is then FK-migrated, and the imaging after FK migration is output to obtain the second part of the input data set. Finally, the VTI file obtained in the simulation process is input into the Paraview software for physical image visualization, and then denoising and color mapping are performed to obtain the target image data set. The images in the data set are divided into training sets and test sets according to a certain ratio. S3, trained using a dual-input Unet based on an improved image separable module: Use the improved network for training, perform deep learning on the training set, select appropriate learning rate planning and measurement criteria, and set the loss function to BCEloss; S4, select the optimal model and output the inversion image: The optimal model is selected based on IoU, and the inverted physical image is output. The category of the output physical object is determined by analyzing the color of the physical object. In addition, the radius and location of the underground physical object can be directly determined based on the output physical image.

2. The U-Net network construction method based on dual input and image separable convolution according to claim 1, characterized in that: The specific steps of step S2 are as follows: S21, denoising the target object image: After using Paraview to obtain the preliminary physical image I, first use the background VTI file without the physical object to visualize the physical image using Paraview to obtain the background B, and then subtract the two IB to obtain the denoised physical geological image I1; S22, color mapping is performed on the physical image: At this time, the color of the physical object in I1 cannot be directly used as the target image. The colors need to be mapped. The three different physical objects are mapped to three colors (0, 255, 255), (255, 0, 255), and (255, 255, 0), and the background is set to white (255, 255, 255) to obtain I2. This makes it convenient to use the Unet network for binary classification on the same channel.

3. The U-Net network construction method based on dual-input and image separable convolution according to claim 1, characterized in that: The specific steps of step S3 are as follows: S31, combining depth-wise separable convolution and ordinary convolution: Inspired by the deep separable convolutional network, we designed a separable convolution in the image dimension. The original deep separable convolution only performs convolution at the depth level, and then performs a 1*1 convolution on the obtained feature map to enlarge the feature dimension to 64 dimensions. S32, input channel transformation: At the time of input, the original input channel is changed to 6 channels, and two 3-channel RGB images, namely the B-Scan image and the FK offset image, are received at the same time, and split into two three-channel images. Each of them passes through a convolution layer containing 32 filters, and each obtains a 32-dimensional feature matrix. After BatchNorm and ReLU, the two feature matrices are connected in the feature dimension and spliced ​​into 64 channels. Finally, they pass through 1*1 convolution of 64 filters to keep the feature dimension at 64 dimensions. S33, the number of parameters is only slightly increased: Assume that the input channel is C, the image is H×W, the number of feature maps in the first layer is M, and the convolution kernel size is K; Because the first layer has two convolutional layers and BatchNorm parameters, the original parameters of the first layer are: In a param =(K×K×C×M+2×M)+(K×K×M×M+2×M) Now we use the IWS module instead. Assuming that the size of the inner channel for each image is S, the parameter amount is: Num′ param1 =K×K×C×S+4×S Num′ param2 =2×S×M+2×M Num′ param3 =K×K×M×M+2×M Num′ param =Number param1 +Number param2 +Number param3 Only for the left input module of the first layer, substituting the actual parameters used, we can get Num param is 40576; For the IWS module, the number of parameters is 42944. For the small modules in the first layer, there are only 2368 more parameters. For the entire model, it is a symmetrical structure and the number of parameters below increases exponentially. The total number of parameters of the model is about 40 million, and the number of parameters increased is less than one ten-thousandth. S34, set the loss function to BCELoss: BCELoss is often used as a loss function in the traditional image segmentation field or grayscale image inversion field; On the basis of the original, first calculate the loss in the channel dimension, and then find the average:

4. The U-Net network construction method based on dual input and image separable convolution according to claim 1, characterized in that: The specific steps of step S4 are as follows: S41, binarize the predicted output image: Because the prediction output is three channels, each channel is a binary classification to determine whether it is a physical object or background. The first channel is the channel to determine whether it is a plastic object (0, 255, 255), the second channel is to determine whether it is air, and the third channel is to determine whether it is metal. Before inputting the image, the image has been divided by 255 to constrain it between 0 and 1. Therefore, before judging IoU, a threshold must be set to binarize the output prediction result. If the output value is less than the threshold, it is a physical object and it is converted to 1, otherwise it is 0, thus converting it to a binary mask of 0 and 1. S42, calculate the intersection-and-combination ratio of each channel: For each channel of the binary mask, calculate the number of pixel elements with an output value of 1, and also calculate the number of pixel elements of the physical object in the target physical image, and you can get the intersection and union of the two.

Images