Dual-input and image separable convolution based u-net network construction method

Abstract

Based on the dual input and image separable convolution U-Net network construction method, the dual input Unet (DIWSUnet) based on image separable convolution is improved to realize the imaging of ground penetrating radar (GPR) data and the inversion of underground real objects. First, the B-Scan scanning image is generated based on the GPR data, and then the GPR data is used for frequency-wavenumber shift (FK) processing to realize the dual input inversion of the B-Scan scanning image and the FK offset image to obtain the final real object image; The dual color input (6 channels) and color output (3 channels) are used to distinguish different real objects, and the underground position of the inverted real object is judged while the category of the object is judged; Based on the deep separable convolution model, the original input module is improved to an image separable convolution module IWS (Image Wise Seperable Convolution) module, which greatly improves the performance with only a small number of parameters.

Description

Technical Field

[0001] This invention belongs to the field of GPR inversion imaging, specifically a method for constructing a U-Net network based on dual inputs and image separable convolution. Background Technology

[0002] Ground-penetrating radar (GPR) uses high-frequency electromagnetic waves to detect underground structures, enabling real-time acquisition of high-resolution underground images. It is widely used in fields such as archaeology, geological exploration, environmental monitoring, and infrastructure assessment. With accelerating urbanization and aging infrastructure, the inspection and maintenance of underground facilities are becoming increasingly important.

[0003] Ground-penetrating radar (GPR) has wide applications in detecting subsurface objects and imaging subsurface structures in geophysical and civil engineering applications, helping us to better understand the structural properties of subsurface objects. In GPR inversion problems, the properties of subsurface scenes can be reconstructed from the electromagnetic (EM) information in the recorded GPR data.

[0004] Subsurface image inversion technology processes and analyzes data acquired through geotextile resonant imaging (GPR) to reconstruct the true features of subsurface objects. This technology helps researchers and engineers accurately identify subsurface obstacles, soil properties, and subsurface objects, providing a reliable basis for engineering design, construction, and safety assessments, particularly for the identification and detection of subsurface cavities. Furthermore, improving the inversion algorithm can enhance imaging accuracy, further promoting the application of GPR technology in complex environments.

[0005] In the field of GPR inversion imaging, there are traditional methods and deep learning-based methods. Among the traditional methods, several classic approaches can reconstruct subsurface structure images from acquired GPRB-scan images, including transfer algorithms, tomographic methods, and Full-Wave Inversion (FWI) techniques. In addition, there are some transfer algorithms, such as Reverse Time Migration (RTM) and Kirchhoff Migration, used to convert unfocused spatiotemporal GPR images into focused GPR images to show the location and size of objects. However, these methods do not accurately display the structural shape and other parameters of the original object and may also have some positional inaccuracies.

[0006] The FWI algorithm requires a forward modeling step in each iteration, which consumes a significant amount of computation. In summary, traditional methods have two main drawbacks: first, the positioning of structural parameters is not precise enough; second, the iterative construction and mapping of the dielectric constant map requires substantial computational resources; and third, in complex scenarios, the constructed dielectric constant mapping is not accurate enough and is affected by factors such as noise.

[0007] In this context, deep learning-based methods have emerged. Currently, many deep learning methods are used to model B-scan images obtained from GPR data, such as UNet. UNet is a deep learning model widely used in image semantic segmentation and image style transfer. Compared with traditional methods, deep learning methods offer significant efficiency and accuracy. Most of these methods are single-input, single-output: inputting a B-scan image and outputting the predicted location of the physical object structure. Most methods also remain single-channel, lacking color channel output to achieve multi-target, multi-class prediction and thus reconstruct the dielectric constant map. Furthermore, most simulated physical objects are relatively large. This invention simulates physical objects (radius 2-5 cm) much smaller than the overall scanned area (2.5 * 0.5 m), and achieves better results. Summary of the Invention

[0008] To address the above issues, this invention proposes a U-Net network construction method based on dual inputs and image-separable convolution. Firstly, this invention uses two inputs simultaneously: a B-scan image and an image processed by FK offset, to improve model accuracy. A color image is then used as input to predict a colored real-world object image. Based on the color mapping relationship from the original real-world object image to the target real-world object image obtained through Paraview modeling, the actual dielectric constant map is reconstructed, and the real-world object is accurately classified. Furthermore, the input to U-Net is improved, inspired by a depth-separable module, resulting in an image-separable convolution module. The IoU (Interval in Uranus) is used as an indicator to select the optimal model, yielding the output inversion image. This allows for comprehensive analysis to determine the actual position, radius, and category of the object.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] The method for constructing a U-Net network based on dual-input and image-separable convolution is characterized by the following steps:

[0011] S1, GPR data simulation:

[0012] Three different common physical objects, including metal, plastic and air, are used to generate GPRmax input files, and then GPRmax simulation is used to generate GPR data.

[0013] S2, GPR data preprocessing and dataset construction:

[0014] For the GPR data obtained in S1, denoising preprocessing is first performed. Specifically, geological information without physical objects is also scanned to obtain background GPR data. Subtraction is then performed to obtain denoised GPR data. Next, B-Scan imaging images are drawn using the denoised GPR data to obtain one of the input datasets. Then, FK offset is performed on the denoised GPR data to output the FK-offset image, which is the second part of the input dataset. Finally, the VTI file obtained during the simulation is input into Paraview software for physical image visualization. Denoising and color mapping are then performed to obtain the target image dataset. The images in the dataset are divided into training and test sets according to a certain ratio.

[0015] S3, trained using a dual-input Unet improved based on image separable modules:

[0016] The improved network is used for training, deep learning is performed on the training set, and an appropriate learning rate and metric are selected. The loss function is set to BCE loss.

[0017] S4, Select the optimal model and output the inversion image:

[0018] The system selects the optimal model based on IoU, outputs the inverted physical image, analyzes the color of the physical object to determine the category of the output physical object, and can directly determine the radius and location of underground physical objects based on the output physical image.

[0019] As a further improvement to the present invention, step S2 is specifically as follows:

[0020] S21, Denoise the target object image:

[0021] After obtaining the initial physical image I using Paraview, the background B is obtained by visualizing the physical image using Paraview with a background VTI file that does not contain physical objects. Then, the two images are subtracted by IB to obtain the denoised physical geological image I1.

[0022] S22, perform color mapping on the physical image:

[0023] The colors of the real objects in I1 obtained at this time cannot be directly used as the target image. The colors need to be mapped, and the three different real objects are mapped to three colors (0,255,255), (255,0,255), and (255,255,0), and the background is set to white (255,255,255) to obtain I2. This makes it convenient to use the Unet network for binary classification on the same channel.

[0024] As a further improvement to the present invention, step S3 is specifically as follows:

[0025] S31 combines depthwise separable convolution and regular convolution:

[0026] Inspired by depth-separated convolutional networks, we designed image-dimensional separable convolutions. The original depth-separable convolutions only perform convolutions at the depth level first, and then perform 1*1 convolutions on the resulting feature maps, thus enlarging the feature dimensions to 64 dimensions.

[0027] S32, Input Channel Conversion:

[0028] During input, the original input channels are changed to 6 channels. Two 3-channel RGB images, namely the B-Scan image and the FK offset image, are received simultaneously. They are split into two 3-channel images, each of which first passes through a convolutional layer containing 32 filters, resulting in a 32-dimensional feature matrix. After passing through BatchNorm and ReLU, the two feature matrices are concatenated along the feature dimension to form a 64-channel image. Finally, it passes through a 1*1 convolution with 64 filters to maintain the feature dimension at 64 dimensions.

[0029] S33, with only a slight increase in parameters:

[0030] Assume the input channel is C, the image is H×W, the number of feature maps in the first layer is M, and the convolution kernel size is K;

[0031] Because the first layer has two convolutional layers and parameters for BatchNorm, the original number of parameters for the first layer is:

[0032] Num param = (K×K×C×M+2×M)+(K×K×M×M+2×M)

[0033] Now, we'll use the IWS module instead. Assuming the inner channel size for each image is S, the number of parameters is:

[0034] Num′ param1 =K×K×C×S+4×S

[0035] Num′ param2 =2×S×M+2×M

[0036] Num′ param3 =K×K×M×M+2×M

[0037] Num′ param =Num′ param1 +Num′ param2 +Num′ param3

[0038] For the left input module of the first layer only, substituting the actual parameters used, we can obtain Num. param The value is 40576;

[0039] For the IWS module, the number of parameters is 42944. For the small modules in the first layer, there are only 2368 more parameters. For the entire model, it is a symmetrical structure and the number of parameters below grows exponentially. The total number of parameters in the model is about 40 million, while the increase in the number of parameters is less than one ten-thousandth.

[0040] S34, Set the loss function to BCELoss:

[0041] In the traditional field of image segmentation or grayscale image inversion, BCELoss is often used as the loss function;

[0042]

[0043] Based on the original calculation, first calculate the loss for each channel, and then calculate the mean:

[0044]

[0045] As a further improvement to the present invention, step S4 is specifically as follows:

[0046] S41, binarize the predicted output image:

[0047] Because the predicted output is three channels, each channel performs its own binary classification to determine whether it is a real object or the background. The first channel is for determining whether it is a plastic object (0, 255, 255), the second channel is for determining whether it is air, and the third channel is for determining whether it is metal. Before the input image, the image has been constrained to between 0 and 1 by dividing it by 255. Therefore, before determining the IoU, a threshold must be set to binarize the output prediction result. If the output value is less than the threshold, it is a real object and is converted to 1; otherwise, it is 0. This results in a binary mask of 0 and 1.

[0048] S42, calculate the cross-connection ratio (CTR) of each channel:

[0049] For each channel's binarized mask, the number of pixel elements with an output value of 1 is calculated, and the number of pixel elements of the actual object in the target image is also calculated. The intersection (INtersection) and union (Union) of these two sets can then be obtained.

[0050]

[0051] Beneficial effects:

[0052] This invention is based on a U-Net network construction method using dual inputs and image separable convolution. It utilizes a dual-input U-Net (DIWSUnet) improved with image separable convolution to invert underground objects from ground-peneturing radar (GPR) data imaging. First, a B-Scan image is generated from the GPR data obtained from the GPR. Then, frequency-wavenumber shift (FK) processing is performed on the GPR data to achieve dual-input inversion of the B-Scan image plus the FK-shifted image, resulting in the final image of the object. Dual color inputs (6 channels) and color outputs (3 channels) are used to distinguish different objects, determining the object's category while inverting its underground location. Inspired by depth-separable convolution models, the original input module is improved to an image-separable convolution module (IWS), significantly improving performance with only a few parameter increases. Attached Figure Description

[0053] Figure 1 This is the overall process for using an improved network to perform ground-penetrating radar GPR inversion imaging.

[0054] Figure 2 This is a structural diagram of an image separable convolutional module;

[0055] Figure 3 This is the corresponding DIWSUNIT network structure diagram;

[0056] Figure 4 This is a comparison chart of the recognition results from the inversion imaging. Detailed Implementation

[0057] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0058] The specific inversion and identification process of this invention is as follows: Figure 1 As shown.

[0059] The improved dual-input Unet (DIWSUnet) based on image separability modules uses a Bscan image obtained from GPR data and an FK offset image obtained from FK offset as inputs to invert and obtain an image of the real object. Specifically, it includes the following steps:

[0060] S1, GPR data simulation:

[0061] Three different common physical objects (metal, plastic, and air) are used to generate GPRmax input files, and then GPR data is generated using GPRmax simulation.

[0062] S2, GPR data preprocessing and dataset construction:

[0063] For the GPR data obtained in S1, denoising preprocessing is first performed. Specifically, geological information without physical objects is also scanned to obtain background GPR data, which is then subtracted to obtain denoised GPR data. Next, B-Scan imaging images are plotted using the denoised GPR data, resulting in one part of the input dataset. Then, the denoised GPR data undergoes FK shifting, outputting the FK-shifted image, which is the second part of the input dataset. Finally, the VTI file obtained during the simulation is input into Paraview software for physical image visualization, followed by denoising and color mapping to obtain the target image dataset. The images in the dataset are divided into training and test sets according to a certain ratio.

[0064] S3, trained using a dual-input Unet improved based on image separable modules:

[0065] The improved network (DIWSUnet) is used for training. Deep learning is performed on the training set, and an appropriate learning rate and metric are selected. The loss function is set to BCE loss.

[0066] S4, Select the optimal model and output the inversion image:

[0067] The optimal model is selected based on IoU, and the retrieved image of the object is output. The object's color is analyzed to determine its category. Furthermore, the radius and location of underground objects can be directly determined from the output image.

[0068] The dual-input Unet (DIWSUnet), based on an improved image separable module, uses Bscan images obtained from ground-penetrating radar (GPR) data and FK-shifted images obtained from FK migration as inputs to invert and obtain images of real objects. Its key feature is that after inputting geological files into Paraview to obtain visualized images of the real objects, denoising and color mapping are then performed to obtain the target dataset. The specific steps are as follows:

[0069] S21, Denoise the target object image:

[0070] After obtaining the initial physical image I using Paraview, the background B is obtained by visualizing the physical image using Paraview with a background VTI file that does not contain the physical object. Then, the two images are subtracted by IB to obtain the denoised physical geological image I1.

[0071] S22, perform color mapping on the physical image:

[0072] The object colors obtained in I1 cannot be directly used as the target image; color mapping is required. The three different object colors are mapped to (0, 255, 255), (255, 0, 255), and (255, 255, 0), respectively, and the background is set to white (255, 255, 255) to obtain I2. This facilitates binary classification on the same channel using the Unet network.

[0073] The dual-input Unet (DIWSUnet) based on the image separable module uses Bscan images obtained from ground-penetrating radar GPR data and FK offset images obtained from FK offset as inputs to invert images of real objects. Its key feature is the use of the dual-input Unet based on the image separable module.

[0074] S31 combines depthwise separable convolution and regular convolution:

[0075] Inspired by depthwise separable convolutional networks, we design image-dimensional separable convolutions. The original depthwise separable convolution performs convolution only at the depth level, then performs a 1x1 convolution on the resulting feature map, enlarging the feature dimension to 64 dimensions. The original intention was to reduce the number of parameters and computational cost. This approach combines depthwise separable convolution with ordinary convolution, performing separable convolution only at the image level. After convolution, BatchNorm, and ReLU, the previously separated features are concatenated along the channel dimension, and then enlarged to 64 dimensions again through a 1x1 convolution.

[0076] S32, Input Channel Conversion:

[0077] During input, the original input channels were changed to 6 channels, which can simultaneously receive two 3-channel RGB images (B-Scan image and FK offset image). These images are split into two 3-channel images, each first passing through a convolutional layer containing 32 filters to obtain a 32-dimensional feature matrix. After passing through BatchNorm and ReLU, the two feature matrices are concatenated along the feature dimension to form a 64-channel image. Finally, it passes through a 1*1 convolution with 64 filters to maintain the feature dimension at 64 dimensions.

[0078] S33, with only a slight increase in parameters:

[0079] We assume the input channels are C, the image is H×W, the number of feature maps in the first layer is M, and the convolution kernel size is K.

[0080] Because the first layer has two convolutional layers and parameters for BatchNorm, the original number of parameters for the first layer is:

[0081] Num param= (K×K×C×M+2×M)+(K×K×M×M+2×M)

[0082] Now, we'll use the IWS module instead. Assuming the inner channel size for each image is S, the number of parameters is:

[0083] Num′ param1 =K×K×C×S+4×S

[0084] Num′ param2 =2×S×M+2×M

[0085] Num′ param3 =K×K×M×M+2×M

[0086] Num′ param =Num′ param1 +Num′ param2 +Num′ param3

[0087] For the left input module of the first layer only, substituting the actual parameters used, we can obtain Num. param 40576

[0088] For the IWS module, the number of parameters is 42944. For the small modules in the first layer, only 2368 more parameters are added. For the entire model, it is a symmetrical structure with the number of parameters increasing exponentially below. The total number of parameters in the model is approximately 40 million, while the increase in the number of parameters is less than one ten-thousandth.

[0089] The specific module structure diagram and DIWSUNIT network structure diagram are as follows: Figure 2 and Figure 3 As shown

[0090] S34, Set the loss function to BCELoss:

[0091] In the traditional fields of image segmentation or grayscale image inversion, BCELoss is often used as the loss function.

[0092]

[0093] Here, we first calculate the loss along the channel dimension for each channel, and then calculate the mean:

[0094]

[0095] The improved dual-input Unet (DIWSUnet) based on image separability modules uses Bscan images obtained from ground-penetrating radar GPR data and FK-shifted images obtained from FK migration as inputs to invert images of real objects. Its key feature is selecting the optimal model based on IoU (Interval of Value) and outputting the inverted image. The specific steps are as follows:

[0096] S41, binarize the predicted output image:

[0097] Because the predicted output has three channels, each channel performs its own binary classification to determine whether it is a real object or background. The first channel determines whether it is a plastic object (0, 255, 255), the second channel determines whether it is air, and the third channel determines whether it is metal. Before inputting the image, the image has already been constrained to between 0 and 1 by dividing it by 255. Therefore, before determining the IoU, a threshold must first be set to binarize the output prediction result. If the output value is less than the threshold, it is a real object, and it is converted to 1; otherwise, it is 0, thus creating a binary mask of 0 and 1.

[0098] S42, calculate the cross-connection ratio (CTR) of each channel:

[0099] For each channel's binarized mask, the number of pixel elements with an output value of 1 is calculated, and the number of pixel elements of the actual object in the target image is also calculated. The intersection and union of these two values ​​can then be obtained.

[0100]

[0101] Next, the average value of each channel is calculated to obtain the predicted IoU value.

[0102] The specific recognition results comparison chart is as follows: Figure 4 As shown.

[0103] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any modifications or equivalent changes made based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A method for constructing a U-Net network based on dual-input and image-separable convolution, characterized in that, Includes the following steps: S1, GPR data simulation: Three different common physical objects, including metal, plastic and air, are used to generate GPRmax input files, and then GPRmax simulation is used to generate GPR data. S2, GPR data preprocessing and dataset construction: For the GPR data obtained in S1, denoising preprocessing is first performed. Specifically, geological information without physical objects is also scanned to obtain background GPR data. Subtraction is then performed to obtain denoised GPR data. Next, B-Scan imaging images are drawn using the denoised GPR data to obtain one of the input datasets. Then, FK offset is performed on the denoised GPR data to output the FK-offset image, which is the second part of the input dataset. Finally, the VTI file obtained during the simulation is input into Paraview software for physical image visualization. Denoising and color mapping are then performed to obtain the target image dataset. The images in the dataset are divided into training and test sets according to a certain ratio. S3, trained using a dual-input Unet improved based on image separable modules: The improved network is used for training, deep learning is performed on the training set, and an appropriate learning rate and metric are selected. The loss function is set to BCE loss. S4, Select the optimal model and output the inversion image: The system selects the optimal model based on IoU, outputs the inverted physical image, analyzes the color of the physical object to determine the category of the output physical object, and can directly determine the radius and location of underground physical objects based on the output physical image.

2. The method for constructing a U-Net network based on dual-input and image-separable convolution as described in claim 1, characterized in that, The specific steps of step S2 are as follows: S21, Denoise the target object image: After obtaining the initial physical image I using Paraview, the background B is obtained by visualizing the physical image using Paraview with a background VTI file that does not contain physical objects. Then, the two images are subtracted by IB to obtain the denoised physical geological image I1. S22, perform color mapping on the physical image: The colors of the real objects in I1 obtained at this time cannot be directly used as the target image. The colors need to be mapped, and the three different real objects are mapped to three colors (0,255,255), (255,0,255), and (255,255,0), and the background is set to white (255,255,255) to obtain I2. This makes it convenient to use the Unet network for binary classification on the same channel.

3. The method for constructing a U-Net network based on dual-input and image-separable convolution as described in claim 1, characterized in that, The specific steps of step S3 are as follows: S31 combines depthwise separable convolution and regular convolution: Inspired by depth-separated convolutional networks, we designed image-dimensional separable convolutions. The original depth-separable convolutions only perform convolutions at the depth level first, and then perform 1*1 convolutions on the resulting feature maps, thus enlarging the feature dimensions to 64 dimensions. S32, Input Channel Conversion: During input, the original input channels are changed to 6 channels. Two 3-channel RGB images, namely B-Scan image and FK offset image, are received simultaneously. They are split into two 3-channel images. Each of them first passes through a convolutional layer containing 32 filters to obtain a 32-dimensional feature matrix. After passing through BatchNorm and ReLU, the two feature matrices are concatenated along the feature dimension to form a 64-channel image. Finally, it passes through a 1*1 convolution with 64 filters to keep the feature dimension at 64 dimensions. S33, with only a slight increase in parameters: Assuming the input channel is C, and the image is... The first layer has M feature maps and K convolution kernels. Because the first layer has two convolutional layers and parameters for BatchNorm, the original number of parameters for the first layer is: ； Now we use the IWS module, i.e., the image separable convolution module, assuming that each image is divided into... Size is The parameter quantity is: ； ； ； ； For the left input module of the first layer only, substituting the actual parameters used, we get... The value is 40576; For the IWS module, the number of parameters is 42944. For the small modules in the first layer, there are only 2368 more parameters. For the entire model, it is a symmetrical structure and the number of parameters increases exponentially below. S34, Set the loss function as follows : It is often used in the fields of traditional image segmentation or grayscale image inversion. As a loss function; ； Based on the original calculation, first calculate the loss for each channel, and then calculate the mean: 。 4. The method for constructing a U-Net network based on dual-input and image-separable convolution as described in claim 1, characterized in that: The specific steps of step S4 are as follows: S41, binarize the predicted output image: Because the predicted output is three channels, each channel performs its own binary classification to determine whether it is a real object or the background. The first channel is for determining whether it is a plastic object (0, 255, 255), the second channel is for determining whether it is air, and the third channel is for determining whether it is metal. Before the input image, the image has been constrained to between 0 and 1 by dividing it by 255. Therefore, before determining the IoU, a threshold must be set to binarize the output prediction result. If the output value is less than the threshold, it is a real object and is converted to 1; otherwise, it is 0. Thus, it is converted into a binary mask of 0 and 1. S42, Calculate the cross-connection ratio (CTR) of each channel: For each channel's binarized mask, calculate the number of pixel elements with an output value of 1, and simultaneously calculate the number of pixel elements representing the actual object in the target image, obtaining the intersection of the two. Union ; 。

Images