Core image super-resolution reconstruction method based on improved generative adversarial network

Abstract

The invention discloses a core image super-resolution reconstruction method based on an improved generative adversarial network. The method mainly comprises the following steps: constructing a core image data set, and performing down-sampling on the obtained data set to obtain a core LR image data set; replacing the residual block with a residual attention module in a generator of the SRGAN to obtain an improved generator network; adding a void space convolution pooling pyramid module into a discriminator of the SRGAN to obtain an improved discriminator network; taking the core HR and LR image training data set as the input of the improved SRGAN network structure model, and training the improved SRGAN network structure model; and finally verifying the trained network structure model by using the core HR and LR image verification data set to complete core image super-resolution reconstruction. The core image super-resolution reconstruction effect is good, and the core image super-resolution reconstruction method is effective.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a core image super-resolution reconstruction method based on an improved generative adversarial network. Background Art

[0002] Core images are valuable analytical data for oil and gas exploration. With the rapid development of industrial cameras and the continuous innovation of image processing algorithms, core image analysis has put forward higher requirements on the quality of core images. The current core image acquisition system generally uses area array cameras or linear array cameras to collect core fluorescence or white light images. In field acquisition tasks, sometimes it is necessary to transmit the collected core images in real time or when the storage device capacity is insufficient, the core images need to be compressed to reduce the amount of image data and reduce the image transmission time. However, after the core image is compressed, the image quality also decreases, which is not conducive to the subsequent core image analysis. At the same time, due to the lack of image details collected by some old equipment and low-resolution acquisition equipment, it is not conducive to the in-depth development of researchers' subsequent research work. Therefore, appropriate super-resolution reconstruction of the acquired core images will help researchers obtain richer and clearer high-resolution core images.

[0003] Super-resolution reconstruction is an important research direction in the field of image processing. It has been widely used in industrial manufacturing, computer vision, medical imaging and remote sensing image processing, and has important research significance for improving image quality. Super-resolution reconstruction of core images refers to the reconstruction of low-resolution core images into high-resolution core images through image processing technology, which can help geologists and petroleum engineers better understand the characteristics and properties of underground rocks.

[0004] In recent years, with the rapid development of deep learning technology, super-resolution reconstruction algorithms based on deep learning have been widely used. Dong et al. proposed a super-resolution reconstruction algorithm SRCNN based on deep convolutional neural network. Although it is better than traditional methods, it cannot effectively extract the global information of the image because the network used is shallow and its receptive field is small. Shi et al. proposed a network model ESPCN based on sub-pixel convolution layer. The algorithm uses the method of directly convolving low-resolution images to obtain high-resolution images, which reduces the computational complexity of SRCNN while improving the reconstruction effect. The research team of Seoul University in South Korea proposed the enhanced deep super-resolution network EDSR, which deleted unnecessary modules of the traditional network. The model is compact and improves the model effect. It not only has better performance than other methods at that time, but also greatly reduces the complexity of the model. Although the above reconstruction algorithm performs well in objective evaluation indicators such as PSNR and SSIM, the visual perception quality of its reconstructed image is not satisfactory. Based on the above background, Ledig et al. first proposed a super-resolution reconstruction algorithm SRGAN based on generative adversarial network. Its main idea is to generate high-quality super-resolution images through adversarial training.

[0005] SRGAN shows good reconstruction visual effects on natural images, but its reconstruction effect on core images is still not good enough. There are blurring and micro-shadow problems in the reconstruction of core texture details, too much detail loss, which is inconsistent with human subjective evaluation. At the same time, its image quality evaluation index is not high. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a core image super-resolution reconstruction method based on an improved generative adversarial network to solve the problems existing in the above-mentioned technology.

[0007] The present invention proposes a core image super-resolution reconstruction method based on an improved generative adversarial network, which mainly includes the following steps:

[0008] Step 1: Construct a high-resolution core image dataset, crop the original image to obtain a core HR image with a size of 1000×1000 pixels, and divide it into a training dataset and a validation dataset;

[0009] Step 2: Downsample the core HR image dataset obtained in step 1 by 4 times to obtain the core LR image dataset;

[0010] Step 3: Replace the original residual block with the residual attention (RAM) module in the generator module of SRGAN to obtain the improved generator network structure model;

[0011] Step 4: Add the Atrous Spatial Convolution Pooling Pyramid (ASPP) module to the discriminator module of SRGAN to obtain the improved discriminator network structure model;

[0012] Step 5: Use the core HR image training dataset obtained in step 1 and the core LR image training dataset obtained in step 2 as inputs of the improved generator network structure model;

[0013] Step 6: Use the core HR image and the core SR image generated in step 5 as inputs to the improved discriminator network structure model, and feed back the difference between the core HR image and the generated core SR image to the generator and the discriminator to update the generator and the discriminator;

[0014] Step 7: By adjusting the parameters to minimize the loss function, the model training is completed to obtain the core image super-resolution model. The core HR image verification dataset obtained in step 1 and the core LR image verification dataset obtained in step 2 are used to verify the trained network structure model to complete the core image super-resolution reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the specific implementation modes of the present invention, the drawings required for use in the description of the specific implementation modes will be briefly introduced below. Obviously, the drawings described below are some implementation modes of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 is the residual attention module;

[0017] Figure 2 It is a dilated spatial convolutional pooling pyramid module;

[0018] Figure 3 It is the improved generator network model proposed by the present invention;

[0019] Figure 4 It is the improved discriminator network model proposed by the present invention;

[0020] Figure 5 It is a control experiment effect diagram of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] As a specific implementation of the present invention, a core image super-resolution reconstruction method based on an improved generative adversarial network can be specifically divided into the following steps:

[0023] Step 1: Construct a high-resolution core image dataset, crop the original image to obtain a core HR image with a size of 1000×1000 pixels, and divide it into a training dataset and a validation dataset;

[0024] Step 2: Downsample the core HR image dataset obtained in step 1 by 4 times to obtain the core LR image dataset;

[0025] Step 3: Replace the original residual block with the residual attention (RAM) module in the generator module of SRGAN to obtain the improved generator network structure model;

[0026] Step 4: Add the Atrous Spatial Convolution Pooling Pyramid (ASPP) module to the discriminator module of SRGAN to obtain the improved discriminator network structure model;

[0027] Step 5: Use the core HR image training dataset obtained in step 1 and the core LR image training dataset obtained in step 2 as inputs of the improved generator network structure model;

[0028] Step 6: Use the core HR image and the core SR image generated in step 5 as inputs to the improved discriminator network structure model, and feed back the difference between the core HR image and the generated core SR image to the generator and the discriminator to update the generator and the discriminator;

[0029] Step 7: By adjusting the parameters to minimize the loss function, the model training is completed to obtain the core image super-resolution model. The core HR image verification dataset obtained in step 1 and the core LR image verification dataset obtained in step 2 are used to verify the trained network structure model to complete the core image super-resolution reconstruction.

[0030] Specifically, Figure 3 As shown in Figure 2, in step 3, in order to better extract the high-level semantic information of the core image, the original residual block in the SRGAN generator module is replaced with a residual attention (RAM) module. The network structure of the residual attention module is as follows: Figure 1 As shown;

[0031] Among them, the channel attention feature map and the spatial attention feature map are added together, then multiplied with the original feature map through a Sigmoid function, and finally combined with the residual network to obtain Figure 1 The residual attention module network structure shown is shown in the following formula:

[0032] F r =R r (F r-1 )=F r-1 +f FA (f trans (F r-1 ))

[0033]

[0034] S r-1 =pool var (X r-1 )

[0035]

[0036] where f trans (.) represents a function consisting of convolution, ReLU and convolution, f FA (.) indicates the fusion attention mechanism, and Respectively represent element addition and product, pool var (.) represents variance pooling, S r-1 ∈R 1×1×C represents the output statistics, X r-1 =f trans (F r-1 ), conv 3×3 ,depth(.) represents 3×3 depth convolution.

[0037] like Figure 4 As shown in Figure 2, in step 4, in order to obtain contextual information of different scales and improve the accuracy of the model, an atrous spatial convolutional pooling pyramid (ASPP) module is added to the discriminator. The network structure of the ASPP module is as follows: Figure 2 As shown;

[0038] Among them, the input feature map is sent to a standard convolution layer with a size of 1×1, three dilated convolutions with a kernel size of 3×3 and dilation rates of 2, 4, and 6 respectively, and a global average pooling layer. Then the feature maps of the dilated convolution layer and the global average pooling layer with different sampling rates are spliced, and finally a convolution layer with a convolution kernel size of 1×1 is used for convolution to generate the final output feature map.

[0039] In step 5, the core HR image training dataset obtained in step 1 and the core LR image training dataset obtained in step 2 are used as the input of the improved generator network structure model. All experiments are carried out in the system version Ubuntu18.04.1, the CPU is Intel Xeon CPU E5-2686 v4, the main frequency is 2.30GHz, the graphics card is NVDIAGeForce RTX3090, and the video memory is 24GB. The experimental environment is Python3.7, and the deep learning framework Pytorch1.8.0 is used to build the Improving-SRGAN network for training. The GPU acceleration software is CUDA 11.1 and CUDNN8.0.5. The Adam optimization algorithm is used to update the network parameters during training iterations, with an initial learning rate of 0.0001, a batch size of 32, 100 iterations, and a network magnification of 4 times.

[0040] In step 6, the core HR image and the core SR image generated in step 5 are used as inputs to the improved discriminator network structure model, and the difference between the core HR image and the generated core SR image is fed back to the generator and the discriminator to update the generator and the discriminator. In this process, the generator will continue to learn and try to make the generated SR image fit the distribution space of the HR image data, and the discriminator will try to distinguish the SR image from the HR image to promote the efficient learning of the generator. This GAN network model is used for training until the discriminator cannot distinguish the difference between the SR image and the HR image, at which point the generator can generate realistic high-resolution images.

[0041] In step 7, the model training is completed by adjusting the parameters to minimize the loss function, and the core image super-resolution model is obtained. The core HR image verification dataset obtained in step 1 and the core LR image verification dataset obtained in step 2 are used to verify the original SRGAN network structure model and the improved SRGAN network structure model, respectively, to obtain the corresponding core image super-resolution reconstruction map.

[0042] The present invention proposes a super-resolution reconstruction method for core images based on an improved generative adversarial network. The residual block in the original generator model is replaced by the improved residual attention module (RAM) in this paper, and the input feature map is weighted in the channel domain and spatial domain to better extract the high-level semantic information of the image. The atrous spatial pyramid pooling module (ASPP) is embedded in the discriminator to enhance the network's perception of the multi-scale information of core image features, thereby more accurately evaluating the quality of the images generated by the generator and effectively improving the reconstructed visual effect of the core image. Figure 5The visual effect diagram of the SRGAN method and the present invention on the reconstruction of the core image is given. As shown in the figure, the left part is a high-resolution image, and the four small pictures on the right are: the upper left is the core LR image, the upper right is the original SRGAN network structure model effect diagram, the lower right is the improved SRGAN network structure model effect diagram, and the lower left is the core HR image; Figure 5 It can be seen that the visual perception effect of the core image reconstructed using the improved SRGAN network structure model is the best.

[0043] The present invention uses the core HR image verification data set obtained in step 1 and the core SR image obtained in step 7 to evaluate the image quality indexes in turn, and compares them with the SRGAN method. The experimental results are shown in Table 1. It can be seen that the present invention has improved various indicators of core image reconstruction compared with the SRGAN method. Among them, the average opinion score has been significantly improved, because the present invention has increased the ability to extract high-level semantic information of core images and the ability to perceive multi-scale information of core image features;

[0044] The image quality evaluation indicators are two objective evaluation indicators, namely, peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), and a subjective evaluation indicator, namely, mean opinion score (MOS) test.

[0045] Table 1 Comparison results of evaluation indicators of core image dataset

[0046]

[0047] In summary, compared with the SRGAN method, the reconstruction effect of the present invention on core images has obvious advantages in both subjective and objective evaluation. Therefore, the present invention is an effective super-resolution reconstruction method for core images.

[0048] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A core image super-resolution reconstruction method based on an improved generative adversarial network, characterized in that The following steps are involved: Step 1: Construct a high-resolution core image dataset, crop the original image to obtain a core HR image with a size of 1000×1000 pixels, and divide it into a training dataset and a validation dataset; Step 2: Downsample the core HR image dataset obtained in step 1 by 4 times to obtain the core LR image dataset; Step 3: Replace the original residual block with the residual attention (RAM) module in the generator module of SRGAN to obtain the improved generator network structure model; Step 4: Add the Atrous Spatial Convolution Pooling Pyramid (ASPP) module to the discriminator module of SRGAN to obtain the improved discriminator network structure model; Step 5: Use the core HR image training dataset obtained in step 1 and the core LR image training dataset obtained in step 2 as inputs of the improved generator network structure model; Step 6: Use the core HR image and the core SR image generated in step 5 as inputs to the improved discriminator network structure model, and feed back the difference between the core HR image and the generated core SR image to the generator and the discriminator to update the generator and the discriminator; Step 7: By adjusting the parameters to minimize the loss function, the model training is completed to obtain the core image super-resolution model. The core HR image verification dataset obtained in step 1 and the core LR image verification dataset obtained in step 2 are used to verify the trained network structure model to complete the core image super-resolution reconstruction.

2. The core image super-resolution reconstruction method based on an improved generative adversarial network according to claim 1 is characterized in that In step 2, the bicubic kernel function in PyTorch is used to downsample the core HR image dataset by 4 times.

3. The core image super-resolution reconstruction method based on an improved generative adversarial network according to claim 1 is characterized in that In step 3, the introduced residual attention (RAM) module performs channel and spatial domain weighting on the input feature map to better extract high-level semantic information of the core image.

4. The core image super-resolution reconstruction method based on an improved generative adversarial network according to claim 1 is characterized in that In step 4, the introduced Atrous Spatial Convolutional Pooling Pyramid (ASPP) module improves the network's perception of multi-scale information of core image features through convolution and pooling operations at different sampling rates, thereby more accurately evaluating the quality of images generated by the generator.

5. The core image super-resolution reconstruction method based on improved generative adversarial network according to claim 1 is characterized in that: In step 3 and step 4, the improved generator module and discriminator module; The structure of the improved generator module is: 1 convolution layer, 1 activation layer, 16 residual attention (RAM) modules with the same parameters, 1 convolution layer, 2 sub-pixel convolution layers and 1 convolution layer are connected in sequence; Among them, each residual attention (RAM) module with the same parameters includes 2 convolutional layers, 1 channel attention module, 1 spatial attention module, 1 attention fusion module, 1 attention weighting module and 1 residual connection module; Each sub-pixel convolution layer consists of 1 convolution layer, 1 2x upsampling sub-pixel layer, and 1 activation layer; The structure of the improved discriminator module is: 1 convolution layer, 1 activation layer, 6 combination layers, 1 atrous spatial convolution pooling pyramid (ASPP) module, 1 fully connected layer, 1 activation layer, 1 fully connected layer and 1 sigmoid activation function are connected in sequence; Among them, the atrous spatial convolutional pooling pyramid (ASPP) module includes 1 convolutional layer, 3 atrous convolutional layers with different atrous rates, 1 global average pooling layer and 1 convolutional layer; Each combined layer consists of 1 convolutional layer, 1 batch normalization layer, and 1 activation layer.

6. The core image super-resolution reconstruction method based on improved generative adversarial network according to claim 1 is characterized in that: The dilated ratios of the dilated convolutions in the dilated convolution layers are set to 2, 4, and 6, respectively, to ensure that the performance of the model is maximized.

Images