Granular porous medium image three-dimensional reconstruction method based on depth generation model

Through the variational autoencoder and generative adversarial network based on the deep generation model, the complexity and accuracy of the three-dimensional image reconstruction of the granular porous medium is solved, and the rapid and accurate three-dimensional reconstruction is achieved, which reduces the acquisition cost and is in practice. It has great value in application.

CN119091030APending Publication Date: 2024-12-06SICHUAN UNIV +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310621709.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing three-dimensional image reconstruction method of particulate porous media is complex, time-consuming and expensive. The simplification of particle geometry by traditional methods affects the accuracy of reconstruction results, and the reconstruction time increases with the increase of particle number and complexity. Rapid growth.

Method used

Using a deep generation model-based method, a variational autoencoder and a generative adversarial network structure are designed. The two-dimensional image features are extracted through the encoder, and the generator generates a three-dimensional structure, and the reconstruction result is ensured through a multi-scale discriminator and a pattern distribution constraint function. Accuracy and diversity.

Benefits of technology

Fast and accurate three-dimensional reconstruction is achieved, the acquisition cost is reduced, and the model basis for physical characteristics research is provided in the fields of petroleum geology, materials, etc., which significantly improves the reconstruction speed.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure BDA0004255586190000033
    Figure BDA0004255586190000033
  • Figure BDA0004255586190000035
    Figure BDA0004255586190000035
  • Figure FDA0004255586170000013
    Figure FDA0004255586170000013
Patent Text Reader

Abstract

The invention discloses a granular porous medium image three-dimensional reconstruction method based on a depth generation model, and the method mainly comprises the following steps: carrying out the preprocessing of a binary granular porous medium CT sequence image, and constructing an experiment data set; designing a network structure for reconstructing a three-dimensional image from a two-dimensional image based on a variational auto-encoder and a generative adversarial network; designing a constraint function based on pattern distribution to constrain the morphological characteristics of the reconstructed image; designing an encoder loss function to extract feature information of the two-dimensional image; designing a GAN loss function to enable a generation result to be close to statistical and morphological characteristics of a real image; and carrying out network training based on the data set, the network and the loss function until a reconstruction result accords with a real expectation, obtaining an image three-dimensional reconstruction model, and completing three-dimensional reconstruction of the two-dimensional granular porous medium image based on the model. The method can accurately and quickly perform three-dimensional reconstruction on the two-dimensional image, and has important application value in the fields of petroleum geology and the like.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a three-dimensional reconstruction method, in particular to a three-dimensional reconstruction method of a granular porous medium image based on a depth generation model, and belongs to the technical field of three-dimensional image reconstruction. Background Art

[0002] Granular porous media are composed of particles of various shapes and sizes, and are of great significance to various fields of science and practical engineering, including rocks, soils, ceramics, alloy materials, etc. The morphological characteristics and spatial distribution of particles have an important influence on the macroscopic physical properties of the entire porous media system, such as porosity, permeability, elasticity, etc. Therefore, studying the microstructure of granular porous media and simulating the geometric morphology of real particle components has important theoretical and practical value.

[0003] Generally speaking, there are two methods for obtaining three-dimensional images of granular porous media: direct imaging and numerical simulation. Although the accuracy of imaging equipment has become increasingly higher with the advancement of science in recent years, obtaining three-dimensional images using this method is not only complicated and time-consuming, but also very expensive; and the study of the physical properties of porous media requires a large number of representative samples. Since the numerical simulation method can use real three-dimensional images as a reference to reconstruct samples that meet real expectations, the use of this method to reconstruct can avoid a large number of acquisition processes.

[0004] Regarding the 3D reconstruction of granular porous media images, the mainstream method is to treat the particle components as separate individuals for discrete modeling. However, these methods have certain disadvantages, such as simplifying the particle geometry, which affects the accuracy of the reconstruction results, and the reconstruction time increases rapidly with the increase in the number and complexity of the particles. Therefore, accurate and fast 3D reconstruction based on real images is still an issue that needs to be focused on and solved. Summary of the invention

[0005] The purpose of the present invention is to provide a three-dimensional reconstruction method of granular porous media images based on a generative adversarial network in order to solve the above problems.

[0006] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0007] A three-dimensional reconstruction method for granular porous media images based on a deep generative model comprises the following steps:

[0008] (1) Preprocess binary granular porous media CT sequence images to construct training and test datasets;

[0009] (2) Design a network structure for reconstructing 3D images from 2D images based on variational autoencoders and generative adversarial networks. The network structure includes an encoder E, a generator G, and a multi-scale discriminator D. D consists of three discriminators D1, D2, and D3 with the same structure.

[0010] (3) Design constraint functions based on pattern distribution The purpose is to constrain the morphological features of the reconstructed image;

[0011] (4) Design the loss function of the network encoder The purpose is to enable the encoder to extract the feature information of the two-dimensional image and map the two-dimensional image from the pixel space to the latent vector space to generate the corresponding feature vector;

[0012] (5) Design the GAN loss function to constrain the reconstructed image generated by the generator to be close to the statistical and morphological characteristics of the real 3D image;

[0013] (6) Based on the above-mentioned data set, network and loss function, the network is trained until the reconstruction result meets the actual expectation, and a model for three-dimensional reconstruction of the image is obtained. Based on the model, the three-dimensional reconstruction of the two-dimensional granular porous medium image is completed.

[0014] The basic principles of the above method are as follows:

[0015] Traditional generative adversarial networks reconstruct the target image from a noise vector that does not contain any information about the training image. The training is often unstable and the accuracy of the results needs to be improved. For granular porous media, since their two-dimensional images are easier to obtain in practice, it is expected that generative adversarial networks can be used to achieve fast end-to-end reconstruction of two-dimensional images into three-dimensional structures. Since variational autoencoders can effectively extract image feature information, the encoder in the variational autoencoder can be used to convert the two-dimensional image from pixel space to latent vector space, providing prior information for the generator. In order to address the diversity of reconstruction results, variables need to be introduced to ensure that the network model can generate different results. In order to ensure that the generated results can reproduce the real particle morphology, it is necessary to design a cost function that constrains the morphological features.

[0016] Specifically, in the above scheme, the network structure described in step (2) includes an encoder E, a generator G, and a multi-scale discriminator D composed of three discriminators D1, D2, and D3; the encoder E and the generator G constitute a variational autoencoder network, and the generator G and the multi-scale discriminator D constitute a generative adversarial network; the encoder E encodes the two-dimensional image x into a latent vector z enc , which is consistent with the Gaussian noise vector z that conforms to the standard normal distribution noise The connection is merged and input to the generator G; the generator G decodes the input vector to generate the predicted three-dimensional structure G(zenc ,z noise ); The three discriminators D1, D2, and D3 have the same network structure. The three-dimensional size of the original training image is n×n×n. The three discriminators are responsible for identifying the three-dimensional size of n×n×n, of the image.

[0017] In the above scheme, in step (3), in order to make the reconstructed three-dimensional structure closer to the morphological characteristics of the real three-dimensional structure, the constraint function based on the pattern distribution is designed. It is defined as:

[0018]

[0019] where ||·||2 is the 2-norm, y pattern represents the mode distribution of the target 3D image, (G(z enc ,z noise )) pattern Represents the pattern distribution of the generated three-dimensional image; the pattern distribution calculation process of the image is as follows: ① Use a convolution kernel with a fixed weight to scan the image point by point to obtain all the patterns, and each pattern is presented in the form of a decimal number after the convolution operation; ② Calculate the number of occurrences of each pattern and normalize it to 0-1 to obtain the pattern distribution of the image; The calculation of pattern distribution is mainly for binary images. Since the output of the neural network may have values ​​other than 0 and 255 grayscale values, in the process of calculating the pattern distribution, it is necessary to set a threshold to binarize the generated image, and then perform pattern extraction and subsequent calculations.

[0020] In the above scheme, the encoder loss function described in step (4) is It is defined as:

[0021]

[0022] in, It describes the pixel difference between the low-level two-dimensional image of the generated three-dimensional image and the input two-dimensional image. Describes the difference between the distribution of the encoder's output vector and the standard normal distribution, measured by Kullback-Leibler divergence, λ rec and λ kl Respectively represent losses and The weight of .

[0023] In the above scheme, the GAN loss function described in step (5) is specifically divided into the discriminator loss function And the generator loss function The definition is as follows:

[0024]

[0025]

[0026] Among them, y1 represents the original target 3D image with a size of n×n×n, y2 and y3 represent the 3D images obtained by downsampling y1 once and twice, respectively. The sizes of y2 and y3 are and Similarly, (G(z enc ,z noise ))1 represents the three-dimensional image of size n×n×n generated by the generator, (G(z enc ,z noise ))2 and (G(z enc ,z noise ))3 indicates that the corresponding sizes are and The downsampled three-dimensional image is a multi-scale discriminator D; the identification result output by the multi-scale discriminator D is a weighted average of the identification results of the three discriminators D1, D2, and D3 for the three-dimensional image of the corresponding size.

[0027] In the above scheme, the total loss function described in step (6) is It is calculated by the weighted sum of various losses and is defined as:

[0028]

[0029] Among them, λ gan and λ pattern Respectively represent losses and The weight of . BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A network structure diagram of a method for three-dimensional reconstruction of granular porous media images based on a deep generative model according to the present invention;

[0031] Figure 2 It is a network structure diagram of the encoder E in the present invention;

[0032] Figure 3 It is a network structure diagram of the generator G in the present invention;

[0033] Figure 4 is a network structure diagram of the multi-scale discriminator D in the present invention;

[0034] Figure 5 3D self-attention module in the present invention;

[0035] Figure 6 Visual comparison of 3D reconstruction results of granular porous media images;

[0036] Figure 7 Comparison of statistical parameters of 3D reconstruction results of granular porous media images;

[0037] Figure 8 Comparison of particle morphology parameters of 3D reconstruction results of granular porous media images; DETAILED DESCRIPTION

[0038] The present invention is further described below with specific embodiments and drawings, but the embodiments are only a specific and detailed description of the implementation method of the present invention and should not be understood as any limitation on the protection content of the present invention. The drawings and the embodiments described below are provided to enable the present invention to be more completely and accurately understood by those skilled in the art.

[0039] The three-dimensional reconstruction method of granular porous media images based on the deep generative model has the following specific operation steps:

[0040] (1) Considering the current computer processing speed and video memory, in the present invention, as an implementation example, the binary granular porous medium original CT sequence images are downsampled to meet the requirements of typical representative units (REV), and then the downsampled images are cropped with a certain step size to obtain 900 128×128×128 3D image samples, and 70% and 30% of the images are randomly selected as training and test data sets, respectively. The bottom 2D image of each 3D image is selected as the input of the network.

[0041] (2) Designing a network structure for reconstructing three-dimensional images from two-dimensional images based on variational autoencoders and generative adversarial networks. Figure 1 The network block diagram shown in the figure includes an encoder E, a generator G and a multi-scale discriminator D; the encoder E and the generator G constitute a variational self-encoder network, and the generator G and the multi-scale discriminator D constitute a generative adversarial network; the encoder E encodes the two-dimensional image x into a latent vector z enc , which is consistent with the Gaussian noise vector z that conforms to the standard normal distribution noise The connection is merged and input to the generator G; the generator G decodes the input vector to generate the predicted three-dimensional structure G(z enc ,z noise ); the discriminator D distinguishes the authenticity of the generated result and the target image, and the identification result is used as a loss function to constrain the generator to generate results that are more in line with the expected reality. The update process of the entire network is: ① fix E and G, update D; ② fix D, optimize E and G.

[0042] (3) Design constraint functions based on pattern distribution In the experiment, in order to balance the accuracy of reconstruction and the speed of calculation, a 3×3 convolution kernel is used as a template to extract the pattern information of the image. The weights of the convolution kernel are fixed, which are 256, 128, 64, 32, 16, 8, 4, 2, and 1. The frequency of each pattern is counted to obtain the pattern distribution of the corresponding image. Representing the difference between the pattern distribution of generated images and real images, as a loss, constrains the generator to learn the morphological features of real images.

[0043] (4) Design the loss function of the network encoder The encoder E designed by the present invention is as follows Figure 2 As shown, E maps the 1×128×128 2D input image to a 128×1×1×1 latent vector z through a series of 2D convolution downsampling operations. enc , loss function The calculation is the pixel difference between the underlying two-dimensional image of the generated three-dimensional image and the input two-dimensional image and the latent vector z enc The difference between the distribution of and the standard normal distribution, the weights λ of the two losses rec and λ kl The values ​​are set to 1 and 0.1 respectively. A two-dimensional self-attention module is added after the first two convolutional layers to capture both short-range and long-range dependencies and to more fully characterize and learn feature information.

[0044] (5) Design the GAN loss function. The generator G and multi-scale discriminator D designed by the present invention are respectively as follows: Figure 3 , 4 As shown in Figure 2, the generator uses 3D deconvolution for upsampling, and a 3D self-attention module is added after the first two deconvolution layers. Its structure is as follows: Figure 5 As shown; the input of the generator is the encoder output vector z enc and a Gaussian noise vector z with a size of 32×1×1×1 that conforms to a standard normal distribution noise The purpose of adding Gaussian noise vectors is to enhance the diversity of the generated results. The multi-scale discriminator D consists of three discriminators D1, D2, and D3 with the same structure, which are responsible for identifying images with three-dimensional sizes of 128×128×128, 64×64×64, and 32×32×32 respectively. The final discriminant result output by the multi-scale discriminator D is the weighted average of the judgment results of D1, D2, and D3. In order to improve the stability of training, the parameter matrix of each layer of convolution in the multi-scale discriminator D is spectrally normalized.

[0045] (6) Based on the above data set, network and loss function, the network is trained until the reconstruction result meets the actual expectation, and the model of three-dimensional reconstruction of the image is obtained. Based on the model, the three-dimensional reconstruction of the two-dimensional granular porous medium image is completed. and The weight λ gan and λ pattern Take 0.1 and 1×10 6 .

[0046] Several other important network structure parameters: the batch size is set to 4, the Adam optimizer is used, the learning rate is set to 0.0004, and the learning rate decay strategy is used to continuously iterate and update the network parameters.

[0047] Specifically, in order to verify the effectiveness of the method of the present invention, the present invention conducted relevant experiments.

[0048] like Figure 6 As shown in the figure, (a) represents an input 2D image, (b) represents the 3D image corresponding to the 2D image and the 40th, 80th and 120th layers of 2D sequence images, and (c) represents a 3D reconstruction result and the corresponding three 2D images. From the visual effect, it can be seen that the reconstructed 3D image can well reproduce the morphological and structural characteristics of the real 3D image. At the same time, it can be seen that the shape and distribution of the particle components are close to the real one, indicating that the algorithm has a good ability to characterize the microstructure of granular porous media.

[0049] In addition to visual comparison, quantitative parameters are also used to verify the reconstruction effect, including autocorrelation function (ACF), multipoint connectivity function (MPC) and several commonly used particle morphology parameters, namely, particle sphericity, equivalent spherical diameter, surface area and volume, such as Figure 7 and 8 As shown. In order to verify the stability of the method, the same two-dimensional input image was reconstructed 10 times, and the target image, the 10 reconstruction results and the average of the reconstruction results were compared. It can be seen that the parameter distributions of the reconstructed image match the parameter distributions of the real image well. Except for individual detail differences, the overall parameter distribution trend remains consistent, which shows the accuracy and stability of the method of the present invention.

[0050] In addition, the present invention only takes 0.13 seconds on an NVIDIA GeForce RTX2080Ti GPU for 3D reconstruction of a 128×128×128 image, which greatly improves the reconstruction speed compared to traditional methods that usually take several hours to complete reconstruction.

[0051] In summary, the present invention is an effective three-dimensional reconstruction method for granular porous medium images based on a deep generative model. The invention can serve and be applied in the fields of petroleum geology, materials, civil engineering, etc., can reduce the acquisition and production costs of granular porous medium images, and provide a model basis for the study of the physical properties of granular porous media. It has great value in practical applications such as oil and gas exploration and material design.

[0052] The above embodiments are only preferred implementation cases of the present invention and are not limitations of the technical solutions described in the present invention. Any technical solution that can be implemented on the basis of the above implementation cases without creative work should be deemed to fall within the protection scope of the present invention.

Claims

1. A 3D reconstruction method for granular porous media images based on a deep generative model, characterized in that The following steps are involved: (1) Preprocess binary granular porous media CT sequence images to construct training and test datasets; (2) Design a network structure for reconstructing 3D images from 2D images based on variational autoencoders and generative adversarial networks. The network structure includes an encoder E, a generator G, and a multi-scale discriminator D. D consists of three discriminators D1, D2, and D3 with the same structure. (3) Design constraint functions based on pattern distribution The purpose is to constrain the morphological features of the reconstructed image; (4) Design the loss function of the network encoder The purpose is to enable the encoder to extract the feature information of the two-dimensional image and map the two-dimensional image from the pixel space to the latent vector space to generate the corresponding feature vector; (5) Design the GAN loss function to constrain the reconstructed image generated by the generator to be close to the statistical and morphological characteristics of the real 3D image; (6) Based on the above-mentioned data set, network and loss function, the network is trained until the reconstruction result meets the actual expectation, and a model for three-dimensional reconstruction of the image is obtained. Based on the model, the three-dimensional reconstruction of the two-dimensional granular porous medium image is completed.

2. The method for three-dimensional reconstruction of granular porous media images based on a deep generative model according to claim 1, characterized in that The network structure described in step (2) includes an encoder E, a generator G, and a multi-scale discriminator D composed of three discriminators D1, D2, and D3; the encoder E and the generator G constitute a variational autoencoder network, and the generator G and the multi-scale discriminator D constitute a generative adversarial network; the encoder E encodes the two-dimensional image x into a latent vector z enc , which is consistent with the Gaussian noise vector z that conforms to the standard normal distribution noise Connect and merge, and input to generator G; The generator G decodes the input vector to generate the predicted three-dimensional structure G(z enc ,z noise ); The three discriminators D1, D2, and D3 have the same network structure. The three-dimensional size of the original training image is n×n×n. The three discriminators are responsible for identifying the three-dimensional size of n×n×n, of the image.

3. The method for three-dimensional reconstruction of granular porous media images based on a deep generative model according to claim 1, characterized in that In step (3), in order to make the reconstructed three-dimensional structure closer to the morphological characteristics of the real three-dimensional structure, a constraint function L based on the pattern distribution is designed. pattern , which is defined as: where ||·||2 is the 2-norm, y pattern represents the mode distribution of the target 3D image, (G(z enc ,z noise )) pattern Represents the pattern distribution of the generated three-dimensional image; the pattern distribution calculation process of the image is as follows: ① Use a convolution kernel with a fixed weight to scan the image point by point to obtain all the patterns, and each pattern is presented in the form of a decimal number after the convolution operation; ② Calculate the number of occurrences of each pattern and normalize it to 0-1 to obtain the pattern distribution of the image; The calculation of pattern distribution is mainly for binary images. Since the output of the neural network may have values ​​other than 0 and 255 grayscale values, in the process of calculating the pattern distribution, it is necessary to set a threshold to binarize the generated image, and then perform pattern extraction and subsequent calculations.

4. The method for three-dimensional reconstruction of granular porous media images based on a deep generative model according to claim 1, characterized in that The encoder loss function described in step (4) is It is defined as: in, It describes the pixel difference between the underlying two-dimensional image of the generated three-dimensional image and the input two-dimensional image. Describes the difference between the distribution of the encoder's output vector and the standard normal distribution, measured by Kullback-Leibler divergence, λ rec and λ kl Respectively represent losses and The weight of .

5. The method for three-dimensional reconstruction of granular porous media images based on a deep generative model according to claim 1, characterized in that The GAN loss function described in step (5) Specifically divided into the discriminator loss function And the generator loss function The definition is as follows: Among them, y1 represents the original target 3D image with a size of n×n×n, y2 and y3 represent the 3D images obtained by downsampling y1 once and twice, respectively. The sizes of y2 and y3 are and Similarly, (G(z enc ,z noise ))1 represents the three-dimensional image of size n×n×n generated by the generator, (G(z enc ,z noise ))2 and (G(z enc ,z noise ))3 indicates that the corresponding sizes are and The downsampled three-dimensional image is a multi-scale discriminator D; the identification result output by the multi-scale discriminator D is a weighted average of the identification results of the three discriminators D1, D2, and D3 for the three-dimensional image of the corresponding size.

6. The method for three-dimensional reconstruction of granular porous media images based on a deep generative model according to claim 1, characterized in that The total loss function described in step (6) It is calculated by the weighted sum of various losses and is defined as: Among them, λ gan and λ pattern Respectively represent losses and The weight of .