A True-Color Processing Method for Rock CT Digital Images Based on Deep Networks

Abstract

This invention relates to a method for true-color conversion of rock CT digital images based on deep networks, comprising the following steps: establishing a database of specimen CT images and specimen color images; preprocessing the specimen CT images and color images to expand the image database; constructing a first generator for generative adversarial networks (GANs) from specimen CT images to specimen-like color images; constructing a second generator for generative adversarial networks (GANs) from specimen color images to specimen-like CT images; constructing a discriminator in the specimen color image domain and a discriminator in the specimen CT image domain; establishing a loss function of CycleGAN to optimize the two generators; after training and obtaining a two-layer recurrent generative adversarial network model, using the first generator to perform color conversion on the CT images.

Description

Technical Field

[0001] This invention designs a method for true-coloring three-dimensional digital images of rock samples using microfocus CT, which belongs to the field of image processing based on recurrent generative adversarial networks. Background Technology

[0002] For three-dimensional specimens, microcomputed tomography (CT) technology can easily obtain CT images of each layer of the specimen and reconstructed pseudo-color specimens. However, CT and pseudo-color images cannot directly reflect information such as the structure, material composition and distribution of the specimen. In order to conduct more intuitive research on the interior of the rock sample, it is necessary to obtain true-color images of the specimen. Currently, this requires the use of edge-cutting destructive testing, but this method is complex and not widely applicable. Therefore, it is considered to use deep learning to train a CT image colorization model to transfer the color information of the real image to the CT image, thereby solving the problem of difficulty in obtaining true-color images of the specimen.

[0003] Currently, image colorization techniques based on deep learning are transitioning from deep convolutional neural networks (DCNNs) to generative adversarial networks (GANs). DCNN-based image colorization techniques are prone to losing image information and miscoloring during the process of extracting image features and generating images. GANs address these issues by improving the loss function. A search revealed a method for colorizing grayscale images based on generative adversarial networks (GANs) with patent number 202210252286.7. This method achieves colorization of grayscale images by constructing a GAN architecture and alternately training the generator and discriminator, reducing manual intervention and improving the drawback of large-size images where pixels cannot be finely colored. However, it only uses generative adversarial loss during training, which cannot guarantee that the generated color image has the same tone as the real image, nor does it consider the problem of difficulty in obtaining paired images. Another method, a method for converting thermal infrared images to visible light color images based on CycleGAN, uses cyclic consistency loss to connect the generators of thermal infrared images and visible light color images, constructing a cyclic generative adversarial neural network. This method achieves image colorization and solves the problem of difficult image pairing, but it does not consider the label mapping loss and cannot guarantee that the generated color image has the same tone as the normal color image. Most image colorization patents based on deep learning networks do not consider the impact of receptive field size on computational efficiency, nor do they provide detailed explanations of the preprocessing process before image training. In summary, current common image colorization methods are prone to problems such as loss of image information and miscoloring, do not take into account the difficulty of collecting paired images, and most deep networks do not provide much information on the size of the input and output images, thus failing to guarantee computational efficiency. Summary of the Invention

[0004] The purpose of this invention is to provide a method for generating true-color three-dimensional digital images of rock samples using microfocus CT scans, which is computationally efficient and capable of generating higher-resolution images. This invention uses a digital automatic slicing digital radiography system to scan a rock sample. First, a non-destructive test of the microfocus CT image is performed, followed by a destructive slicing test, constructing an original digital-CT image dataset. The resolution of the dataset is unified, and unstable boundary pixels are cropped. Then, cyclic boundary processing is performed on the images. Next, the window size and multiple sets of window parameters are specified, and the digital-CT images are cut and sampled according to the window to expand the image database. Then, flipping, translation, and contrast adjustment are used to further expand the image database. Based on the image domain transformation information of the sample CT images and color images in the image database, the network is trained using generative adversarial loss, cyclic consistency loss, and label mapping loss to obtain a cyclic generative adversarial network model that meets preset convergence conditions. This model includes: a first generator for generating sample-type color images, a discriminator for judging the sample-type color images generated by the first generator, a second generator for generating sample-type CT images, and a discriminator for judging the sample-type CT images generated by the second generator. Under the condition that the two-layer recurrent generative adversarial network model meets the preset convergence conditions, the first generator is used to colorize the CT images. This invention solves the problem of true colorization of CT images by learning image domain transformation information from CT images and sample color images in an image database. It can truly colorize micro-focus CT three-dimensional digital images, facilitating subsequent intuitive analysis of the internal structure and material composition of the sample, as well as numerical simulation modeling. The technical solution of this invention is as follows.

[0005] A method for true-color processing of rock CT digital images based on deep networks includes the following steps:

[0006] Step 1: Establish a database of CT images and color images of the specimen: The rock specimen is scanned by a digital automatic slicing digital photography system. The micro-focus CT three-dimensional digital image is obtained through non-destructive testing of micro-focus CT images; the original digital image is obtained through destructive slicing testing; and a database of CT images and color images of the specimen is established.

[0007] Step 2: Preprocess the CT and color images of the sample to expand the image database. The method is as follows:

[0008] Step 2.1: Standardize the resolution of CT images and color images of the sample;

[0009] Step 2.2: Cropping the boundaries of the CT and color images of the sample;

[0010] Step 2.3: Perform loop boundary processing on the cropped image;

[0011] Step 2.4: Expand the image database and divide the image database into a training image database and a test image database according to a preset ratio. The method for expanding the image database is as follows: Assume the size of the sample CT image is [m, n], and the size of the color image is [n1, m1]. Randomly give multiple groups of window parameters to expand the image database. For a randomly given group of window parameters, the processing method is as follows: Generate a window [w1, w2] of a predetermined size at a random position in the image, satisfying [w1 < n, w2 < m], inside the image before processing the loop boundary. Randomly give the window an angle thelt and an offset: [offx, offy], where the random value range of offx is [0, n - w1], and the random value range of offy is [0, m - w2]. The parameters of this window are: [offx, offy, thelt]; According to the window parameters, with the window center coordinates as the reference, offset and rotate the window to obtain the CT image corresponding to the window, and obtain the paired color image according to the same window parameters;

[0012] Step Three: According to the image domain conversion information of the sample CT image and the sample color image in the image database, construct a first generator for the generative adversarial network from the sample CT image to the sample-like color image; construct a second generator for the generative adversarial network from the sample color image to the sample-like CT image;

[0013] Step Four: Construct a discriminator for the sample color image domain, and use the discriminator to distinguish the difference in the detailed structure between the sample-like color image generated by the first generator and the detailed structure of the normal sample color image; construct a discriminator for the sample CT image domain, and use the discriminator to distinguish the difference in the detailed structure between the sample-like CT image generated by the second generator and the detailed structure of the normal sample CT image;

[0014] The discriminator for the sample color image domain and the discriminator for the sample CT image domain are based on the downsampling convolutional neural network. Based on the PatchGAN discriminator, the input image is downsampled multiple times to obtain images of different local areas, and the images are separately sent into the PatchGAN for calculation. The calculation results are weighted and added by the results obtained through upsampling as the final output of the discriminator, so as to fuse the local image features and the overall image characteristics;

[0015] Step Five: Use the first generator for converting the sample CT image to the sample-like color image and the discriminator for the sample color image domain to construct a generative adversarial loss Use the second generator for converting the sample color image to the sample-like CT image and the discriminator for the sample CT image domain to construct a generative adversarial loss The above and together constitute the generative adversarial loss L ganThe input is a CT image of the sample. A first generator generates a color image of the sample. Then, a second generator restores the generated color image of the sample to a CT image of the sample. This image is then compared with the real CT image to form a cycle consistency loss. Input a color image of the sample, use a second generator to generate a sample-like CT image, then use a first generator to restore the generated sample-like CT image to a sample-like color image, and compare it with the real color image to form a cycle consistency loss. and Together they constitute the cycle consistency loss L cyc Input a sample CT image, use a second generator to generate a sample-class CT image, and compare it with the input sample CT image to construct an identifier mapping loss. Input a color image of the sample, generate a color image of the sample class using the first generator, and compare it with the input color image of the sample to form the label mapping loss. and Together they constitute the identifier mapping loss L ide The above L gan L cyc L ide Together they constitute the loss function of CycleGAN, which is used to optimize the two generators;

[0016] Step Six: The CycleGAN is composed of the first and second generators built in Step Three, the discriminator in the color image domain and the discriminator in the CT image domain of the sample built in Step Four, and the generative adversarial loss, cycle consistency loss, and label mapping loss constructed in Step Five. Based on the image domain transformation information of the sample CT image and sample color image in the image database obtained in Step One and Step Two, the CycleGAN is used to connect the two generators of sample CT image to sample color image and sample color image to sample CT image. Adversarial training is carried out through continuous iteration. When the epoch value reaches the predetermined value, the training stops. After obtaining the two-layer CycleGAN model, the first generator is used to colorize the CT image.

[0017] Furthermore, the bilinear interpolation algorithm is used to scale the CT images and color images to unify their resolution.

[0018] Furthermore, the image database is expanded using processing methods including flipping, translating, and adjusting contrast.

[0019] Furthermore, the method for step three is as follows:

[0020] Step 3.1: Based on the image domain transformation information of the sample CT image and sample color image in the image database, construct the first generator for the generative adversarial network for converting the sample CT image to the sample-like color image;

[0021] Based on the image domain transformation information of the sample CT images and sample color images in the image database, a decoder structure, a convolutional residual structure, and an encoder structure are designed in the first generator network. The decoder structure is used for downsampling to compress image features and employs a convolutional module, IN regularization, and Leaky ReLU activation function. The convolutional residual structure is used for feature extraction, connecting the decoder structure and the encoder structure, and restoring and enhancing the data. It also employs a convolutional module, IN regularization, and ReLU activation function. The encoder structure is used for upsampling to restore the image and employs a deconvolutional module, IN regularization, and ReLU activation function.

[0022] Step 3.2: Based on the image domain transformation information of the sample CT image and sample color image in the image database, construct a second generator for the generative adversarial network for converting the sample color image to the sample-like CT image;

[0023] Based on the image domain transformation information of the sample CT images and sample color images in the image database, a decoder structure, a convolutional residual structure, and an encoder structure are designed in the second generator network. The decoder structure is used for downsampling to compress image features and employs a convolutional module, IN regularization, and the Leaky ReLU activation function. The convolutional residual structure is used for feature extraction, connecting the decoder structure and the encoder structure, and restoring and enhancing the data. It also employs a convolutional module, IN regularization, and the ReLU activation function. The encoder structure is used for upsampling to restore the image and employs a deconvolutional module, IN regularization, and the ReLU activation function.

[0024] Step 3.3: The padding of all convolutional layers in the constructed first and second generator networks is done using ReflectionPad2d to make the image symmetrical along the edges in all directions to increase the image resolution.

[0025] Furthermore, the method for processing the loop boundary of the cropped image is as follows: using the reflection method, the pixels are symmetrically arranged with the outermost pixel as the axis, and the image periphery is extended to avoid boundary exceedance caused by rotation when expanding the image.

[0026] The beneficial effects of this invention are:

[0027] 1. Based on the learning of image domain conversion information of rock CT images and rock color images in the image database, this invention realizes true colorization of micro-focus CT three-dimensional digital images, solves the problem of true colorization of CT images, and can generate three-dimensional samples that contain both CT image structural information and true image color information, which facilitates subsequent intuitive analysis of the internal structure and material composition of rock samples and numerical simulation modeling.

[0028] 2. During the construction of the image database, bilinear interpolation was used to unify the resolution of the original digital-CT images, and unstable pixels at the edges of the original digital-CT images were cropped to ensure the stability of the model. Before expanding the image dataset, the images were processed for cyclic boundaries to avoid boundary exceedance caused by rotation during the expansion of the image database. When expanding the dataset, a method was adopted to generate windows of a predetermined size at random positions in the image, and to randomly give the window offset distance and rotation angle. The image was then expanded by cutting it multiple times according to the window. This method controlled the size of the input image to the model and ensured computational efficiency.

[0029] 3. The padding of all convolutional layers in the constructed first and second generator networks uses ReflectionPad2d to make the image symmetrical along the edges in all directions, thereby increasing the image resolution.

[0030] 4. In order to better focus on the structural details of rock images, the PatchGAN discriminator is used, which can integrate local image features and global image characteristics. This enables the extraction and representation of local image features, which is beneficial for generating higher resolution images.

[0031] 5. The loss function of CycleGAN is composed of generative adversarial loss, cycle consistency loss, and label mapping loss. This makes the generated images as similar as possible to real images, while solving the problem of the difficulty in obtaining paired datasets of experimental CT images and experimental color images. It eliminates the need for high-precision pairing of experimental CT images and experimental color images, thus achieving the goal of unsupervised learning from sample CT images to color-like images. At the same time, it preserves the consistency of the input and output color composition, so that the hue does not change. Attached Figure Description

[0032] Figure 1 This is the overall flowchart;

[0033] Figure 2 This is a flowchart of converting CT images of samples into color images based on CycleGAN.

[0034] Figure 3 This is a diagram of the generator network structure of CycleGAN in this invention;

[0035] Figure 4 This is a diagram of the discriminator network structure of CycleGAN in this invention;

[0036] Figure 5 The results of this invention are shown in the figure, where (a) is a color image of the real sample, (b) is a CT image of the sample, (c) is a color image of the sample, and (d) is a three-dimensional reconstruction of the sample. Detailed Implementation

[0037] The specific embodiments of this invention will be described in further detail below with reference to the accompanying drawings and examples.

[0038] This invention considers using deep learning to train a CT image colorization model, transferring color information from real images to CT images, thus solving the problem of obtaining true-color images of specimens. During the acquisition of the training set, the process is complex because the real images of each layer of the specimen need to be machine-cut before acquisition, making it difficult to obtain paired datasets. This invention uses a CycleGAN (Recurrent Adversarial Network) model to solve the problem of true colorization in CT images. It can truly colorize micro-focus CT 3D digital images, that is, generate images that include both CT image structural information and real image color information through a generative recurrent adversarial neural network. This facilitates intuitive analysis of the internal structure and material composition of the specimen, as well as numerical simulation modeling.

[0039] According to the embodiments of the present invention, see Figure 1-4 As shown in this embodiment, a method for true-color processing of rock CT digital images based on deep networks is disclosed, which includes the following steps:

[0040] Step 1: Acquire internal CT and true color images of the three-dimensional sample. A rock sample is scanned layer by layer according to its size and shape using a digital automatic slicing digital photography system. First, a non-destructive test of the micro-focus CT image is performed, followed by a destructive test of the slicing. A database of sample CT and color images is established, and the dataset is divided into domain A and domain B, where domain A is the sample CT image and domain B is the sample color image.

[0041] Step 2: Unify the resolution of the CT and color images of the sample; crop the boundaries of the CT and color images to avoid unstable surrounding pixels; perform cyclic boundary processing on the cropped images to prevent boundary overflow caused by rotation when expanding the image database; to control the input image size and ensure computational efficiency, generate windows of a predetermined size at random positions in the image, and randomly assign window offset distance and rotation angle. Cut the image multiple times according to the window to expand the image database. Then, use flipping, translation, and contrast adjustment to further expand the image database, and divide the image database into training and testing databases.

[0042] Step 2.1: Unify the resolutions of the sample CT images and color images;

[0043] Regarding the problem of inconsistent resolutions between the sample CT images and color images, use the bilinear interpolation algorithm. Call the resize function through Pycharm and select the Image.ANTIALIAS parameter to scale the CT images and color images, and unify the resolutions of the CT images and color images;

[0044] Step 2.2: Crop the boundaries of the sample CT images and color images;

[0045] Use the Pycharm compiler to execute the cropping program to crop the unstable pixels at a certain distance from the upper, lower, left, and right boundaries of the CT and color images, avoiding the influence of unstable surrounding pixels on the model;

[0046] Step 2.3: Perform cyclic boundary processing on the cropped images;

[0047] Use the reflection method. Call the cv2 library through Pycharm, use the copyMakeBorder function and select the cv2.BORDER_REFLECT_101 parameter to symmetrically extend the pixels with the outermost pixels as the axis for the image, and perform extension processing on the periphery of the image to avoid boundary exceeding caused by rotation when expanding the image;

[0048] Step 2.4: To control the size of the input image and ensure computational efficiency, use Python + Opencv to generate a window [w1, w2] of a predetermined size at a random position in the image, satisfying [w1 < n, w2 < m], inside the image before cyclic boundary processing;

[0049] Randomly assign an angle thelt and an offset: [offx, offy] to the window. The random value range of offx is: [0, n - w1], and the random value range of offy is [0, m - w2]. The parameters of this window are: [offx, offy, thelt].

[0050] Based on the window parameters, the window is offset and rotated using the center of the window as a reference. Then, Python + OpenCV is used to identify and cut the window, and CT images of the corresponding windows are obtained through cutting. Paired color images are also obtained according to the same window parameters. In order to increase the sample capacity of the image database and prevent overfitting in the training of the neural network, multiple sets of window parameters are randomly given, and the images are cut according to the windows to expand the image database. Then, PyCharm calls the PIL and skimage libraries to implement flipping, translation, and contrast adjustment to further expand the image database. Finally, the image database is divided into a training image database and a test image database in an 8:2 ratio.

[0051] Step 3: Based on the image domain transformation information of the sample CT image and sample color image in the image database, construct a first generator for the generative adversarial network (GAN) from the sample CT image to the sample-like color image; construct a second generator for the GAN from the sample color image to the sample-like CT image.

[0052] Step 3.1: Based on the image domain transformation information of the sample CT image and sample color image in the image database, construct the first generator for the generative adversarial network for converting the sample CT image to the sample-like color image;

[0053] Based on the image domain transformation information of the sample CT images and sample color images in the image database, a decoder structure, a convolutional residual structure, and an encoder structure are designed in the first generator network. The decoder structure is used for downsampling to compress image features and employs a convolutional module, IN regularization, and Leaky ReLU activation function. The convolutional residual structure is used for feature extraction to connect the decoder structure and the encoder structure and to restore and enhance the data. It also employs a convolutional module, IN regularization, and ReLU activation function. The encoder structure is used for upsampling to restore the image and employs a deconvolutional module, IN regularization, and ReLU activation function.

[0054] Step 3.2: Based on the image domain transformation information of the sample CT images and rock color images in the image database, construct a second generator for the generative adversarial network used to convert the sample color images to sample-like CT images;

[0055] Based on the image domain transformation information of the sample CT images and sample color images in the image database, a decoder structure, a convolutional residual structure, and an encoder structure are designed in the second generator network. The decoder structure is used for downsampling to compress image features and employs a convolutional module, IN regularization, and Leaky ReLU activation function. The convolutional residual structure is used for feature extraction, connecting the decoder structure and the encoder structure, and restoring and enhancing the data. It also employs a convolutional module, IN regularization, and ReLU activation function. The encoder structure is used for upsampling to restore the image and employs a deconvolutional module, IN regularization, and ReLU activation function.

[0056] The convolutional residual structures in the first and second generators consist of 9 residual blocks. Let CIL represent a network layer composed of convolution (Conv), regularization (InstanceNorm), and Leaky ReLU activation functions; CTIR represent a network layer composed of deconvolution (ConvTranspose), regularization (InstanceNorm), and ReLU activation functions; CIR represent a network layer composed of convolution (ConV), regularization (InstanceNorm), and ReLU activation functions; and CIT represent a network layer composed of convolution (Conv), regularization (InstanceNorm), and Tanh activation functions. Let CIL... mn -K indicates a CIL containing K m×m convolutional kernels with a stride of n, CTIR mn -K indicates a CTIR with K m×m convolutional kernels and a stride of n, CIT mn -K represents a CIT with K m×m convolutional kernels and a stride of n, R×[CIR] mn -K]2 represents R residual blocks containing 2 sets of K m×m convolutional kernels with stride n. Then the decoder structure, convolutional residual structure, and encoder structure of the first and second generators can be represented as:

[0057] Decoder architecture: CIL 71 -64, CIL 32 -128, CIL 32 -256

[0058] Convolutional residual structure: 9×[CTR] 31 -256]2

[0059] Encoder structure: CTIR 32 -128, CTIR 32 -64, CIT 71 -1

[0060] Step 3.3: The padding of all convolutional layers in the constructed first and second generator networks is done using ReflectionPad2d to make the image symmetrical along the edges in all directions, thereby increasing the image resolution.

[0061] Step 4: Construct a discriminator for the color image domain of the sample, and use the discriminator to distinguish the difference in detail structure between the sample-type color image generated by the first generator and the normal sample color image; construct a discriminator for the CT image domain of the sample, and use the discriminator to distinguish the difference in detail structure between the sample-type CT image generated by the second generator and the normal sample CT image.

[0062] The discriminator architectures for the color image domain and the CT image domain of the sample are based on downsampling convolutional neural networks. To better focus on the structural details of the sample images, a PatchGAN discriminator is used. The PatchGAN discriminator employs convolutional modules, IN regularization, and the Leaky ReLU activation function. Based on the PatchGAN discriminator, the input image is downsampled multiple times to obtain images of different localities. These images are then fed into the PatchGAN for computation. The computation results are weighted and summed using the upsampling results as the final output of the discriminator. This integrates local image features and overall image characteristics, and performs differential discrimination through each patch, achieving the extraction and representation of local image features, which is beneficial for generating higher resolution images. Simultaneously, the weighted summation and averaging of the final classification feature map discrimination results can also distinguish the overall differences between the sample-type color images generated by the first generator, the sample-type CT images generated by the second generator, and the real sample images.

[0063] Let CIR denote a network layer consisting of convolution (ConV), regularization (InstanceNorm), and ReLU activation functions, and CIL denote a network layer consisting of convolution (ConV), regularization (InstanceNorm), and Leaky ReLU activation functions. Let CIR... mn -K indicates a CIR or CIL containing K m×m convolutional kernels with a stride of n. mn -K represents a CIL containing K m×m convolutional kernels with a stride of n. Therefore, the architecture of PatchGAN can be represented as:

[0064] CIR 42 -64, CIL 42 -128, CIL 42 -256, CIL 41 -512, CIL 41 -1

[0065] Step 5: Construct an adversarial loss using the first generator that converts the CT image of the sample into a sample-like color image and the discriminator in the sample color image domain. A second generator, which converts a color image of the specimen into a CT image of the specimen, is used to construct an adversarial loss function with a discriminator in the CT image domain of the specimen. The above and Together they constitute the generative adversarial loss L gan To make the generated image as similar as possible to the real image; inputting a sample CT image, a sample-type color image is generated using a first generator, and then the sample-type color image is restored to a sample-type CT image using a second generator, and compared with the real CT image to form a cycle consistency loss. Input a color image of the sample, use a second generator to generate a sample-like CT image, then use a first generator to restore the sample-like CT image to a sample-like color image, and compare it with the true color image to form a cycle consistency loss. and Together they constitute the cycle consistency loss L cyc This method addresses the challenge of obtaining paired datasets for specimen CT images and specimen color images, eliminating the need for highly accurate pairing of original digital-CT images of the specimen. It achieves unsupervised learning from specimen CT images to near-color images. The method inputs a specimen CT image, generates a specimen-like CT image using a second generator, and compares it with the input specimen CT image to construct an identifier mapping loss. Input a color image of the sample, generate a color image of the sample class using the first generator, and compare it with the input color image of the sample to form the label mapping loss. and Together they constitute the identifier mapping loss L ide This maintains the consistency of the input and output color composition, ensuring that the hue remains unchanged; the aforementioned L gan L cyc L ide Together they constitute the loss function of CycleGAN, which is used to optimize the two generators;

[0066] Step 5.1: Construct an adversarial loss using the first generator that converts the CT image of the specimen into a color image of the specimen and the discriminator in the color image domain of the specimen.

[0067]

[0068] in, This represents the generative adversarial loss that maps an image in the X domain to an image in the Y domain. This represents the expected value of the distribution of image samples in the Y domain; G represents the expected value of the distribution of image samples in the X domain; Y(x) represents the Y-domain image after mapping the X-domain image; D Y This represents a discriminator used to distinguish between the real Y-domain image and the mapped Y-domain image;

[0069] A second generator, which converts a color image of the specimen into a CT image of the specimen, is used to construct an adversarial loss function with a discriminator in the CT image domain of the specimen.

[0070]

[0071] in, This represents the generative adversarial loss that maps a Y-domain image to an X-domain image; Represents the expected value of the distribution of image samples in the X domain; G represents the expected value of the distribution of image samples in the Y domain; X (y) represents the X-domain image after mapping the Y-domain image; D X This represents a discriminator used to distinguish between real X-domain images and mapped X-domain images;

[0072] The above and Together they constitute the generative adversarial loss:

[0073] Ensure that the generated image is as similar as possible to the real image;

[0074] Step 5.2: Input the CT image of the sample, generate a sample-type color image using the first generator, then use the second generator to restore the sample-type color image to the sample-type CT image, and compare it with the real CT image to form the cycle consistency loss.

[0075]

[0076] in, This represents the distance between the image in the X domain that is mapped to the Y domain and then back to the X domain, and the real image. G represents the expected value of the distribution of image samples in the X domain; Y (x) represents the Y-domain image after mapping the X-domain image; G X (y) represents the X-domain image mapped from the Y-domain image; x represents the real image; ||G X (G Y (x))-x||1 represents G X (G Y L1 norm of (x))-x;

[0077] Input a color image of the sample, use a second generator to generate a sample-like CT image, then use a first generator to restore the sample-like CT image to a sample-like color image, and compare it with the true color image to form a cycle consistency loss.

[0078] in, This represents the distance between the image mapped from the Y-domain to the X-domain and then back to the Y-domain, and the real image. G represents the expected value of the distribution of image samples in the Y domain; X (y) represents the X-domain image after mapping the Y-domain image; G Y (x) represents the Y-domain image mapped from the X-domain image; y represents the real image; ||G Y (G X (y))-y||1 represents G Y (G X L1 norm of (y))-y;

[0079] The above and Together they constitute the cycle consistency loss:

[0080] This solves the problem of difficulty in obtaining paired datasets of specimen CT images and specimen color images, eliminating the need for high-precision pairing of original digital-CT images of the specimen, thus achieving the goal of unsupervised learning from specimen CT images to near-color images;

[0081] Step 5.3: Input the sample CT image, generate a sample-type CT image using the second generator, and compare it with the input sample CT image to construct the label mapping loss.

[0082]

[0083] in, This represents the distance between the X-domain image and the real image as mapped from the X-domain image; G represents the expected value of the distribution of image samples in the X domain; X (x) represents the X-domain image after mapping the X-domain image; x represents the real image; ||G X (x)-x||1 represents G X L1 norm of (x)-x;

[0084] Input a color image of the sample, generate a color image of the sample class using the first generator, and compare it with the input color image of the sample to form the label mapping loss.

[0085]

[0086] in, This represents the distance between the Y-domain image and the real image as mapped from the Y-domain image. G represents the expected value of the distribution of image samples in the Y domain; Y (y) represents the Y-domain image after mapping the Y-domain image; y represents the real image; ||G Y (y)-y||1 represents G Y L1 norm of (y)-y;

[0087] The above and Together they constitute the identifier mapping loss:

[0088] It maintains the consistency of the input and output color composition, ensuring that the hue remains unchanged;

[0089] Step 5.4: The above L gan L cyc L ide The loss function that together constitutes CycleGAN is:

[0090] L=λ1L gan +λ2L cyc +λ3L ide (7)

[0091] Where L is the loss function of CycleGAN, and λ1, λ2, and λ3 represent the weights of the generative adversarial loss, the cycle consistency loss, and the label mapping loss, respectively.

[0092] Step Six: Using PyCharm, construct a CycleGAN network model using the first and second generators built in Step Three, the discriminator for the sample color image domain and the sample CT image domain built in Step Four, and the generative adversarial loss, cycle consistency loss, and label mapping loss constructed in Step Five. Based on the image domain transformation information of the sample CT image and sample color image obtained in Step One and Step Two from the image database, use the CycleGAN network to connect the two generators for converting the sample CT image to the sample color image and the sample color image to the sample CT image. Perform adversarial training through continuous iteration, and stop training when the epoch value reaches a predetermined value.

[0093] First generator: It is used to receive the sample CT image in domain A, and generate a color image fake_B that resembles domain B based on the sample CT image; In addition, the first generator also needs to receive the sample CT image fake_A that resembles domain A generated by the second generator, and obtain the reconstructed image rec_B in domain B based on fake_A.

[0094] The second generator is used to receive the sample color image in domain B and generate a sample CT image fake_A in domain A based on the received rock color image. In addition, the second generator also needs to receive the color visible light image fake_B in domain B generated by the first generator and obtain the reconstructed image rec_A in domain A based on fake_B.

[0095] Sample color discriminator: Used to distinguish the detailed construction of the sample color image in the B domain and the fake_B class color image generated by the first generator;

[0096] Sample CT discriminator: used to discriminate the detailed construction of the sample CT image in domain A and the fake_A-like color CT image generated by the second generator;

[0097] Step 7: Train a two-layer recurrent generative adversarial network model using deep learning, ensuring that the model meets the preset convergence conditions. Use the first generator to colorize the CT image of the sample to be processed. Then, import the true-color images of each layer of the sample into Avizo software and use interpolation algorithms to perform three-dimensional reconstruction, achieving true-color CT three-dimensional digital images that reflect the three-dimensional structural information of the sample. Place the trained network in the cloud, and users can achieve colorization of CT images by calling cloud services.

[0098] Step 7.1: Train a two-layer recurrent generative adversarial network (GAN) model using deep learning, ensuring that the GAN model satisfies a preset convergence condition, including:

[0099] Calculate the cosine similarity between the images generated by the validation set recurrent generative adversarial network and the original images:

[0100]

[0101] Where X represents a one-dimensional vector formed by connecting the RGB values ​​of each pixel in the image generated by CycleGAN, and Y represents a one-dimensional vector formed by connecting the RGB values ​​of each pixel in the original image.

[0102] When the cosine similarity reaches 0.9 or above, it proves that the two-layer recurrent generative adversarial network model satisfies the preset convergence condition.

[0103] Step 7.2: The first generator is used to colorize the CT image of the sample to be processed; then the true-color images of each layer of the sample are imported into Avizo software and three-dimensional reconstruction is performed using an interpolation algorithm to realize true-color CT three-dimensional digital images, reflecting the three-dimensional structural information of the sample. The learned network is placed in the cloud, and users can realize the colorization of CT images by calling cloud services.

[0104] In this embodiment, a granite sample is used as an example. The CycleGAN is trained according to the following steps:

[0105] 1) A granite sample was scanned using a digital automatic slicing digital photography system. After non-destructive testing of microfocus CT images, a three-dimensional digital image of the granite was obtained. Then, a destructive slicing test was performed to obtain the original digital image of the granite. A database of granite CT images and color images was established. A total of 144 pairs of CT images with a pixel size of 1226×1226 and color images with a pixel size of 3811×3692 were obtained. The dataset of granite CT images and granite color images was divided into domain A and domain B.

[0106] 2) Using the bilinear interpolation algorithm, the resize function is called through PyCharm, and the Image.ANTIALIAS parameter is selected to reduce the resolution of the color image and unify the resolution of the CT image and the color image to 1226×1226.

[0107] The PyCharm compiler was used to execute a cropping program to crop unstable pixels that are a certain distance away from the top, bottom, left, and right boundaries of the CT and color images. After cropping, the resolution of the CT and color images is 1024×1024.

[0108] Using the reflection method, PyCharm calls the cv2 library, uses the copyMakeBorder function with the cv2.BORDER_REFLECT_101 parameter, and performs symmetrical processing on the image with the outermost pixel as the axis. The image periphery is then extended, and the resolution of the extended CT image and color image is 1280×1280.

[0109] To control the input image size and ensure computational efficiency, a window of a predetermined size [256, 256] is generated at a random location in the image, located inside the image before the cyclic boundary processing. Multiple sets of window parameters [offx, offy, thelt] are randomly given, and the window is offset and rotated according to the parameters with the window center as the reference. The image is then cut according to the window to expand the image database. After that, the image database is further expanded to 11520 pairs of images by flipping, translating, and adjusting the contrast. The image database is then divided into a training image database and a test image database in an 8:2 ratio.

[0110] 3) The granite CT image in domain A is input into the first generator in the CycleGAN model built with PyCharm. After passing through the decoder structure, convolutional residual structure, and encoder structure, the output is a fake B-domain image with the same size as the input. The fake B-domain image fake B is then input into the second generator to obtain the reconstructed domain A image rec_A.

[0111] 4) The granite color image in domain B is input into the second generator in the CycleGAN model built with PyCharm. After passing through the decoder structure, convolutional residual structure, and encoder structure, the output is a fake A domain image fake_A with the same size as the input. The fake A domain image fake_A is then input into the second generator to obtain the reconstructed B domain image rec_B.

[0112] 5) The two discriminators in the CycleGAN model built with PyCharm perform similarity judgment on the granite CT images in domain A and the granite-like CT images generated by the second generator, respectively. They also perform similarity judgment on the granite color images in domain B and the granite-like color images generated by the first generator. The output results are fed back into the loss function to update the generator parameters. The iteration stops when the number of epochs reaches the set 200.

[0113] 6) The loss function used in the recurrent generative adversarial network model includes generative adversarial loss, recurrent consistency loss and label mapping loss. The weights of the three losses are set to 1, 10 and 5. After continuous iterative optimization, if the final model can achieve a cosine similarity of 0.9 or higher between the images generated by the recurrent generative adversarial network on the validation set and the original images, it can be used as a generator model for converting granite CT images into granite color images.

[0114] 7) The 1222 granite CT images were colorized using a generator model. The 1222 true-color granite images were then imported into Avizo software and three-dimensional reconstruction was performed using an interpolation algorithm. This achieved true-color CT three-dimensional digital images of granite focal points, reflecting the three-dimensional structural information of the granite. The learned true-color granite CT network model was placed in the cloud, and users could achieve the colorization of granite CT images by calling cloud services.

[0115] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A true color processing method for rock CT digital images based on a deep network, comprising the following steps: Step 1: Establish a database of specimen CT images and specimen color images: Scan the rock specimen through a digital automatic slicing digital photography system, and obtain a micro-focus CT three-dimensional digital image through non-destructive testing of the micro-focus CT image; perform slicing destructive testing to obtain the original digital image, and establish a database of specimen CT images and specimen color images; Step 2: Preprocess the specimen CT images and color images to expand the image database, and the method is as follows: Step 2.1: Unify the resolutions of the specimen CT images and color images; Step 2.2: Crop the boundaries of the specimen CT images and color images; Step 2.3: Perform cyclic boundary processing on the cropped images; Step 2.4: Expand the image database, and divide the image database into a training image database and a test image database according to a preset ratio. The method for expanding the image database is as follows: Assume that the size of the specimen CT image is [m, n], and the size of the color image is [n1, m1]. Randomly give multiple groups of window parameters to expand the image database. For a group of randomly given window parameters, the processing method is as follows: Generate a window [w1, w2] of a predetermined size at a random position in the image, satisfying [w1 < n, w2 < m], inside the image before cyclic boundary processing. Randomly give the window an angle thelt and an offset: [offx, offy], where the random value range of offx is: [0, n - w1], and the random value range of offy is [0, m - w2]. The parameters of this window are: [offx, offy, thelt]; According to the window parameters, with the window center coordinates as the reference, offset and rotate the window to obtain the corresponding CT image of the window, and obtain the paired color image according to the same window parameters; Step 3: According to the image domain conversion information of the specimen CT images and specimen color images in the image database, construct a first generator for the generative adversarial network from the specimen CT image to the specimen-like color image; construct a second generator for the generative adversarial network from the specimen color image to the specimen-like CT image; Step 4: Construct a discriminator for the specimen color image domain, and use the discriminator to distinguish the difference between the detailed structure of the specimen-like color image generated by the first generator and the detailed structure of the normal specimen color image; construct a discriminator for the specimen CT image domain, and use the discriminator to distinguish the difference between the detailed structure of the specimen-like CT image generated by the second generator and the detailed structure of the normal specimen CT image; The discriminator for the specimen color image domain and the discriminator for the specimen CT image domain are based on a downsampling convolutional neural network. Based on the PatchGAN discriminator, the input image is downsampled multiple times to obtain different local images, and the images are respectively sent into the PatchGAN for calculation. The calculation results are weighted and added after being upsampled to obtain the final output of the discriminator, so as to fuse the local image features and the overall image characteristics; Step 5: Construct an adversarial loss using the first generator that converts the CT image of the sample into a sample-like color image and the discriminator in the sample color image domain. A second generator, which converts a color image of the specimen into a CT image of the specimen, is used to construct an adversarial loss function with a discriminator in the CT image domain of the specimen. The above and Together they constitute the generative adversarial loss L gan The input is a CT image of the sample. A first generator generates a color image of the sample. Then, a second generator restores the generated color image of the sample to a CT image of the sample. This image is then compared with the real CT image to form a cycle consistency loss. Input a color image of the sample, use a second generator to generate a sample-like CT image, then use a first generator to restore the generated sample-like CT image to a sample-like color image, and compare it with the real color image to form a cycle consistency loss. and Together they constitute the cycle consistency loss L cyc Input a sample CT image, use a second generator to generate a sample-class CT image, and compare it with the input sample CT image to construct an identifier mapping loss. Input a color image of the sample, generate a color image of the sample class using the first generator, and compare it with the input color image of the sample to form the label mapping loss. and Together they constitute the identifier mapping loss L ide The above L gan L cyc L ide Together they constitute the loss function of CycleGAN, which is used to optimize the two generators; Step Six: The CycleGAN is composed of the first and second generators built in Step Three, the discriminator in the color image domain and the discriminator in the CT image domain of the sample built in Step Four, and the generative adversarial loss, cycle consistency loss, and label mapping loss constructed in Step Five. Based on the image domain transformation information of the sample CT image and sample color image in the image database obtained in Step One and Step Two, the CycleGAN is used to connect the two generators of sample CT image to sample color image and sample color image to sample CT image. Adversarial training is carried out through continuous iteration. When the epoch value reaches the predetermined value, the training stops. After obtaining the two-layer CycleGAN model, the first generator is used to colorize the CT image.

2. The method for true-color processing of rock CT digital images according to claim 1, characterized in that, The bilinear interpolation algorithm is used to scale CT images and color images to unify their resolution.

3. The method for true-color processing of rock CT digital images according to claim 1, characterized in that, The image database was further expanded using processing methods including flipping, translating, and adjusting contrast.

4. The method for true-color processing of rock CT digital images according to claim 1, characterized in that, The method for step three is as follows: Step 3.1: Based on the image domain transformation information of the sample CT image and sample color image in the image database, construct the first generator for the generative adversarial network for converting the sample CT image to the sample-like color image; Based on the image domain transformation information of the sample CT images and sample color images in the image database, a decoder structure, a convolutional residual structure, and an encoder structure are designed in the first generator network. The decoder structure is used for downsampling to compress image features and employs a convolutional module, IN regularization, and Leaky ReLU activation function. The convolutional residual structure is used for feature extraction, connecting the decoder structure and the encoder structure, and restoring and enhancing the data. It also employs a convolutional module, IN regularization, and ReLU activation function. The encoder structure is used for upsampling to restore the image and employs a deconvolutional module, IN regularization, and ReLU activation function. Step 3.2: Based on the image domain transformation information of the sample CT image and sample color image in the image database, construct a second generator for the generative adversarial network for converting the sample color image to the sample-like CT image; Based on the image domain transformation information of the sample CT images and sample color images in the image database, a decoder structure, a convolutional residual structure, and an encoder structure are designed in the second generator network. The decoder structure is used for downsampling to compress image features and employs a convolutional module, IN regularization, and the Leaky ReLU activation function. The convolutional residual structure is used for feature extraction, connecting the decoder structure and the encoder structure, and restoring and enhancing the data. It also employs a convolutional module, IN regularization, and the ReLU activation function. The encoder structure is used for upsampling to restore the image and employs a deconvolutional module, IN regularization, and the ReLU activation function. Step 3.3: The padding of all convolutional layers in the constructed first and second generator networks is done using ReflectionPad2d to make the image symmetrical along the edges in all directions, thereby increasing the image resolution.

5. The method for true-color processing of digital images of rocks for CT according to claim 1, characterized in that, The method for performing cyclic boundary processing on the cropped image is as follows: using the reflection method, the pixels are symmetrically arranged with the outermost pixel as the axis, and the image periphery is extended to avoid boundary exceedance caused by rotation when expanding the image.

Images