CT image virtual thin layer reconstruction method and medium

Abstract

The invention discloses a CT image virtual thin layer reconstruction method and a medium, and belongs to the technical field of information, and the method comprises the steps: obtaining the resolution of an input CT image as a first resolution, and dividing the input CT image into a plurality of image blocks; reconstructing the image block with the first resolution by using a generator in a generative adversarial network (GAN) to generate an image block with a second resolution, the generator comprising a plurality of cascaded convolutional layers and at least one pixel anti-shuffling layer; a discriminator is used for carrying out authenticity discrimination on the second resolution image block reconstructed by the generator to obtain an authenticity probability value, and the discriminator adopts a U-Net structure; according to the authenticity probability value output by the discriminator, the loss function comprises the weighted sum of the adversarial loss, the perception loss and the pixel loss; a gradient descent algorithm is utilized to optimize parameters of the generator, and the weight of each layer in the generator is adjusted through a back propagation algorithm. Aiming at low image resolution in CT image reconstruction in the prior art, the method improves the discrimination rate.

Description

Technical Field

[0001] The present application relates to the field of information technology, and more specifically, to a method and medium for virtual thin-layer reconstruction of CT images. Background Art

[0002] With the development of medical imaging technology, CT scanning has become an important tool for diagnosing and treating many diseases. In order to reduce storage space and improve processing efficiency, only thicker slices (such as 3mm or 5mm) are usually reconstructed clinically. However, the stored thick layer images may not provide enough details in some cases, thus affecting the accuracy of diagnosis. Although thinner slice thickness (such as 0.75mm) can provide more details, it will take up a lot of storage space and increase the difficulty of processing. In addition, once the original acquisition data is cleared, it is impossible to reconstruct thinner images.

[0003] Virtual Thin Slice is a technology that reconstructs thicker slice CT images into thinner slice images through computational methods. It can significantly improve the detail resolution of the image without increasing the radiation dose. This technology is of great significance to clinical diagnosis and can help doctors observe the lesion more accurately and improve the accuracy and efficiency of diagnosis.

[0004] However, existing virtual thin layer reconstruction techniques still have some limitations. Traditional interpolation-based methods, such as linear interpolation and cubic interpolation, are computationally simple but have difficulty restoring high-frequency details of images and have limited reconstruction quality. Sparse representation-based methods, such as compressed sensing and dictionary learning, can restore image details to a certain extent, but have slow reconstruction speeds and lack real-time performance. In addition, these methods usually require manual design of complex mathematical models and prior knowledge, have limited generalization capabilities, and are difficult to adapt to different data distributions and imaging conditions.

[0005] In recent years, deep learning technology has made significant progress in the field of image super-resolution reconstruction. Compared with traditional methods, deep learning methods can automatically learn mapping relationships from large amounts of data without the need to manually design complex mathematical models, and have stronger feature extraction and mapping capabilities. However, existing virtual thin layer reconstruction methods based on deep learning still have some shortcomings. For example, although simple convolutional neural networks (CNNs) can quickly reconstruct images, the generated images often lack realism and have problems such as over-smoothing and texture loss. Although generative adversarial networks (GANs) can generate more realistic images, the training process is unstable and prone to artifacts and noise. In addition, existing methods usually only focus on the visual quality of images and lack constraints on the authenticity of reconstructed images, which may lead to large differences between the generated images and the real images, affecting the reliability of diagnosis. Summary of the invention

[0006] 1. Technical issues to be solved

[0007] In view of the problem of low image resolution in CT image reconstruction existing in the prior art, the present application provides a method and medium for virtual thin-layer reconstruction of CT images, which improves the reconstruction resolution of CT images without increasing storage space by improving and optimizing the Real-ESRGAN model.

[0008] 2. Technical solution

[0009] The purpose of this application is achieved through the following technical solutions.

[0010] One aspect of the present application provides a method for virtual thin-layer reconstruction of a CT image, comprising: obtaining the resolution of an input CT image as a first resolution, and dividing the input CT image into a plurality of image blocks; reconstructing the image block of the first resolution using a generator in a generative adversarial network GAN to generate an image block of a second resolution, wherein the second resolution is greater than the first resolution, and the generator comprises: a plurality of cascaded convolutional layers for extracting features of the image block of the first resolution step by step; a plurality of residual dense blocks RRDB, each RRDB embedding a plurality of residual dense blocks RDB; wherein each RDB comprises a plurality of densely connected convolutional layers, and performs a nonlinear transformation on the input feature map through a convolution operation to enhance the feature extraction capability; at least one pixel de-shuffling layer, wherein the pixel de-shuffling layer is a pixel de-shuffling layer; ... Between the convolution layer and the RRDB, the feature map output by the convolution layer is upsampled according to a preset scale factor to reduce the spatial size of the feature map and increase the number of channels; the discriminator is used to discriminate the authenticity of the second-resolution image block reconstructed by the generator to obtain an authenticity probability value, and the discriminator adopts a U-Net structure; according to the authenticity probability value output by the discriminator, the loss function of the generator is calculated, and the loss function includes the weighted sum of adversarial loss, perceptual loss and pixel loss; according to the loss function, the parameters of the generator are optimized by the gradient descent algorithm, and the weights of each layer in the generator are adjusted by the back propagation algorithm to minimize the loss function; the trained generator is used to reconstruct the CT image of the first resolution to generate a CT image with improved resolution.

[0011] Furthermore, a discriminator is used to perform authenticity discrimination on the image blocks reconstructed by the generator, including: the discriminator adopts a U-Net structure, which includes an encoder and a decoder; the encoder is used to extract the features of the image blocks, and the decoder is used to restore the spatial resolution of the features; multiple jump connections are introduced between the downsampling path of the encoder and the upsampling path of the decoder, and the jump connection is used to splice the output feature map of the downsampling convolution layer in the encoder with the output feature map of the upsampling convolution layer of the corresponding level in the decoder to enhance feature fusion; a spectral normalization layer is added after the convolution layer of the encoder, and the weight matrix of the convolution layer is constrained by the spectral norm. The gradient norm of the discriminator is limited by spectral norm regularization, so as to control the update speed of the discriminator and reduce artifacts and noise in the reconstructed image.

[0012] Furthermore, the loss function of the generator is calculated, including: calculating the adversarial loss, perceptual loss and pixel loss of the generator: the adversarial loss is calculated using the relative authenticity average RaGAN loss function, and the relative authenticity probability difference between the reconstructed image block and the real image block is estimated by the discriminator to measure the authenticity of the generated image; the perceptual loss uses a pre-trained VGG convolutional neural network to extract multi-scale feature maps from the reconstructed image block and the real image block, calculates the L1 norm distance of each feature space, and performs weighted summation to measure the perceptual difference between the generated image and the real image; the pixel loss is used to measure the pixel-level difference by calculating the L1 norm distance between the reconstructed image block and the corresponding real image block in the pixel space; the adversarial loss, perceptual loss and pixel loss are weightedly summed to obtain the final loss function of the generator; the gradient is calculated according to the final loss function of the generator, and the network weights of the generator are updated using the gradient descent algorithm to minimize the loss function.

[0013] Furthermore, the adversarial loss of the generator is calculated, including: the discriminator receives the image block reconstructed by the generator and the corresponding real image block as input, and outputs the authenticity probability difference of the reconstructed image block relative to the real image block; according to the authenticity probability difference output by the discriminator, the RaGAN adversarial loss of the generator is calculated to evaluate the difference in authenticity of the generated image relative to the real image; by minimizing the RaGAN adversarial loss, the weight of the generator is updated using the gradient descent algorithm to improve the quality of the generated image.

[0014] Furthermore, the perceptual loss of the generator is calculated, including: inputting the image blocks reconstructed by the generator and the corresponding real image blocks into the pre-trained VGG convolutional neural network respectively; extracting feature maps of the reconstructed image blocks and the real image blocks in multiple convolutional layers of the VGG network; for each convolutional layer, calculating the L1 norm distance between the reconstructed image blocks and the real image blocks in the feature space of the corresponding layer to quantify the difference between the two at the perceptual level; weighted summing the L1 norm distances of each feature space to obtain the perceptual loss; and by minimizing the perceptual loss, using the gradient descent algorithm to update the network weights of the generator to improve the perceptual quality of the generated image.

[0015] Furthermore, the pixel loss of the generator is calculated, including: comparing the image block reconstructed by the generator with the real image block at the corresponding position pixel by pixel in the pixel space; for each pixel position, calculating the absolute value of the difference between the pixel values ​​of the reconstructed image block and the real image block at the corresponding position; summing the absolute values ​​of the pixel value differences of all pixel positions to obtain the L1 norm distance between the reconstructed image block and the real image block in the pixel space as the pixel loss; by minimizing the pixel loss, the gradient descent algorithm is used to update the network weights of the generator to improve the pixel-level accuracy of the generated image.

[0016] Furthermore, the generator includes multiple RRDBs, each RRDB embeds multiple RDBs, wherein: the residual dense block RRDB receives the feature map output by the convolution layer as input; each RRDB contains multiple RDBs, and the RDB is used to perform nonlinear transformation on the input feature map to enhance the feature extraction capability; each RDB contains multiple densely connected convolutional layers, which are connected in sequence, and the input of each convolutional layer is the concatenation of the outputs of all previous convolutional layers; the output of the RDB is the residual feature map obtained by adding the output of the last convolutional layer to the input feature map of the RDB; the output of the RRDB is the final feature map obtained by adding the output feature maps of multiple RDBs to the input feature map of the RRDB.

[0017] Furthermore, the generator includes at least one pixel deshuffle layer, which is located between the convolution layer and the RRDB, including: the pixel deshuffle layer receives the feature map output by the convolution layer as input; according to a preset scale factor, a pixel deshuffle operation is performed on the input feature map, and the pixels of the feature map are rearranged according to a preset pattern, and recombined into multiple sub-feature maps; the spatial size of the rearranged sub-feature map is smaller than the original feature map, thereby reducing the spatial size of the feature map and increasing the number of channels; the feature map after the pixel deshuffle operation is input into the residual dense block RRDB for further feature extraction and enhancement.

[0018] Another aspect of the present application also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by a processor, a method for virtual thin-layer reconstruction of CT images of the present application is provided.

[0019] 3. Beneficial effects

[0020] Compared with the prior art, the advantages of this application are:

[0021] This application uses the generator in the generative adversarial network (GAN) to gradually extract and fuse the multi-scale features of the image through cascaded convolutional layers, residual dense blocks (RRDB) and pixel de-shuffling layers to enhance feature extraction and representation capabilities. This enables the generator to recover high-frequency details from low-resolution CT images and generate clearer and more detailed high-resolution images. Compared with traditional interpolation methods, the method of this application can better preserve the texture and edge information of the image and effectively improve the reconstruction resolution of CT images.

[0022] This application introduces a discriminator to discriminate the authenticity of the reconstructed image as a guide for generator optimization. The discriminator adopts a U-Net structure, extracts image features through the encoder, and restores the spatial resolution of the features through the decoder, and introduces a jump connection between the encoder and the decoder to enhance the feature fusion capability. At the same time, a spectral normalization layer is added after the convolution layer of the encoder, and the update speed of the discriminator is controlled by the spectral norm constraint to reduce artifacts and noise in the reconstructed image. This adversarial learning mechanism can prompt the generator to generate more realistic and natural reconstructed images and improve the authenticity of the reconstructed images.

[0023] The loss function of the present application comprehensively considers adversarial loss, perceptual loss and pixel loss, and balances the visual quality and authenticity of the reconstructed image. Among them, the adversarial loss is calculated using the relative authenticity average (RaGAN) loss function, and the relative authenticity probability difference between the reconstructed image and the real image is estimated by the discriminator to guide the generator to generate a more realistic image. Perceptual loss uses a pre-trained VGG convolutional neural network to extract multi-scale feature maps of the reconstructed image and the real image, and calculates the L1 norm distance in the feature space to ensure that the reconstructed image is perceptually close to the real image. Pixel loss ensures the similarity of the reconstructed image at the pixel level by calculating the L1 norm distance between the reconstructed image and the real image in the pixel space. Through this multi-task learning approach, the method of the present application can generate reconstructed images with high visual quality, rich details and more consistent with the real image. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 An exemplary flow chart for generating virtual thin slices of CT images of the present application;

[0025] Figure 2 This is a schematic diagram of the generator structure of this application;

[0026] Figure 3 This is a schematic diagram of the discriminator structure of this application;

[0027] Figure 4 Schematic diagram of the loss function calculation process of this application. DETAILED DESCRIPTION

[0028] The present application is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0029] Figure 1 The invention discloses an exemplary flow chart for generating a virtual thin layer of a CT image of the present invention, comprising: obtaining the resolution of an input CT image as a first resolution, and dividing the input CT image into a plurality of image blocks; reconstructing the image block of the first resolution by using a generator in a generative adversarial network GAN to generate an image block of a second resolution, wherein the second resolution is greater than the first resolution, and the generator comprises: a plurality of cascaded convolutional layers for extracting features of the image block of the first resolution step by step; a plurality of residual dense blocks RRDB, each RRDB embedding a plurality of residual dense blocks RDB; wherein each RDB comprises a plurality of densely connected convolutional layers, and performs a nonlinear transformation on the input feature map through a convolution operation to enhance the feature extraction capability; at least one pixel deshuffling layer, located between the convolutional layers; Between the convolution layer and the RRDB, the feature map output by the convolution layer is upsampled according to the preset scale factor to reduce the spatial size of the feature map and increase the number of channels; the discriminator is used to discriminate the authenticity of the second-resolution image block reconstructed by the generator to obtain the authenticity probability value, and the discriminator adopts the U-Net structure; according to the authenticity probability value output by the discriminator, the loss function of the generator is calculated, and the loss function includes the weighted sum of adversarial loss, perceptual loss and pixel loss; according to the loss function, the parameters of the generator are optimized by the gradient descent algorithm, and the weights of each layer in the generator are adjusted by the back propagation algorithm to minimize the loss function; the trained generator is used to reconstruct the CT image of the first resolution to generate a CT image with improved resolution.

[0030] Virtual Thin Slice technology is a technology that reconstructs CT images with thicker slice thickness into images with thinner slice thickness through computational methods. It can significantly improve the detail resolution of the image without increasing the radiation dose.

[0031] Figure 2This is a schematic diagram of the generator structure of the present application. Step 1: Obtain the slice thickness and spatial resolution information of the input CT image as the first resolution. For example, the slice thickness of the input CT image is 5mm and the pixel spacing is 1mm×1mm. According to the preset image block size, the input CT image is divided into multiple non-overlapping image blocks. The size of the image block can be set according to actual needs. For example, the CT image can be divided into 256×256 or 512×512 image blocks. The divided image blocks are used as the input of the generator to prepare for virtual thin layer reconstruction. Basic network: The generator adopts the same SR network architecture as Real-ESRGAN. The generator consists of multiple residual dense blocks (RRDBs), and each RRDB contains multiple residual dense blocks (RDBs). The structure of RRDB can effectively enhance the feature extraction and fusion capabilities and improve the quality of the generated image. Remove the batch normalization layer in the generator. The batch normalization layer may cause the loss of image details, so removing the batch normalization layer can retain more image detail information. The input of the generator is a low-resolution (LR) CT image block, which is processed by the generator and outputs a high-resolution (HR) reconstructed image block.

[0032] Step 2: Use the generator in the generative adversarial network GAN to reconstruct the image block of the first resolution to generate an image block of the second resolution. Specifically, the CT image block of the first resolution is input into the generator for feature extraction and reconstruction. The first part of the generator consists of multiple cascaded convolutional layers, which are used to extract the features of the first resolution image block step by step. The first convolutional layer performs initial feature extraction on the input image block to obtain a low-level feature map. Subsequent convolutional layers gradually extract higher-level features based on the low-level feature map to enhance the feature representation capability. The number and parameters of the convolutional layers can be set according to specific needs to balance the feature extraction capability and computational complexity.

[0033] After the convolutional layer, at least one pixel deshuffle layer is introduced to upsample the feature map output by the convolutional layer. The pixel deshuffle layer rearranges the feature map according to a preset scale factor (such as ×2 or ×4), reducing the spatial size of the feature map and increasing the number of channels. When the scale factor is ×2, the pixel deshuffle layer halves the spatial size of the feature map and increases the number of channels by 4 times. When the scale factor is ×4, the pixel deshuffle layer reduces the spatial size of the feature map to 1 / 4 of the original size and increases the number of channels by 16 times. Through the pixel deshuffle operation, most of the calculations are performed in a lower resolution space, reducing the consumption of GPU memory and computing resources.

[0034] The output feature maps of the pixel deshuffle layer are fed into the main ESRGAN architecture, namely multiple residual dense blocks (RRDBs). Each RRDB contains multiple residual dense blocks (RDBs) to further enhance the feature extraction capability. Each RDB consists of multiple densely connected convolutional layers, which perform nonlinear transformations on the input feature maps through convolution operations. The convolutional layers inside the RDB are densely connected, that is, the input of each convolutional layer is the concatenation of the outputs of all previous convolutional layers, which helps the flow of information and the propagation of gradients. The output of the RDB is the residual feature map obtained by adding the output of the last convolutional layer to the input feature map of the RDB.

[0035] The output feature maps of multiple RRDBs are added to the input feature maps of RRDB to obtain the final feature representation. The residual structure of RRDB facilitates the direct propagation of gradients and alleviates the gradient vanishing problem in deep networks. The cascade of multiple RRDBs can further enhance the feature extraction and fusion capabilities and generate high-quality reconstructed images. After being processed by multiple RRDBs, an enhanced feature representation is obtained, and then the feature map is gradually enlarged to the target resolution through a series of upsampling convolution layers. The upsampling convolution layer uses techniques such as deconvolution or sub-pixel convolution to enlarge the spatial size of the feature map while reducing the number of channels. The upsampling process can use multiple consecutive upsampling convolution layers to achieve higher magnification.

[0036] The last convolutional layer converts the upsampled feature map into a high-resolution reconstructed image block, that is, generates a second-resolution CT image block. The second resolution is greater than the first resolution, achieving super-resolution reconstruction of the image. The reconstructed high-resolution CT image block has higher spatial resolution and richer detail information. The high-resolution (HR) CT image block is the output result of the generator.

[0037] Figure 3This is a schematic diagram of the discriminator structure of the present application. Step 3: Use the discriminator to discriminate the authenticity of the second resolution image block reconstructed by the generator to obtain the authenticity probability value. The discriminator adopts a U-Net structure and consists of two parts: an encoder and a decoder. Encoder part: The encoder is used to extract the features of the input image block, and through a series of convolutional layers and downsampling operations, gradually reduce the spatial size of the feature map, and increase the number of feature channels. After each convolutional layer of the encoder, a spectral normalization layer is added to constrain the spectral norm of the weight matrix of the convolutional layer. Spectral normalization controls the update speed of the discriminator by limiting the gradient norm of the discriminator, which helps to stabilize the training process of GAN and reduce artifacts and noise in the reconstructed image. Spectral norm regularization can effectively limit the gradient norm of the discriminator, prevent the gradient of the discriminator from exploding or disappearing, and thus improve the training stability of GAN. During the downsampling process of the encoder, the output feature map of each downsampled convolutional layer is saved for subsequent splicing with the feature map of the decoder.

[0038] Decoder: The decoder is used to restore the spatial resolution of the features. Through a series of upsampling convolutional layers and skip connections, the spatial size of the feature map is gradually increased while reducing the number of feature channels. In the upsampling path of the decoder, multiple skip connections are introduced to concatenate the output feature map of the corresponding level of the downsampling convolutional layer in the encoder with the output feature map of the corresponding level of the upsampling convolutional layer in the decoder. The purpose of the skip connection is to fuse the low-level features extracted by the encoder with the high-level features restored by the decoder to enhance the feature representation capability and the restoration of spatial details. Through the skip connection, the discriminator can use both low-level and high-level features to make a more comprehensive and accurate authenticity judgment on the generated image block. After being processed by the encoder and decoder, the discriminator outputs the authenticity value of each pixel, indicating the probability that the pixel belongs to the real image. The output of the discriminator has the same spatial size as the input image block, and each pixel corresponds to a authenticity value. The value range of the authenticity value is usually between [0,1]. The closer the value is to 1, the more likely the pixel is from the real image, and the closer the value is to 0, the more likely the pixel is from the generated image. The authenticity values ​​output by the discriminator are averaged or pooled to obtain the authenticity probability of the entire image block. The authenticity probability of an image block indicates the possibility that the image block as a whole belongs to the real image. The authenticity probability can be used as an evaluation indicator of the reconstruction quality of the generator to guide the training and optimization of the generator. The authenticity probability of each image block is the output result of the discriminator.

[0039] Figure 4This is a schematic diagram of the loss function calculation process of the present application. Step 4: According to the authenticity probability value output by the discriminator, the loss function of the generator is calculated. The loss function includes the weighted sum of adversarial loss, perceptual loss and pixel loss: Calculate the adversarial loss of the generator. The adversarial loss is calculated using the relative authenticity average (RaGAN) loss function. The RaGAN loss function estimates the relative authenticity probability difference between the reconstructed image block and the real image block through the discriminator to measure the authenticity of the generated image. The specific calculation steps are as follows: The image block reconstructed by the generator and the corresponding real image block are input into the discriminator to obtain their authenticity probability values. Calculate the difference between the authenticity probability of the reconstructed image block and the authenticity probability of the real image block to obtain the relative authenticity probability difference. Take the average of the relative authenticity probability difference to obtain the value of the RaGAN loss function. The formula of the RaGAN loss function is expressed as: L_RaGAN=E[log(D(x_real)-D(x_fake))], where D represents the discriminator, x_real represents the real image block, and x_fake represents the reconstructed image block. By minimizing the RaGAN loss function, the authenticity probability of the image patches generated by the generator will be close to the authenticity probability of the real image patches, thus improving the quality of the generated images.

[0040] Calculate the perceptual loss of the generator. The perceptual loss uses the pre-trained VGG convolutional neural network to extract multi-scale feature maps of the reconstructed image blocks and the real image blocks. The specific calculation steps are as follows: Input the reconstructed image block and the corresponding real image block into the pre-trained VGG network to extract feature maps of multiple feature spaces. For the feature map of each feature space, calculate the L1 norm distance between the reconstructed image block and the real image block. Perform a weighted sum of the L1 norm distances of different feature spaces to obtain the value of the perceptual loss. The formula for perceptual loss is expressed as: L_percep=Σw_i*||φ_i(x_fake)-φ_i(x_real)||_1, where φ_i represents the i-th feature space of the VGG network and w_i represents the weight of the feature space. By minimizing the perceptual loss, the image blocks generated by the generator are closer to the real image blocks in the feature space, improving the perceptual quality of the generated image.

[0041] Calculate the pixel loss of the generator. Pixel loss measures the pixel-level difference of the reconstructed image by calculating the L1 norm distance between the reconstructed image block and the corresponding real image block in the pixel space. The specific calculation steps are as follows: Compare the reconstructed image block and the corresponding real image block in the pixel space. Calculate the L1 norm distance between the reconstructed image block and the real image block, that is, calculate the sum of the absolute value difference pixel by pixel. The formula for pixel loss is expressed as: L_pixel=||x_fake-x_real||_1, where x_fake represents the reconstructed image block and x_real represents the real image block. By minimizing the pixel loss, the image block generated by the generator is closer to the real image block at the pixel level, improving the pixel-level similarity of the reconstructed image.

[0042] Calculate the total loss function of the generator. The total loss function of the generator is the weighted sum of the adversarial loss, perceptual loss and pixel loss. The specific calculation steps are as follows: Set weight coefficients for the adversarial loss, perceptual loss and pixel loss respectively to indicate their importance in the total loss function. The adversarial loss, perceptual loss and pixel loss are weighted and summed according to the weight coefficients to obtain the total loss function of the generator. The formula of the total loss function is expressed as: L_total = λ_1*L_RaGAN+λ_2*L_percep+λ_3*L_pixel, where λ_1, λ_2, and λ_3 represent the weight coefficients of the adversarial loss, perceptual loss and pixel loss, respectively. By minimizing the total loss function, the image blocks generated by the generator are optimized in terms of authenticity, perceptual quality and pixel-level similarity, thereby improving the overall quality of the reconstructed image.

[0043] Step 5: Calculate the total loss function of the generator based on the adversarial loss, perceptual loss, and pixel loss calculated in step 4. The formula of the total loss function is expressed as: L_total = λ_1*L_RaGAN+λ_2*L_percep+λ_3*L_pixel, where λ_1, λ_2, and λ_3 represent the weight coefficients of the adversarial loss, perceptual loss, and pixel loss, respectively. Calculate the gradient of the loss function with respect to the parameters of the generator. Use automatic differentiation tools (such as PyTorch's autograd or TensorFlow's GradientTape) to calculate the gradient of the loss function with respect to each trainable parameter in the generator. The specific calculation steps are as follows: Input the input data of the generator (such as a low-resolution image patch) and the corresponding real high-resolution image patch into the generator to obtain a reconstructed high-resolution image patch. Calculate the loss function of the generator (adversarial loss, perceptual loss, and pixel loss) based on the reconstructed high-resolution image patch and the real high-resolution image patch. Use the automatic differentiation tool to backpropagate the loss function and calculate the gradient of the loss function with respect to each trainable parameter in the generator. Get the gradient value of each trainable parameter in the generator, which represents the rate of change of the loss function on this parameter.

[0044] Update the parameters of the generator according to the gradient. Use a gradient descent algorithm (such as stochastic gradient descent (SGD), Adam optimizer, etc.) to update the trainable parameters in the generator according to the calculated gradient. The specific update steps are as follows: Set the learning rate, which represents the step size of each parameter update. For each trainable parameter in the generator, calculate the update amount of the parameter according to its gradient value and learning rate. Subtract the update amount from the current value of the parameter to get the new value of the parameter. Repeat the above steps to update all trainable parameters in the generator. The formula for parameter update is expressed as: Where θ represents the trainable parameters of the generator, η represents the learning rate, Represents the gradient of the loss function with respect to the parameters. The updated generator parameters are used for the next iteration of the training process. Repeat the following steps until the predetermined number of training rounds is reached or the convergence condition is met: Input the low-resolution image block into the updated generator to obtain a reconstructed high-resolution image block. Calculate the loss function of the generator, including adversarial loss, perceptual loss, and pixel loss. Calculate the gradient of the loss function with respect to the updated generator parameters. Update the parameters of the generator based on the gradient and learning rate. Through multiple iterative optimizations, the parameters of the generator are gradually adjusted so that it can generate more realistic and high-quality high-resolution image blocks. When the predetermined number of training rounds is reached or the convergence condition is met, the parameters of the trained generator model are saved to a file. The saved generator model can be used in the subsequent inference process to perform super-resolution reconstruction on new low-resolution image blocks.

[0045] Step 6: Use the trained generator to reconstruct the first-resolution CT image to generate a CT image with improved resolution. Read the first-resolution CT image data that needs to be reconstructed with super-resolution into the memory. Preprocess the CT image data, such as normalization and resizing, to meet the input requirements of the generator. Input the preprocessed first-resolution CT image data into the trained generator model one by one. The generator receives the first-resolution CT image data as input and generates the corresponding high-resolution CT image through the forward propagation process. The generator gradually maps the first-resolution CT image data to the high-resolution space through multiple convolutional layers and upsampling layers. The specific reconstruction process is as follows: The first-resolution CT image data is extracted through a series of convolutional layers to learn the low-level and high-level feature representations of the image. The spatial resolution of the feature map is gradually increased through upsampling layers (such as transposed convolutional layers or sub-pixel convolutional layers). In the process of upsampling, the generator uses the learned feature representation to generate the corresponding high-resolution feature map. The high-resolution feature map is mapped to the target high-resolution CT image space through the last convolutional layer. The architecture and parameters of the generator have learned the mapping relationship from low resolution to high resolution during the training phase, so they can be directly used for super-resolution reconstruction. The output of the generator is the reconstructed high-resolution CT image. The high-resolution CT image output by the generator is saved to memory or written to a file. The reconstructed high-resolution CT image is used for subsequent tasks such as medical diagnosis, lesion detection, and treatment planning. High-resolution CT images provide more details and spatial information, which helps to improve the accuracy and reliability of medical applications.

[0046] The invention of the present application and its implementation methods are described schematically above. The description is not restrictive. Without departing from the spirit or basic features of the present application, the present application can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention of the present application. The actual structure is not limited to this. Any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the present invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all fall within the scope of protection of this patent. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. A method for virtual thin-layer reconstruction of CT images, comprising: Acquire a resolution of an input CT image as a first resolution, and divide the input CT image into a plurality of image blocks; The image block of the first resolution is reconstructed using a generator in a generative adversarial network GAN to generate an image block of a second resolution, where the second resolution is greater than the first resolution, and the generator includes: A plurality of cascaded convolutional layers are used to extract features of the first resolution image block level by level; Multiple residual dense blocks RRDB, each RRDB embeds multiple residual dense blocks RDB; each RDB contains multiple densely connected convolutional layers, which perform nonlinear transformation on the input feature map through convolution operations to enhance feature extraction capabilities; At least one pixel deshuffle layer, located between the convolutional layer and the RRDB, upsamples the feature map output by the convolutional layer according to a preset scaling factor to reduce the spatial size of the feature map and increase the number of channels; The discriminator is used to discriminate the authenticity of the second resolution image block reconstructed by the generator to obtain the authenticity probability value. The discriminator adopts the U-Net structure. According to the authenticity probability value output by the discriminator, the loss function of the generator is calculated. The loss function contains the weighted sum of adversarial loss, perceptual loss and pixel loss. According to the loss function, the parameters of the generator are optimized using the gradient descent algorithm, and the weights of each layer in the generator are adjusted through the back propagation algorithm to minimize the loss function; The trained generator is used to reconstruct the CT image of the first resolution to generate a CT image with improved resolution.

2. The method for virtual thin-layer reconstruction of CT images according to claim 1, characterized in that: The discriminator is used to judge the authenticity of the image blocks reconstructed by the generator, including: The discriminator adopts the U-Net structure, which consists of two parts: an encoder and a decoder. The encoder is used to extract the features of the image block, and the decoder is used to restore the spatial resolution of the features. Multiple skip connections are introduced between the downsampling path of the encoder and the upsampling path of the decoder. The skip connections are used to concatenate the output feature map of the downsampling convolutional layer in the encoder with the output feature map of the upsampling convolutional layer of the corresponding level in the decoder to enhance feature fusion. A spectral normalization layer is added after the convolution layer of the encoder to constrain the spectral norm of the weight matrix of the convolution layer. The spectral norm regularization is used to limit the gradient norm of the discriminator, control the update speed of the discriminator, and reduce artifacts and noise in the reconstructed image.

3. The method for virtual thin-layer reconstruction of CT images according to claim 2, characterized in that: Calculate the loss function of the generator, including: Calculate the adversarial loss, perceptual loss, and pixel loss of the generator: The adversarial loss is calculated using the relative authenticity average RaGAN loss function. The relative authenticity probability difference between the reconstructed image block and the real image block is estimated by the discriminator to measure the authenticity of the generated image. The perceptual loss uses a pre-trained VGG convolutional neural network to extract multi-scale feature maps from the reconstructed image blocks and the real image blocks, calculate the L1 norm distance of each feature space, and perform weighted summation; The pixel loss is calculated by calculating the L1 norm distance between the reconstructed image block and the corresponding real image block in the pixel space; The adversarial loss, perceptual loss and pixel loss are weighted summed to obtain the final loss function of the generator; The gradient is calculated based on the final loss function of the generator, and the gradient descent algorithm is used to update the network weights of the generator to minimize the loss function.

4. The method for virtual thin-layer reconstruction of CT images according to claim 3, characterized in that: Calculate the adversarial loss of the generator, including: The discriminator receives the image block reconstructed by the generator and the corresponding real image block as input, and outputs the authenticity probability difference of the reconstructed image block relative to the real image block; Calculate the RaGAN adversarial loss of the generator based on the authenticity probability difference output by the discriminator; The weights of the generator are updated using the gradient descent algorithm by minimizing the RaGAN adversarial loss.

5. The method for virtual thin-layer reconstruction of CT images according to claim 4, characterized in that: Calculate the perceptual loss of the generator, including: The image blocks reconstructed by the generator and the corresponding real image blocks are respectively input into the pre-trained VGG convolutional neural network; Extract feature maps of reconstructed image patches and real image patches in multiple convolutional layers of the VGG network; For each convolutional layer, the L1 norm distance between the reconstructed image block and the real image block in the feature space of the corresponding layer is calculated; The perceptual loss is obtained by weighted summing of the L1 norm distances of each feature space; The network weights of the generator are updated using the gradient descent algorithm by minimizing the perceptual loss.

6. The method for virtual thin-layer reconstruction of CT images according to claim 5, characterized in that: Calculate the pixel loss of the generator, including: Compare the image block reconstructed by the generator with the real image block at the corresponding position pixel by pixel in the pixel space; For each pixel position, the absolute value of the difference between the pixel values ​​of the reconstructed image block and the real image block at the corresponding position is calculated; The absolute values ​​of the pixel value differences at all pixel positions are summed to obtain the L1 norm distance between the reconstructed image block and the real image block in the pixel space as the pixel loss; The network weights of the generator are updated using the gradient descent algorithm by minimizing the pixel loss.

7. The method for virtual thin-layer reconstruction of CT images according to any one of claims 2 to 6, characterized in that: The generator contains multiple RRDBs, each of which embeds multiple RDBs, including: The residual dense block RRDB receives the feature map output by the convolutional layer as input; Each RRDB contains multiple RDBs, which are used to perform nonlinear transformations on input feature maps to enhance feature extraction capabilities; Each RDB contains multiple densely connected convolutional layers, which are connected in sequence. The input of each convolutional layer is the concatenation of the outputs of all previous convolutional layers. The output of RDB is the residual feature map obtained by adding the output of the last convolutional layer to the input feature map of RDB; The output of RRDB is the final feature map obtained by adding the output feature maps of multiple RDBs and the input feature map of RRDB.

8. The method for virtual thin-layer reconstruction of CT images according to claim 7, characterized in that: The generator contains at least one pixel deshuffle layer between the convolutional layer and the RRDB, including: The pixel deshuffle layer receives the feature map output by the convolutional layer as input; According to the preset scale factor, a pixel deshuffle operation is performed on the input feature map, and the pixels of the feature map are rearranged according to a preset pattern to recombine into multiple sub-feature maps; The spatial size of the rearranged sub-feature map is smaller than the original feature map, which reduces the spatial size of the feature map and increases the number of channels; The feature map after the pixel deshuffle operation is input into the residual dense block RRDB.

9. A computer-readable storage medium storing computer instructions, which implement the method according to any one of claims 1 to 8 when executed by a processor.

Images