A medical image super-resolution reconstruction method for low-dose CT

Abstract

The application discloses a medical image super-resolution reconstruction method for low-dose CT, which comprises the following steps: constructing a CT image reconstruction data set; constructing an improved ESRGAN medical image super-resolution reconstruction network model; training the improved model by using the reconstruction data set; testing and evaluating the improved model; the improved model is improved based on the ESRGAN network, a Haar module is introduced before shallow feature extraction in the ESRGAN generation network, an MM module is designed to replace the RRDB module of the deep feature extraction network, a CBAM module is inserted between the deep feature extraction network and the reconstruction network, an Inverse Haar module is introduced after the reconstruction network, and the training loss function is modified. The method can improve the resolution of the original CT image, significantly reduce the noise level of the image, enhance the contrast and improve the detail resolution capability, and finally realize the purpose of improving the CT image quality.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a medical image super-resolution reconstruction method for low-dose CT. BACKGROUND

[0002] At present, medical CT images have become an important auxiliary tool for clinical medical diagnosis. In the process of CT imaging, excessive X-ray radiation dose will increase the risk of cancer for patients. To reduce the harm of CT scanning, the X-ray radiation dose used is generally reduced, that is, low-dose CT. Low-dose CT scanning plays a key role in clinical practice, effectively reducing the risk of cancer for radiologists and patients.

[0003] At present, the mainstream image super-resolution reconstruction method is to perform image super-resolution reconstruction through a deep learning model after training. At present, most experts, scholars and enterprises improve and design based on advanced general super-resolution reconstruction algorithms, such as SRCNN, VDSR and SRGAN algorithms. At present, most medical CT imaging is to reduce the harm of X-rays to the human body by using low-dose X-rays to reduce the risk of harm to the human body caused by X-rays. However, the use of low-dose X-rays will introduce obvious noise to the generated CT image. When performing super-resolution reconstruction on the low-dose CT image, if the influence of noise on the reconstructed image is not considered, the reconstructed CT super-resolution image will have problems of low contrast and fuzzy details. Most technical improvements are to directly learn the end-to-end mapping from the low-dose CT image to the full-dose CT image in the image spatial domain, so as to achieve the effect of super-resolution reconstruction. However, this kind of reconstruction method ignores the distribution characteristics of noise in the image frequency domain, that is, the noise information in the low-dose CT image is mainly distributed in the high-frequency component, and the low-frequency components of the low-dose CT image and the full-dose CT image change little. Therefore, this kind of direct learning in the image spatial domain is easy to cause the reconstructed image to have obvious noise, resulting in problems of low contrast, fuzzy details and reduced clarity, and even affecting the diagnosis results of doctors and causing more serious harm to patients.

[0004] Inventive patent application publication CN115953491A discloses a low-dose CT image super-resolution reconstruction method based on double-branch joint learning, which is characterized by parallel processing of CT image denoising reconstruction and super-resolution reconstruction. At the same time, the double-branch joint learning framework can learn the potential relationship between the multi-task through the interaction of the denoising task and the super-resolution task, and then effectively reconstruct a high-resolution CT image. The above method has two shortcomings: first, the denoising task and the super-resolution reconstruction task are trained and learned in parallel, which greatly increases the complexity of the model, making it difficult to train the model, and the trained model has poor effect, which cannot effectively suppress the noise of the reconstructed image, and is not conducive to application in actual low-dose CT image super-resolution reconstruction task. Second, the residual network used in the feature extraction module in the super-branch has the problems of insufficient utilization of residual features and loss of details. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a medical image super-resolution reconstruction method for low-dose CT, which effectively solves the problem of obvious noise in the reconstructed CT image and reduces the difficulty of model training; to solve the second problem, an MM module is designed, which uses a receptive field expansion dense residual structure to extract high-frequency details of the image when performing deep feature extraction, effectively enriching the details and textures of the reconstructed image.

[0006] To achieve the above purpose, the present application adopts the following technical solution: a medical image super-resolution reconstruction method for low-dose CT, comprising the following steps:

[0007] Step S1: Construct a CT image reconstruction dataset, select an existing low-dose CT reconstruction dataset, which includes a plurality of pairs of low-full dose CT images, use a bicubic interpolation algorithm to downsample the low-full dose CT images in the dataset, and then construct a training / test dataset, denoted as {X, Y}, X is a low-resolution low-dose CT image, and Y is a high-resolution full-dose CT image;

[0008] Step S2: Construct an improved ESRGAN medical image super-resolution reconstruction network model, the improved model is obtained by improving the ESRGAN network structure, the improved model introduces a Haar module before shallow feature extraction in the ESRGAN generator network, uses an MM module to replace the RRDB module of the deep feature extraction network, inserts a CBAM module between the deep feature extraction network and the reconstruction network, introduces an Inverse Haar module after the reconstruction network, and modifies the loss function during training;

[0009] Step S3: input the training data set obtained in step S1 into the improved ESRGAN medical image super-resolution reconstruction network model constructed in step S2 for training, sequentially pass through the shallow feature extraction network, the deep feature extraction network and the reconstruction network of the improved ESRGAN medical image super-resolution reconstruction network model, obtain the super-resolution reconstruction result of the low-dose CT image through forward propagation, calculate the loss value between the reconstructed image and the actual image through the modified loss function, and constantly optimize the parameters of the network model through the back propagation algorithm;

[0010] Step S4: use the test data set obtained in step S1 to evaluate the improved ESRGAN medical image super-resolution reconstruction network model, and use the peak signal-to-noise ratio and the structural similarity to evaluate the effect.

[0011] Step S5: use the improved model obtained in step S4, and use the CT image to be super-resolved as the model input to perform super-resolution reconstruction; after reconstruction, a low-noise and high-resolution CT image can be obtained.

[0012] In a preferred embodiment, in step S2, the Haar module is composed of a Haar wavelet transform, and the Haar module is introduced before the shallow feature extraction in the ESRGAN generation network. After filtering by the Haar module, different frequency sub-band signals are obtained.

[0013] In a preferred embodiment, in step S2, the MM module is composed of three RFRD modules in a residual connection manner, wherein β is 0.2; the RFRD module includes five RFDM modules, an LReLU activation function is connected after the first four RFDM modules, the input of each RFDM module is the output feature splicing of all previous RFDM modules, and the final output is the result obtained by splicing all feature maps through the fifth RFDM module; the RFDM module first uses a 1x1 convolution kernel to change the feature channel number of the input picture, then processes the features through three branches, the first branch is composed of a 3x3 convolution kernel with a hole rate of 1, the second branch is composed of a 3x3 convolution kernel and a 3x3 convolution kernel with a hole rate of 3 connected in turn, and the third branch is composed of a 5x5 convolution kernel and a 3x3 convolution kernel with a hole rate of 5, then the output feature maps of the three branches are fused through a concatenate layer, and the input feature maps are connected in a skip connection manner, and then processed through an LReLU activation function.

[0014] In a preferred embodiment, in step S2, the CBAM module is composed of a CAM channel attention module and a SAM spatial attention module. The CAM channel attention is obtained by maximum pooling and average pooling of the feature map in the channel dimension, and then a channel weight vector is generated through a fully connected layer and an activation function. The SAM spatial attention is obtained by maximum pooling and average pooling of the feature map generated by the channel attention, and then a spatial weight vector is generated through a fully connected layer and an activation function. The CBAM module is inserted between the deep feature extraction network and the reconstruction network to process the feature map output by the deep feature extraction, strengthen the attention of different channels and spatial positions of the network, and improve the representation ability of the network.

[0015] In a preferred embodiment, in step S2, the Inverse Haar module is composed of a Haar wavelet inverse transform. The Inverse Haar module is inserted after the ESRGAN reconstruction network to obtain the CT reconstruction image by performing Haar wavelet inverse transform on the frequency subband reconstruction signal.

[0016] In a preferred embodiment, in step S2, the modified loss function is composed of three parts.

[0017]

[0018]

[0019] L GAN = logD(y) + log(1-D(1-D(G(x))) Formula three;

[0020] L = L1 + λ1L p + λ2L GAN Formula four;

[0021] wherein L1 represents a pixel-level loss function, represents the predicted value of the kth pixel of the network model, y k represents the true value of the high-resolution CT image at the target pixel, and K is the total number of pixels; L p represents a local perception loss function, which refers to a method of taking the feature difference between the predicted image and the target image as a loss function, and can obtain more image details, represents that VGG19 is used as a feature extraction function of the network, n represents the nth layer of the VGG19 network used for feature extraction, and N represents the total number of feature extraction layers; L GANrepresents an adversarial loss function, D(*) represents the probability that the real data is discriminated as true via the discriminator, G(x) represents the image reconstructed by the generation network, y represents a high-resolution label image; L represents a total loss function, and lambda 1 and lambda 2 represent loss hyperparameters.

[0022] Compared with the prior art, the present application has the following beneficial effects: the present application can perform super-resolution reconstruction on a low-dose CT image, aiming at the problem of obvious noise after super-resolution reconstruction of the low-dose CT image, adopts learning and reconstruction on different frequency sub-band signals, and can significantly reduce the noise level of the image, enhance the contrast and improve the detail resolution capability while improving the resolution of the original CT image, so as to finally realize the purpose of improving the CT image quality, and provide great convenience for downstream medical tasks. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a generator structure diagram of an original ESRGAN model of the present application;

[0024] Figure 2 is a generator structure diagram of an improved ESRGAN medical image super-resolution reconstruction network model of the present application;

[0025] Figure 3 is a structure schematic diagram of a CBAM module of an embodiment of the present application;

[0026] Figure 4 is a structure schematic diagram of an MM module of an embodiment of the present application;

[0027] Figure 5 is a structure schematic diagram of an RFDM module of an embodiment of the present application. DETAILED DESCRIPTION

[0028] The present application will be further described below in conjunction with the drawings and embodiments.

[0029] It should be noted that the following detailed description is all exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the present application belongs.

[0030] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application; as used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form, and furthermore, it should be understood that when the terms “comprise” and / or “include” are used in the specification, they indicate the presence of a feature, step, operation, device, component and / or combination thereof.

[0031] The embodiment discloses a medical image super-resolution reconstruction method for low-dose CT, as shown in the formula: Figures 1-5 The method comprises the following steps:

[0032] Step S1: constructing a CT image reconstruction data set in an actual medical scenario, selecting an existing low-dose CT reconstruction data set, wherein a plurality of pairs of low-dose CT images and full-dose CT images are included, performing down-sampling processing on the low-dose CT images and full-dose CT images in the data set by using a bicubic interpolation algorithm, and then constructing a training / test data set, denoted as {X, Y}, wherein X is a low-resolution low-dose CT image, and Y is a high-resolution full-dose CT image;

[0033] Step S2: constructing an improved ESRGAN medical image super-resolution reconstruction network model, wherein the improved model is obtained by improving an ESRGAN network structure, the improved model introduces a Haar module before shallow feature extraction in an ESRGAN generator network, uses an MM module to replace an RRDB module of a deep feature extraction network, inserts a CBAM module between the deep feature extraction network and a reconstruction network, introduces an Inverse Haar module after the reconstruction network, and modifies a loss function during training;

[0034] Step S3: inputting the training data set obtained in step S1 into the improved ESRGAN medical image super-resolution reconstruction network model constructed in step S2 for training, sequentially passing through the shallow feature extraction network, the deep feature extraction network and the reconstruction network of the improved ESRGAN medical image super-resolution reconstruction network model, and obtaining a super-resolution reconstruction result of a low-dose CT image through forward propagation, calculating a loss value between a reconstructed image and an actual image through the modified loss function, and constantly optimizing parameters of the network model through a back propagation algorithm;

[0035] Step S4: using the test data set obtained in step S1 to evaluate indexes of the improved ESRGAN medical image super-resolution reconstruction network model, and using a peak signal-to-noise ratio and a structural similarity for effect evaluation;

[0036] Step S5: using the improved model obtained in step S4 to perform super-resolution reconstruction on a CT image to be super-resolved by taking the CT image as input of the model; and a low-noise and high-resolution CT image can be obtained after reconstruction.

[0037] In the embodiment, in the step S2, the Haar module is composed of a Haar wavelet transform, and the Haar module is introduced before shallow feature extraction in the ESRGAN generation network. After filtering by the Haar module, different frequency sub-band signals are obtained. The Haar module is mainly designed for the problem of obvious noise in the low-dose CT image. By converting the input image into different frequency sub-band signals for training, the noise interference on the reconstructed CT image is reduced, and the contrast and definition of the reconstructed CT image are improved.

[0038] In the embodiment, in the step S2, as shown in Figure 4 The MM module is composed of three RFRD modules in a residual connection manner, wherein β is 0.2; the RFRD module includes five RFDM modules, an LReLU activation function is connected after the first four RFDM modules, the input of each RFDM module is the output feature spliced from all the previous RFDM modules, and the final output is the result obtained by splicing all the feature maps and then processing by the fifth RFDM module; as shown in Figure 5 The RFDM module first changes the feature channel number of the input picture by using a 1*1 convolution kernel, then processes the features through three branches, the first branch is composed of a 3*3 convolution kernel with a hole rate of 1, the second branch is composed of a 3*3 convolution kernel and a 3*3 convolution kernel with a hole rate of 3 connected in turn, and the third branch is composed of a 5*5 convolution kernel and a 3*3 convolution kernel with a hole rate of 5, then the output feature maps of the three branches are fused through a concatenate layer, and the input feature maps are connected in a jump connection manner, and then processed by an LReLU activation function. In view of the problem of insufficient high-frequency detail feature extraction in the deep feature extraction process, the traditional method only uses a large convolution kernel or single convolution for down-sampling, which leads to the problem that the high-frequency details of the image cannot be accurately restored by the later reconstruction network. The MM module solves the problem of insufficient high-frequency detail feature extraction in the deep feature extraction process, effectively captures more context-related information in a larger range and different scales, and improves the high-frequency detail reconstruction effect of the improved ESRGAN medical image super-resolution reconstruction network model;

[0039] In this embodiment, in step S2, the CBAM module is composed of a CAM channel attention module and a SAM spatial attention module. The CAM channel attention is obtained by maximum pooling and average pooling of the feature map in the channel dimension, and then a channel weight vector is generated through a fully connected layer and an activation function, which is used to dynamically adjust the feature contribution of different channels. The SAM spatial attention is obtained by maximum pooling and average pooling of the feature map generated by the channel attention, and then a spatial weight vector is generated through a fully connected layer and an activation function, which is used to dynamically adjust the feature contribution of different spatial positions. The CBAM module is inserted between the deep feature extraction network and the reconstruction network to process the feature map output by the deep feature extraction, strengthen the attention of different channels and spatial positions of the network, and improve the representation ability of the network.

[0040] In this embodiment, in step S2, the Inverse Haar module is composed of a Haar wavelet inverse transform. The Inverse Haar module is inserted after the ESRGAN reconstruction network to obtain the CT reconstruction image by performing Haar wavelet inverse transform on the frequency subband reconstruction signal.

[0041] In this embodiment, in step S2, the modified loss function is composed of three parts.

[0042]

[0043]

[0044] L GAN = logD(y) + log(1-D(1-D(G(x))) Formula three;

[0045] L = L1 + λ1L p + λ2L GAN Formula four;

[0046] wherein L1 represents a pixel-level loss function, represents the predicted value of the kth pixel of the network model, y k represents the true value of the high-resolution CT image at the target pixel, and K is the total number of pixels; L p represents a local perception loss function, which refers to a method of taking the feature difference between the predicted image and the target image as a loss function, and can obtain more image details, represents that VGG19 is used as a feature extraction function of the network, n represents the n-th layer of the VGG19 network used for feature extraction, N represents the total number of feature extraction layers, and K is the total number of pixels in the local region; L GANrepresents an adversarial loss function, D(*) represents a probability that the real data is discriminated as true via the discriminator, G(x) represents an image reconstructed by the generation network, y represents a high-resolution label image; L represents a total loss function, and λ1 and λ2 represent loss hyperparameters.

[0047] Based on the same inventive concept, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps of the aforementioned medical image super-resolution reconstruction method for low-dose CT.

[0048] Based on the same inventive concept, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the aforementioned medical image super-resolution reconstruction method for low-dose CT.

[0049] Those skilled in the art should understand that embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0050] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the functions specified in one or more blocks.

[0051] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the functions specified in one or more blocks.

[0052] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable devices to generate a computer implemented process, so that the instructions executed on the computer or other programmable devices provide a process for implementing the flowchart Figure 1 one flow or multiple flows and / or the functions specified in the block Figure 1 one block or multiple blocks.

[0053] The above embodiments are only to illustrate the technical idea of the present application, and cannot limit the protection scope of the present application. Any modification made according to the technical idea of the present application on the basis of the technical scheme falls within the protection scope of the present application.

Claims

1. A medical image super-resolution reconstruction method for low-dose CT, characterized by, The method comprises the following steps: Step S1: constructing a CT image reconstruction data set, selecting an existing low-dose CT reconstruction data set, wherein a plurality of pairs of low-full dose CT images are included, using a bicubic interpolation algorithm to downsample the low-full dose CT images in the data set, and then constructing a training / test data set, denoted as {X, Y}, X is a low-resolution low-dose CT image, and Y is a high-resolution full-dose CT image; Step S2: constructing an improved ESRGAN medical image super-resolution reconstruction network model, wherein the improved model is obtained by improving the ESRGAN network structure, the improved model introduces a Haar module before the shallow feature extraction in the ESRGAN generator network, uses an MM module to replace the RRDB module of the deep feature extraction network, inserts a CBAM module between the deep feature extraction network and the reconstruction network, introduces an Inverse Haar module after the reconstruction network, and modifies the loss function during training; Step S3: inputting the training data set obtained in step S1 into the improved ESRGAN medical image super-resolution reconstruction network model constructed in step S2 for training, sequentially passing through the shallow feature extraction network, the deep feature extraction network and the reconstruction network of the improved ESRGAN medical image super-resolution reconstruction network model, and obtaining the super-resolution reconstruction result of the low-dose CT image through forward propagation, calculating the loss value between the reconstructed image and the actual image through the modified loss function, and continuously optimizing the parameters of the network model through the back propagation algorithm; Step S4: using the test data set obtained in step S1 to evaluate the improved ESRGAN medical image super-resolution reconstruction network model, and using the peak signal-to-noise ratio and the structural similarity to evaluate the effect; Step S5: using the improved model obtained in step S4, using the CT image to be super-resolved as the model input to perform super-resolution reconstruction; and obtaining a low-noise and high-resolution CT image after reconstruction; In step S2, the MM module is composed of three RFRD modules in a residual connection manner, wherein β is 0.2; the RFRD module comprises five RFDM modules, an LReLU activation function is connected after the first four RFDM modules, the input of each RFDM module is the output feature splicing of all previous RFDM modules, and the final output is the result obtained by the fifth RFDM module after splicing all feature maps; the RFDM module first uses a 1x1 convolution kernel to change the feature channel number of the input picture, then processes the features through three branches, the first branch is composed of a 3x3 convolution kernel with a hole rate of 1, the second branch is composed of a 3x3 convolution kernel and a 3x3 convolution kernel with a hole rate of 3 connected in turn, and the third branch is composed of a 5x5 convolution kernel and a 3x3 convolution kernel with a hole rate of 5, then the output feature maps of the three branches are fused through a concatenate layer, and the input feature maps are connected in a jump connection manner, and then processed through an LReLU activation function.

2. The method of claim 1, wherein, In the step S2, the Haar module is composed of a Haar wavelet transform, and the Haar module is introduced before shallow feature extraction in the ESRGAN generation network. After filtering by the Haar module, different frequency subband signals are obtained.

3. The method of claim 1, wherein, In the step S2, the CBAM module is composed of a CAM channel attention module and a SAM spatial attention module. The CAM channel attention is obtained by maximum pooling and average pooling of a feature map in a channel dimension, and then a channel weight vector is generated by a full connection layer and an activation function. The SAM spatial attention is obtained by maximum pooling and average pooling of a feature map generated by the channel attention, and then a spatial weight vector is generated by a full connection layer and an activation function. The CBAM module is inserted between a deep feature extraction network and a reconstruction network, processes a feature map output by the deep feature extraction, strengthens the attention of different channels and spatial positions of the network, and improves the representation ability of the network.

4. The method of claim 1, wherein, In the step S2, the Inverse Haar module is composed of a Haar wavelet inverse transform, and the Inverse Haar module is inserted after the ESRGAN reconstruction network to obtain a CT reconstruction image by performing Haar wavelet inverse transform on a frequency subband reconstruction signal.

5. The method of claim 1, wherein, In the step S2, the modified loss function is composed of three parts. L GAN = logD(y) + log(1 - D(1 - D(G(x)))) Equation Three; L = L1 + λ1L p + λ2L GAN Equation Four; Wherein, L1 represents a pixel-level loss function, represents the predicted value of the network model for the kth pixel, y k represents the true value of the high-resolution CT image at the target pixel, K is the total number of pixels; L p represents a local perception loss function, which refers to a method of taking the feature difference between the predicted image and the target image as a loss function, and more image details can be obtained, represents VGG19 as a feature extraction function of the network, n represents the n-th layer using the VGG19 network to extract features, and N represents the total number of feature extraction layers; L GAN represents an adversarial loss function, D(*) represents the probability that the real data is judged to be real by the discriminator, G(x) represents the image reconstructed by the generation network, y represents the high-resolution label image; L represents the total loss function, and λ1 and λ2 represent loss hyperparameters.

Images