Medical low-dose CT image restoration method based on frequency domain transformer

Abstract

The application discloses a medical low-dose CT image restoration method based on a frequency domain Transformer, and comprises the following steps: S1, collecting a training data set and a verification data set; S2, constructing an image restoration model, the image restoration model comprising image decomposition modules from front to back, N encoding modules, N decoding modules and an image reconstruction module; the encoding module comprises a Transformer module and a convolution layer, the decoding module comprises a linear convolution layer, a Transformer module and a reversible fusion module; S3, training the image restoration model, adjusting the parameters of the image restoration model in real time according to the evaluation effect, and obtaining a trained image restoration model; S4, inputting a low-dose CT image to be restored into the trained image restoration model, and obtaining a final restored CT image. The restoration method significantly reduces the model calculation cost, effectively removes noise and artifacts and the like in the low-dose CT image, and makes the reconstructed high-quality CT image better retain low-frequency and high-frequency details.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method for restoring low-dose CT images in medicine based on frequency domain Transformer. Background Technology

[0002] Computed tomography (CT) is a medical imaging technique that uses X-rays and computer algorithms to generate three-dimensional images of the body's interior. CT scans transmit X-rays through the body from different angles, and the computer then combines this data to create a high-resolution three-dimensional image that can be used to diagnose many different diseases and injuries. However, X-rays pose a potential radiation hazard to the human body, so it is essential to minimize their use in CT imaging. Therefore, ensuring high-precision medical diagnosis while minimizing radiation dose is a crucial challenge in the development of CT imaging technology. However, there is an unhealthy relationship between reducing radiation dose and high-quality CT imaging; reducing the CT dose leads to a decrease in projection information, resulting in a significant increase in noise and artifacts in the reconstructed images. Therefore, achieving high-quality CT imaging with low radiation doses while meeting medical diagnostic requirements is a pressing need in the current CT field.

[0003] Based on different processing stages, low-dose CT image processing methods can be mainly divided into three categories: First, projection domain preprocessing methods, which directly perform filtering or interpolation in the projection domain, and then reconstruct the CT image using the traditional Filtered Back Projection (FBP) algorithm. Since FBP is a global mapping, and filtering operations can cause local errors to spread globally, it affects overall imaging accuracy. Second, iterative reconstruction methods between the projection and image domains, which utilize the statistical characteristics of projection data and prior information from image data for modeling, and then incorporate them into a statistical reconstruction model for iterative numerical solutions. The disadvantages of this type of method are the difficulty in modeling prior information and selecting parameters, computational complexity, long iteration time, and difficulty in clinical translation. Third, image post-processing methods, which directly remove noise from the reconstructed CT image. A typical representative method is deep learning based on convolutional neural networks, which has the advantages of high algorithm training efficiency and fast inference speed; the difficulty lies in the difficulty of statistically modeling real noise, and the overall performance depends on model design. However, existing convolutional neural network methods often mine local prior information of images, while global structural priors have not been studied in depth. The Transformer method based on the global attention mechanism models global image prior information, which effectively improves the restoration performance, but lacks in-depth research on low-dose CT images, and the improvement in the quality of reconstructed CT images is limited. Moreover, the global attention mechanism requires significant computational cost, and its algorithm performance needs to be further improved. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned shortcomings in the prior art and provide a method for medical low-dose CT image restoration based on frequency domain Transformer.

[0005] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0006] A method for restoring low-dose CT images in medicine based on frequency domain Transformer includes the following steps:

[0007] Step S1: Collect training dataset and validation dataset. The training dataset consists of multiple pairs of low-dose CT images and normal-dose CT images, and the validation dataset consists of multiple low-dose CT images.

[0008] Step S2: Construct an image restoration model using a Transformer network with a UNet structure. The image restoration model includes an image decomposition module, N encoding modules, N decoding modules, and an image reconstruction module, from front to back, where N is an integer greater than or equal to 1. Each encoding module includes a Transformer module and a convolutional layer, and each decoding module includes a linear convolutional layer, a Transformer module, and a reversible fusion module. The Transformer module uses a multi-scale token mixing strategy and multi-head transpose attention to aggregate local and global image features. The reversible fusion module fuses encoded and decoded features through reversible transformation operations.

[0009] The image restoration model works as follows:

[0010] First, the image decomposition module is used to decompose the low-dose CT image into low-frequency and high-frequency components through wavelet transform. The mixed low-frequency and high-frequency component image information is then processed by N encoding modules and N decoding modules. The image reconstruction module outputs the frequency domain features of the processed image and obtains a difference image of the same scale as the input low-dose CT image through inverse wavelet transform. Finally, the difference image is added to the input low-dose CT image to reconstruct the restored low-dose CT image.

[0011] Step S3: Train the image restoration model using training set data, and evaluate the CT image enhancement model using validation set data during the training process. Adjust the parameters of the image restoration model in real time according to the evaluation results to obtain the trained image restoration model.

[0012] Step S4: Input the low-dose CT image to be restored into the trained image restoration model to obtain the final restored CT image.

[0013] In the technical solution of this invention, a Transformer network with a symmetrical UNet structure is used to construct an image restoration model. This model decomposes low-dose CT images into the frequency domain using wavelet transform. The image restoration model directly processes the decomposed mixed high-frequency and low-frequency components, followed by two processing stages: encoding and decoding. The encoding stage primarily eliminates noise, artifacts, and other factors in the high-frequency and low-frequency data, while the decoding stage reconstructs the input mixed frequency domain information from the encoded features. Finally, the mixed low-frequency and high-frequency features are transformed back to the image domain using inverse wavelet transform. Existing low-dose CT image restoration methods process directly in the CT image domain. This application transforms degraded low-dose CT images into the wavelet domain for processing. Simultaneously, it employs differential learning to construct an end-to-end low-dose CT image processing framework. The frequency-domain Transformer-based CT image restoration method not only significantly reduces the model's computational cost but also effectively removes noise and artifacts from low-dose CT images, resulting in high-quality reconstructed CT images that better preserve low-frequency and high-frequency details.

[0014] As a preferred embodiment of the present invention, the Transformer module of the present invention integrates multi-scale token mixing (MSTM) operation and lightweight multi-head transposed attention (MHTA) operation mechanism to aggregate local and global image features. The formula of the Transformer module is as follows:

[0015] Y′=MSTM(Y)

[0016]

[0017] Where Y is the input feature vector. This is the output feature vector. This invention employs a Transformer module to construct a symmetric UNet-type encoding and decoding module, which integrates MSTM and MHTA. Y is the input feature vector. First, the MSTM operation utilizes multi-scale convolution to extract diverse local features from mixed frequency domain features; while the MHTA operation utilizes an efficient attention mechanism to extract global features. The new Transformer module, constructed by integrating these two modules, is more capable of mining local and global image features with lower computational cost. The Transformer module used in this invention differs from most current mechanisms based on Swin-Transformer that use a sliding window to calculate local attention mechanisms or ViT structures to calculate global attention methods. The proposed Transformer module integrates MSTM and MHTA mechanisms to aggregate local and global image features.

[0018] As a more preferred embodiment of the present invention, the specific formula for MSTM operation is as follows:

[0019] Y1,Y2,Y3,Y4=Split(Y)

[0020] Y′=Concat(Conv 1×1 (Y1),Conv 3×3 (Y2),Conv 5×5 (Y3),MaxPool(Y4))

[0021] The specific formula for MHTA calculation is as follows:

[0022]

[0023] Atten(Q,K,V)=V·Softmax(K·Q / α)

[0024] in The input feature vector is first segmented along the channel dimension into four feature vectors Y1, Y2, Y3, and Y4. These are then processed by three convolutional modules with different kernel sizes and a max-pooling module. The processed feature vectors are then concatenated along the channel dimension to synthesize a new feature vector, which is further fed into the MHTA operation. The MHTA module first generates Query(Q), Key(K), and Value(V) through convolution operations. yes The feature vector output after the multi-head transpose attention operation can effectively model local and global feature dependencies. The attention map is obtained through the matrix dot product operation between the K and V vectors. Compared to feature maps calculated by conventional global attention mechanisms In this regard, the method of this application significantly reduces resource requirements in terms of both computation and storage.

[0025] As a preferred embodiment of the present invention, in step S2, the reversible fusion module is used to enhance the characterization capability of the fusion features, and the specific formula of the reversible fusion module is defined as follows:

[0026] F′ E =F E +φ(F D )

[0027] F′ D =F D ⊙exp(ρ(F′ E ))+η(F′ E )

[0028] Where F E and F D These are features from the encoding and decoding modules, respectively; ⊙ is a matrix dot product operation, and φ(·), ρ(·), and η(·) represent nonlinear transformation functions, which can be implemented using any existing neural network module. This invention employs a reversible feature fusion module in the decoding module to fuse encoder and decoder features. Unlike conventional feature fusion methods, which directly fuse shallow and deep features through concat or feature summation, this invention utilizes a ResBlock differential network module to construct a reversible fusion module, thereby enhancing the representational power of the fused features.

[0029] As a preferred embodiment of the present invention, two encoding modules are provided, two decoding modules are provided, and a Transformer layer is inserted between the encoding modules and the decoding modules.

[0030] As a preferred embodiment of the present invention, the image decomposition module is further provided with a convolutional layer for projecting frequency domain data onto the feature space; the kernel size of each convolutional layer is 3×3 and the stride is 1; the image reconstruction module is also provided with a convolutional layer for outputting the transformed frequency domain features; the kernel size of each convolutional layer is 3×3 and the stride is 1.

[0031] As a preferred embodiment of the present invention, a discrete wavelet decomposition layer is inserted between each of the encoding modules, and an inverse wavelet transform layer is inserted between each of the decoding modules. This replaces the conventional upsampling operator. The present invention inserts wavelet transform between each encoding module to replace the conventional downsampling operator, simultaneously generating multi-scale encoded features. In the encoding stage, for each encoding module, unlike the conventional UNet network structure which uses MaxPooling or convolution to downsample features, which easily leads to feature information loss, the present invention uses wavelet transform for feature downsampling. This further decomposes features into low-frequency and high-frequency components without increasing any trainable parameters, while simultaneously reducing the feature scale and improving the global modeling capability of the Transformer module. In the decoding stage, for the features output from each decoding layer, inverse wavelet transform is used to merge the decoded low-frequency and high-frequency information to generate new decoded features, while simultaneously increasing the feature scale, ultimately reconstructing a difference image with the same scale as the input image.

[0032] As a preferred embodiment of the present invention, a discrete wavelet decomposition layer, the Transformer module, the inverse wavelet transform layer, and the reversible fusion module are sequentially inserted between the encoding module and the decoding module.

[0033] As a more preferred embodiment of the present invention, the specific working process of the image restoration model is as follows:

[0034] Step S21: Using the image decomposition module, the low-dose CT image is decomposed into low-frequency and high-frequency components through wavelet transform, and the mixed low-frequency and high-frequency components are projected into the feature space through a convolutional layer.

[0035] Step S22: Use N coding modules to process the mixed low-frequency and high-frequency image information; in each coding module, wavelet transform is used to replace the conventional downsampling operator, and multi-scale coding features are generated at the same time.

[0036] Step S23 involves sequentially processing the data through the discrete wavelet decomposition layer, the Transformer module, the inverse wavelet transform layer, and the reversible fusion module.

[0037] Step S24: The encoded features are processed using the decoding module. Each layer of the decoding module uses inverse wavelet transform instead of the conventional upsampling operator. The reversible fusion module is used between each upsampling layer to fuse the shallow encoded features.

[0038] Step S25: Using the image reconstruction module, the image frequency domain features processed by the convolutional layer are processed by inverse wavelet transform to obtain a difference image of the same scale as the input low-dose CT image. Then, the difference image is summed with the input low-dose CT image to obtain the reconstructed normal-dose CT image.

[0039] As a preferred embodiment of the present invention, the method uses a three-layer UNet-structured image restoration model to process mixed low-frequency and high-frequency data; the final output is the processed image frequency domain features, which are obtained by inverse wavelet transform to obtain the corresponding difference image, and then summed with the input image to obtain the final reconstructed image. The process is defined as follows:

[0040]

[0041] Where F(·) represents the proposed Transformer-based restoration model, It is a reconstructed low-dose CT image.

[0042] As a preferred embodiment of the present invention, in step S3, the image enhancement model is trained using supervised training, and the objective function formula is as follows:

[0043]

[0044] Where W represents the model parameters to be learned, y i It is the input low-dose CT image pair, x i These are the corresponding normal dose CT image pairs; H(·) is the wavelet transform operation, H -1 (·) is the inverse wavelet transform operation. The loss function is represented by the L1 norm, which is used for supervised training in this invention.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0046] 1. This invention proposes a Transformer-based CT image restoration method. It employs a symmetrical UNet-type Transformer network to extract high-frequency and low-frequency features from CT images, while simultaneously encoding a CT image degradation model. A multi-scale token mixing strategy and a multi-head transposed attention module are used to construct the Transformer module, enhancing the model's ability to extract local and global image features. In the encoding stage, the model uses multi-scale encoding to process the decomposed mixed frequency domain features of the CT image. Wavelet transform is inserted into each layer of the encoding module to reduce the feature scale, while further decomposing the features into low-frequency and high-frequency components. In the decoding stage, inverse wavelet transform is inserted into each decoding layer to upsample the features, and skip-layer connections are used to fuse shallow and deep frequency domain features. Finally, the model reconstructs the difference image using inverse wavelet transform to restore a high-quality CT image. Low-dose-normal-dose CT sample pairs are constructed to train the frequency-domain Transformer-based CT image restoration network. The low-dose CT image restoration method proposed in this invention can achieve high-quality CT image restoration with very few training parameters, making it an efficient and easy-to-use medical low-dose CT image restoration method.

[0047] 2. This invention studies a low-dose CT image restoration method based on wavelet transform. By combining the advantages of existing CNNs and Transformers in modeling local and global dependencies of images respectively, it processes low-frequency and high-frequency information of degraded images. It eliminates image noise and artifacts in the wavelet domain and extracts local and global prior information of CT images through frequency domain decomposition. At the same time, it reduces the computational cost and achieves a better trade-off between low model complexity and high-quality CT reconstruction. A low-dose CT image restoration method based on frequency domain Transformer is constructed. Attached image description:

[0048] Figure 1 This is a schematic diagram of the principle framework of the medical low-dose CT image restoration method based on the frequency domain Transformer of this invention;

[0049] Figure 2 Here is a structural diagram of the Transformer block (T-Block) proposed in this invention, where (a) is a structural diagram of a single Transformer module; (b) is a structural diagram of a multi-scale token hybrid strategy operation; and (c) is a structural diagram of a multi-head transpose attention operation.

[0050] Figure 3 This is a schematic diagram of the reversible fusion module structure used in the decoding stage of this invention, where (a) is the reversible fusion framework; (b) is the ResBlock structure block, which is mainly used to model the three nonlinear mapping functions involved in (a).

[0051] Figure 4 Examples of low-dose CT images and related normal-dose CT images used in Test Examples 1 and 2;

[0052] Figure 5 These are the online PSNR training variation curves for each variant in Test Example 1 on the validation set;

[0053] Figure 6 These are the restoration results of each variant method for low-dose CT images in Test Example 1;

[0054] Figure 7 The results of the restoration of low-dose CT images using the method of the present invention and six existing methods are shown in Test Example 2. Detailed Implementation

[0055] The present invention will be further described in detail below with reference to experimental examples and specific embodiments. However, this should not be construed as limiting the scope of the above-mentioned subject matter of the present invention to the following embodiments; all technologies implemented based on the content of the present invention fall within the scope of the present invention.

[0056] Example 1

[0057] like Figure 1 As shown, this embodiment proposes a medical low-dose CT image restoration method based on frequency domain Transformer, including the following steps:

[0058] Step S1: Collect training dataset and validation dataset. The training dataset consists of multiple pairs of low-dose CT images and normal-dose CT images, and the validation dataset consists of multiple low-dose CT images. The low-dose CT images in the training dataset correspond to the normal-dose CT images.

[0059] Step S2: Construct an image restoration model using a Transformer network with a UNet structure. The image restoration model includes an image decomposition module, N encoding modules, N decoding modules, and an image reconstruction module from front to back. Each encoding module includes a Transformer module and a convolutional layer, and each decoding module includes a linear convolutional layer, a Transformer module, and a reversible fusion module. The Transformer module uses a multi-scale token mixing strategy and multi-head transpose attention to aggregate local and global image features. The reversible fusion module fuses encoded and decoded features through reversible transformation operations.

[0060] As described above, both the encoding and decoding modules include a Transformer module. The Transformer module integrates multi-scale token mixing strategy operations and multi-head transpose attention operations to aggregate local and global image features, such as... Figure 2As shown, the formulas in the Transformer module are as follows:

[0061] Y′=MSTM(Y)

[0062]

[0063] Where Y is the input feature vector. It is the output feature vector. MSTM is the multi-scale token hybrid strategy operation, MHTA is the multi-head transpose attention operation, and LN is the layer normalization operation.

[0064] The specific formula for MSTM calculation is as follows:

[0065] Y1,Y2,Y3,Y4=Split(Y)

[0066] Y′=Concat(Conv 1×1 (Y1),Conv 3×3 (Y2),Conv 5×5 (Y3),MaxPool(Y4))

[0067] The specific formula for MHTA calculation is as follows:

[0068]

[0069] Atten(Q,K,V)=V·Softmax(K·Q / α)

[0070] in It is the input feature vector, such as Figure 2 (b) The input feature vector is first divided into four feature vectors Y1, Y2, Y3, and Y4 along the channel dimension, and then fed into three convolutional modules with different kernel sizes and a max-pooling module for processing, such as... Figure 2 (c) The processed feature vectors are then concatenated along the channel dimension to synthesize new feature vectors, which are further fed into the MHTA operation; the MHTA module first generates Query(Q), Key(K), and Value(V) through convolution operations, where yes The feature vector output after the multi-head transpose attention operation can effectively model local and global feature dependencies. The attention map is obtained through the matrix dot product operation between the K and V vectors. Compared to feature maps calculated by conventional global attention mechanisms To put it simply...

[0071] Reversible fusion modules are used to enhance the representational power of fused features, such as... Figure 3As shown, the specific formula definition for the reversible fusion module is as follows:

[0072] F′ E =F E +φ(F D )

[0073] F′ D =F D ⊙exp(ρ(F′ E ))+η(F′ E )

[0074] Where F E and F D These are features from the encoding and decoding modules, respectively; ⊙ is a matrix dot product operation, and φ(·), ρ(·), and v(·) represent nonlinear transformation functions, which can be implemented using any existing neural network module. This invention employs a simple Resblock implementation to enhance the representational power of the fused features, as shown in the implementation below. Figure 3 As shown in (b).

[0075] In this invention, discrete wavelet decomposition layers are inserted between encoding modules, and inverse wavelet transform layers are inserted between each decoding module, replacing conventional upsampling operators. Wavelet transforms are inserted between each encoding module to replace conventional downsampling operators, generating multi-scale encoded features. During the encoding stage, for each encoding module, the method uses wavelet transform for feature downsampling, further decomposing features into low-frequency and high-frequency components without adding any trainable parameters, while reducing the feature scale and improving the global modeling capability of the Transformer module. In the decoding stage, for the features output from each decoding layer, inverse wavelet transform is used to merge the decoded low-frequency and high-frequency information to generate new decoded features, while increasing the feature scale, ultimately reconstructing a difference image at the same scale as the input image.

[0076] The image decomposition module also includes at least one convolutional layer (one in this embodiment) to project frequency domain data into the feature space. Each convolutional layer has a kernel size of 3×3 and a stride of 1. The image reconstruction module also includes at least one convolutional layer (one in this embodiment) to output a 4-channel frequency domain feature through a 3×3 convolutional layer. Each convolutional layer has a kernel size of 3×3 and a stride of 1.

[0077] Specifically, the image decomposition module includes a discrete wavelet decomposition layer and a 3×3 convolutional layer. This embodiment has two encoding modules, each including a Transformer module and a 3×3 convolutional layer. There are also two decoding modules, each including a 1×1 convolutional layer, a Transformer module, and a reversible fusion module. An additional discrete wavelet decomposition layer, Transformer module, inverse wavelet transform layer, and reversible fusion module are inserted between the encoding and decoding modules. The image reconstruction module includes a 3×3 convolutional layer, an inverse wavelet transform layer, and an addition module. The addition module is used to add the difference image to the input low-dose CT image, such as... Figure 1 As shown, the specific working process of the image restoration model is as follows:

[0078] Step S21: Using the image decomposition module, the low-dose CT image is decomposed into low-frequency and high-frequency components through wavelet transform, and the mixed low-frequency and high-frequency components are projected into the feature space through a convolutional layer.

[0079] Step S22: Use N coding modules to process the mixed low-frequency and high-frequency image information; in each coding module, wavelet transform is used to replace the conventional downsampling operator, and multi-scale coding features are generated at the same time.

[0080] Step S23 involves sequentially processing the data through the discrete wavelet decomposition layer, the Transformer module, the inverse wavelet transform layer, and the reversible fusion module.

[0081] Step S24: The encoded features are processed using the decoding module. Each decoding module uses inverse wavelet transform instead of the conventional upsampling operator. A reversible fusion module is used between each upsampling layer to fuse the shallow encoded features.

[0082] Step S25: Using the image reconstruction module, the image frequency domain features processed by the convolutional layer are processed by inverse wavelet transform to obtain a difference image of the same scale as the input low-dose CT image. Then, the difference image is summed with the input low-dose CT image to obtain the reconstructed normal-dose CT image.

[0083] The above process involves obtaining the corresponding difference image through inverse wavelet transform, summing it with the input image to obtain the final reconstructed image. The process is defined as follows:

[0084]

[0085] Where F(·) represents the proposed Transformer-based restoration model, It is a reconstructed low-dose CT image.

[0086] Step S3: Train the image restoration model using the training set data, and evaluate the CT image restoration model using the validation set data during the training process. Adjust the parameters of the image restoration model in real time according to the evaluation results to obtain the trained image restoration model.

[0087] The image restoration model employs supervised training, and the objective function is defined as follows:

[0088]

[0089] Where W represents the model parameters to be learned, y i It is the input low-dose CT image pair, x i These are the corresponding normal dose CT image pairs; H(·) is the wavelet transform operation, H -1 (·) is the inverse wavelet transform operation. The loss function is represented by the L1 norm, which is used in this invention to construct the loss function.

[0090] Step S4: Input the low-dose CT image to be restored into the trained image restoration model to obtain the final restored CT image.

[0091] Test Example 1

[0092] To verify the effectiveness of the restoration method provided in Example 1, this example verifies the role and impact of some key modules used in the method proposed in this invention, including: 1. Wavelet transform (including inverse wavelet transform) processing; 2. The impact of the multi-scale token hybrid strategy operation mechanism used in the Transformer module on the performance of this method; 3. The impact of the reversible fusion module on performance. The training and validation datasets used in the experiment are from the publicly available medical low-dose CT reconstruction benchmark dataset Mayo.

[0093] The conditions for different test cases are as follows:

[0094] 1-1: The baseline model adopts a standard UNet-type Transformer structure, in which the basic Transformer module is constructed using only a single multi-head transposed attention mechanism (MHTA); the upsampling and downsampling operations in the encoding and decoding modules are implemented through convolution and transposed convolution.

[0095] 1-2: The +DWT / IWT model is based on the baseline model with the addition of wavelet transform and inverse wavelet transform;

[0096] 1-3: The +MSTM model adds multi-scale token hybrid strategy computation to the baseline model;

[0097] 1-4: The +DWT / IWT+MSTM model is based on the baseline model and adds wavelet transform and inverse wavelet transform, as well as multi-scale token hybrid strategy operation.

[0098] 1-5: The Full model is based on the baseline model with the addition of +DWT+MSTM+iFFM, which is the method of this invention.

[0099] The experimental results for different test cases are shown in Table 1.

[0100] Table 1 Comparison of metrics for different module methods

[0101]

[0102] As can be seen from the data in Table 1 above, the +DWT model method reduces model parameters and computational cost while improving model quantization metrics. Compared to the baseline model, the method's overall performance is further improved when multi-scale token mixing strategy operations are added, but the model also introduces more training parameters. Using a fusion of DWT and MSTM modules can effectively alleviate the problem of introducing more training parameters, while the model performance is basically unaffected. Finally, adding the reversible fusion module to the decoder constitutes the complete method proposed in this invention. It can be seen that the model introduces a very small number of training parameters, but the overall performance is further improved.

[0103] This further validated the impact of each module in Table 1 on model performance during the training phase. Figure 4 This displays low-dose CT images and corresponding normal-dose CT images used for testing on the Mayo dataset; the first row is the low-dose CT image input, and the second row is the corresponding normal-dose CT image, where the low-dose CT is a simulated 1 / 4 dose CT imaging result. Figure 5 As shown, this invention statistically analyzed the online PSNR training variation curves for each variant on the validation set. It can be seen that the complete method proposed in this invention has a faster convergence speed. Compared to the baseline model, the proposed method exhibits more stable training and faster convergence.

[0104] This test case also compares the visualization restoration results of each variant method on low-dose CT images, such as... Figure 6 As shown. Among them. Figure 6 (a) Low-dose CT image; (b), (c), (d), and (e) correspond to the baseline model, +DWT model, +MSTM model, and Full model, respectively; (f) is a normal-dose CT image. It can be observed that the method proposed in this invention is significantly more effective than the baseline model in removing noise and artifacts and preserving structural details.

[0105] Test Example 2

[0106] To illustrate the difference between the restoration method of this invention and existing low-dose CT image processing methods, this test case selected six existing methods for comparative experiments. Each method was trained according to the experimental settings adopted in this invention to ensure a fair and reasonable comparison. The six comparative methods are:

[0107] Method 2-1: The method proposed by Dabov et al., referenced in "K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, "Image denoising by sparse 3-d transform-domain collaborative filtering," IEEE Transactions on Image Processing, vol.16, pp.2080–2095, 2007.";

[0108] Method 2-2: The method proposed by Chen et al., referenced in "H.Chen, Y.Zhang, M.Kalra, F.Lin, Y.Chen, P.Liao, J.Zhou, and G.Wang, "Low-dose ct with a residual encoder-decoder convolutional neural network," IEEE Transactions on Medical Imaging, vol.36, pp.2524–2535, 2017.";

[0109] Method 2-3: The method proposed by Yang et al., the reference is: "Q. Yang, P. Yan, Y. Zhang, H. Yu, Y. Shi, X. Mou, M. Kalra, Y. Zhang, L. Sun, and G. Wang, "Low-dose ct image denoising using a generative adversarial network with wasserstein distance and perceptual loss," IEEE Transactions on Medical Imaging, vol.37, pp.1348–1357, 2018.”;

[0110] Methods 2-4: The methods proposed by Liang et al., referenced as: "T.Liang, Y.Jin, Y.Li, T.Wang, S.Feng, and C.Lang, "Edcnn: Edge enhancement-based densely connected network with compound loss for low-dose denoising," 2020 15th IEEE International Conference on Signal Processing (ICSP), vol.1, pp.193–198, 2020.";

[0111] Methods 2-5: The methods proposed by Luthra et al., reference: "A. Luthra, H. Sullakh, T. Mittal, A. Iyer, and SKYadav, "Eformer: Edge enhancement based transformer for medical image denoising," ArXiv, vol.abs / 2109.08044, 2021.";

[0112] Methods 2-6: The methods proposed by Wang et al., referenced as: "D. Wang, F. Fan, Z. Wu, R. Liu, F. Wang, and H. Yu, "Ctformer: Convolution-free token2token dilated visiontransformer for low-dose ct denoising," ArXiv, vol.abs / 2202.13517, 2022.";

[0113] The method proposed in this invention was tested on the Mayo test set along with the six existing methods mentioned above. The experimental results are shown in Table 2. LDCT refers to low-dose CT images. Among the existing methods, method 2-1 is a classic image restoration method, namely the BM3D method (3D block matching algorithm). Methods 2-2, 2-3, and 2-4 are conventional methods based on convolutional neural networks. Methods 2-5 and 2-6 are Transformer-based methods. According to the data in the table, compared with other methods, this invention achieves a better trade-off between the number of model parameters and the required computational cost, and the quantization indicators (PSNR and SSIM) are improved most significantly. PSNR is improved by 1-2 dB, and the SSIM indicator is also optimal.

[0114] Table 2 Comparison of test indicators between the method of the present invention and existing methods

[0115] LDCT 38.13 0.961 -- -- Method 2-1 42.10 0.983 -- -- Method 2-2 43.36 0.989 1.85 462.48 Method 2-3 40.08 0.979 34.07 14.76 Method 2-4 42.71 0.987 0.08 21.33 Methods 2-5 43.12 0.988 1.11 67.45 Methods 2-6 42.76 0.987 1.45 219.42 Method of the present invention 43.80 0.990 5.67 72.02

[0116] This test case selected a representative low-dose CT image from the test set, which contains some metallic streak artifacts and low-dose noise. The restoration results of the method proposed in this invention were compared with those of the six existing methods mentioned above. Figure 7 As shown. Figure 7 (a) is a low-dose CT image. Figure 7 In the table, (b), (c), (d), (e), (f), and (g) represent methods 2-1, 2-2, 2-3, 2-4, 2-5, and 2-6, respectively. Figure 7 (h) represents the method proposed in this invention. Figure 7 In Figure (i), a normal-dose CT image is shown. It is evident from the figure that the method proposed in this invention significantly improves the removal of noise and artifacts from low-dose CT images while preserving image structural details. Furthermore, the quantitative metrics (PSNR and SSIM) are significantly improved compared to other methods. In summary, the method of this invention achieves excellent restoration results in tasks such as removing noise and stripe artifacts from low-dose CT images. Through comprehensive comparison with recent related methods, the method of this invention demonstrates better restoration performance in terms of both quantitative metrics and visual perception, achieving a better trade-off between model computational complexity and performance metrics, making it more suitable for practical applications. In conclusion, this invention is an effective method for restoring low-dose CT images in medicine.

[0117] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for restoring low-dose medical CT images based on frequency domain Transformer, characterized in that, Includes the following steps: Step S1: Collect training dataset and validation dataset. The training dataset consists of multiple pairs of low-dose CT images and normal-dose CT images, and the validation dataset consists of multiple low-dose CT images. Step S2: Construct an image restoration model using a Transformer network with a UNet structure. The image restoration model includes an image decomposition module, N encoding modules, N decoding modules, and an image reconstruction module from front to back. Each encoding module includes a Transformer module and a convolutional layer, and each decoding module includes a linear convolutional layer, a Transformer module, and a reversible fusion module. The Transformer module uses a multi-scale token mixing strategy and multi-head transpose attention to aggregate local and global image features. The reversible fusion module fuses the encoded and decoded features through reversible transformation operations. The image restoration model works as follows: Using the image decomposition module, the low-dose CT image is decomposed into low-frequency and high-frequency components through wavelet transform. The mixed low-frequency and high-frequency component image information is processed by N encoding modules and N decoding modules. Using the image reconstruction module, the frequency domain features of the processed image are output, and the difference image is obtained through inverse wavelet transform. Then, it is added to the input low-dose CT image to reconstruct the restored low-dose CT image. Step S3: Train the image restoration model using training set data, and evaluate the CT image enhancement model using validation set data during the training process. Adjust the parameters of the image restoration model in real time according to the evaluation results to obtain the trained image restoration model. Step S4: Input the low-dose CT image to be restored into the trained image restoration model to obtain the final restored CT image.

2. The method for restoring low-dose CT images in medicine based on frequency domain Transformer according to claim 1, characterized in that, The formula for the Transformer module is as follows: Y′=MSTM(Y) Where Y is the input feature vector. It is the output feature vector, MSTM is the multi-scale token hybrid strategy operation, MHTA is the multi-head transpose attention operation, and LN is the layer normalization operation.

3. The method for restoring low-dose CT images in medicine based on frequency domain Transformer according to claim 2, characterized in that, The specific formula for MSTM calculation is as follows: Y1,Y2,Y3,Y4=Split(Y) Y′=Concat(Conv 1×1 (Y1),Conv 3×3 (Y2),Conv 5×5 (Y3),MaxPool(Y4)) The specific formula for MHTA calculation is as follows: Atten(Q,K,V)=V·Softmax(K·Q / α) in It is the input feature vector.

4. The method for restoring low-dose CT images in medicine based on frequency domain Transformer according to claim 1, characterized in that, The specific formula for the reversible fusion module is defined as follows: F′ E =F E +φ(F D ) F′ D =F D ⊙exp(ρ(F′ E ))+η(F′ E ) Where F E and F D These are features derived from the encoding module and the decoding module, respectively. ⊙ is a matrix dot product operation, and φ(·), ρ(·), and η(·) represent nonlinear transformation functions, which are implemented by the residual network module.

5. The method for restoring low-dose CT images in medicine based on frequency domain Transformer according to claim 1, characterized in that, There are two encoding modules and two decoding modules.

6. The method for restoring low-dose CT images in medicine based on frequency domain Transformer according to claim 1, characterized in that, The image decomposition module is further provided with at least one convolutional layer for projecting frequency domain data onto the feature space; the kernel size of each convolutional layer is 3×3 and the stride is 1; the image reconstruction module is further provided with at least one convolutional layer for outputting the transformed frequency domain features; the kernel size of each convolutional layer is 3×3 and the stride is 1.

7. The method for restoring low-dose CT images in medicine based on frequency domain Transformer according to claim 6, characterized in that, A discrete wavelet decomposition layer is inserted between the encoding modules, and an inverse wavelet transform layer is inserted between the decoding modules.

8. The method for restoring low-dose CT images in medicine based on frequency domain Transformer according to claim 7, characterized in that, A discrete wavelet decomposition layer, the Transformer module, an inverse wavelet transform layer, and the reversible fusion module are sequentially inserted between the encoding module and the decoding module.

9. The method for restoring low-dose CT images in medicine based on frequency domain Transformer according to claim 8, characterized in that, The specific working process of the image restoration model is as follows: Step S21: Using the image decomposition module, the low-dose CT image is decomposed into low-frequency and high-frequency components through wavelet transform, and the mixed low-frequency and high-frequency components are projected into the feature space through a convolutional layer. Step S22: Use N coding modules to process the mixed low-frequency and high-frequency image information; in each coding module, wavelet transform is used to replace the conventional downsampling operator, and multi-scale coding features are generated at the same time. Step S23 involves sequentially processing the data through the discrete wavelet decomposition layer, the Transformer module, the inverse wavelet transform layer, and the reversible fusion module. Step S24: The encoded features are processed using the decoding module. Each layer of the decoding module uses inverse wavelet transform instead of the conventional upsampling operator. The reversible fusion module is used between each upsampling layer to fuse the shallow encoded features. Step S25: Using the image reconstruction module, the image frequency domain features processed by the convolutional layer are processed by inverse wavelet transform to obtain a difference image of the same scale as the input low-dose CT image. Then, the difference image is summed with the input low-dose CT image to obtain the reconstructed normal-dose CT image.

10. The method for restoring low-dose CT images in medicine based on frequency domain Transformer according to claim 1, characterized in that, In step S3, the image enhancement model is trained using supervised training, and the objective function is defined as follows: Where W represents the model parameters to be learned, y i It is the input low-dose CT image pair, x i These are the corresponding normal dose CT image pairs; H(·) is the wavelet transform operation, H -1 (·) is the inverse wavelet transform operation. This represents the loss function.

Images