Low-dose CT denoising method based on Swin Transformer
By building a deep denoising network based on Swin Transformer, the serious image noise problem in low-dose CT imaging is solved, and the image quality is significantly improved and detailed information is retained.
Patent Information
- Application Number
- CN202210721111.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-24
- Publication Date
- 2025-02-14
- Estimated Expiration
- 2042-06-24
AI Technical Summary
The projected images contain severe noise during low-dose CT imaging, resulting in degradation of image quality.
The denoising method based on Swin Transformer is adopted to build a denoising depth network, including a noise estimation subnet and a denoising and image recovery subnet based on Swin Transformer. By fusion of image features and noise levels, image noise removal is achieved.
Effectively reduce image errors, retain detailed information, significantly improve the quality of low-dose CT images, remove noise and restore image details.
Smart Images
Figure CN115049554B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer medical imaging technology, and in particular to a low-dose CT denoising method based on Swin Transformer. Background Art
[0002] Computed Tomography (CT) technology uses X-rays to collect projection information of the scanned object from different directions, and uses reconstruction algorithms to calculate the distribution of linear attenuation coefficients inside the object. It can detect the internal structure information of the object non-invasively and non-contactly and perform high-resolution imaging.
[0003] With the widespread use of CT imaging systems in medical diagnosis, the hazards that may be caused by X-ray radiation doses have gradually become the focus of public attention. Due to the cumulative effect of radiation, when the radiation dose exceeds the safe range, it will affect the physiological health of the human body, increase the risk of diseases such as tumors and gene mutations, and even have the possibility of damaging human tissues and organs. Reducing the radiation dose in the diagnostic process plays an important role in improving the above problems. The concept of low-dose CT (Low-Dose Computed Tomography, LDCT) was first proposed by Naidich et al. (DP Naidich, CH Marshall, C. Gribbin, RSA Rams, and DI M McCauley, Low-dose CT of the lungs: preliminary observations [J], Radiology, 1990, 175 (3): 729-731.), and has gradually received widespread attention from scholars in the field of CT imaging.
[0004] At present, there are two main methods for implementing LDCT in practical applications: one is to reduce the intensity of X-rays during projection acquisition by reducing the current of the X-ray source tube; the other is to reduce the radiation dose by reducing the number of projection acquisitions during the actual scanning process, that is, the sparse sampling process. Reducing the tube current is simple and easy to implement, and has become the main method for obtaining LDCT images in clinical practice. However, reducing the intensity of X-ray photons will increase the photon statistical noise of the projection data, resulting in serious degradation of the quality of the reconstructed CT image. Summary of the invention
[0005] In view of the situation that the projection image contains severe noise during low-dose CT imaging, the present invention provides a low-dose CT denoising method based on Swin Transformer. The method constructs a denoising deep network, which can utilize the image features of the projection image, fully mine the distribution characteristics of the projection image information, reduce the image error to the detail information retention ability, and finally realize the low-dose CT noise removal.
[0006] The present invention provides a low-dose CT denoising method based on Swin Transformer, comprising:
[0007] Step 1: construct and train a denoising deep network, which includes a noise estimation subnetwork and a Swin Transformer-based denoising and image restoration subnetwork;
[0008] Step 2: Input the CT projection to be denoised into the trained noise estimation subnetwork to obtain the noise level estimation map;
[0009] Step 3: The CT projection to be denoised and the corresponding noise level estimation map are simultaneously input into the trained Swin Transformer-based denoising and image restoration subnetwork to obtain the denoised CT projection.
[0010] Furthermore, the training of the denoising deep network specifically includes:
[0011] Step 1.1: Construct a noise simulation generative model in low-dose CT projections;
[0012] Step 1.2: adjusting the noise parameters in the noise simulation generation model to construct a low-dose CT projection image set under different noise levels, recorded as a simulation data set;
[0013] Step 1.3: Use the simulation data set to train the denoising deep network to obtain a trained noise estimation subnetwork and a Swin Transformer-based denoising and image restoration subnetwork.
[0014] Furthermore, the noise model is expressed by formula (1):
[0015]
[0016] Where i represents the path of the X-ray, k represents the conversion gain of X-ray photons to electrons, N0 represents the initial incident intensity of the X-ray, and σ 2 represents the variance of Gaussian noise. i represents the original number of photons, Represents the number of photons after adding noise.
[0017] Furthermore, in step 1, the loss function shown in formula (2) is used in the process of training the denoising deep network:
[0018] Loss = λ1L SL (x,y)+λ2L1(x,y) (2)
[0019] Among them, L SL represents the structural similarity loss, λ1 and λ2 represent the structural similarity loss L SL and the weight parameters of L1 loss, x is the output image predicted by the network, and y is the label image corresponding to x.
[0020] Furthermore, the noise estimation subnetwork consists of four convolutional layers.
[0021] Furthermore, the Swin Transformer-based denoising and image restoration subnetwork includes three feature information processing modules with the same structure, which are respectively recorded as the first feature information processing module, the second feature information processing module and the third feature information processing module; the first feature information processing module and the second feature information processing module respectively perform feature processing on the input CT projection to be denoised and the corresponding noise level estimation map, and the outputs of the two feature information processing modules are feature fused and input into the third feature information processing module, and the output of the third feature information processing module is the denoised CT projection.
[0022] Furthermore, the feature information processing module specifically includes three convolutional layers and two Swin Transformer modules, which are respectively recorded as a first convolutional layer, a second convolutional layer, a third convolutional layer, a first Swin Transformer module, and a second Swin Transformer module;
[0023] The feature information processing process of the feature information processing module specifically includes: the first convolution layer performs feature extraction on the input feature map, and records the extracted feature information as the first feature information; the first feature information is processed by the first SwinTransformer module and the second convolution layer in sequence, and the feature information output after processing is recorded as the second feature information; the first feature information and the second feature information are feature fused and then processed by the second Swin Transformer module and the third convolution layer in sequence, and the feature information output after processing is recorded as the third feature information; the first feature information, the second feature information and the third feature information are feature fused as the output of the feature information processing module.
[0024] Furthermore, the intermediate features obtained by the first feature information processing module from the denoised CT projection processing are transmitted to the first Swin Transformer module in the second feature information processing module.
[0025] Beneficial effects of the present invention:
[0026] The present invention first estimates and extracts the noise under different scanning conditions through the noise estimation subnetwork, and then the denoising and image restoration subnetwork takes the noisy CT projection image and the noise level map as input, and performs global structural feature extraction and detail restoration based on the denoising and image restoration subnetwork of SwinTransformer. In order to improve the noise removal and detail restoration capabilities during the network training process, a loss constraint combining structural similarity loss and L1 loss is innovatively proposed. By generating a large amount of training data, the network training can be well completed, and the low-dose CT noise removal problem can be quickly and effectively solved through the obtained training parameters. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 One of the flow charts of the low-dose CT denoising method based on Swin Transformer provided in an embodiment of the present invention;
[0028] Figure 2 A schematic diagram of a noise estimation subnetwork provided by an embodiment of the present invention;
[0029] Figure 3 A schematic diagram of a denoising and image restoration subnetwork provided by an embodiment of the present invention;
[0030] Figure 4 A schematic diagram of denoising by a denoising deep network provided by an embodiment of the present invention;
[0031] Figure 5 A schematic diagram of test results provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0032] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0033] Example 1
[0034] like Figure 1 As shown, an embodiment of the present invention provides a low-dose CT denoising method based on Swin Transformer, comprising the following steps:
[0035] S101: construct and train a denoising deep network, wherein the denoising deep network includes a noise estimation subnetwork and a denoising and image restoration subnetwork based on Swin Transformer;
[0036] Specifically, the noise in the CT projection image is mainly contained in the high-frequency information, and the low-frequency information not only contains the main content of the image, but also the weakened image texture information. The image texture information in the low-frequency information can be used to more efficiently remove the noise in the high-frequency information.
[0037] Based on the above characteristics, the embodiment of the present invention designs a denoising network for projection images. In order to remove noise from the projection image and retain and restore image detail information, the denoising network is divided into two subnetworks: a noise estimation subnetwork and a denoising and image restoration subnetwork. The noise estimation subnetwork is used to estimate and obtain a noise level estimation map from the CT projection image using noise characteristics as a loss constraint. The denoising and image restoration subnetwork is used to subsequently use the CT projection image and the noise level estimation map as input to obtain the final denoising result.
[0038] S102: Inputting the CT projection to be denoised into the trained noise estimation subnetwork to obtain a noise level estimation map;
[0039] S103: Inputting the CT projection to be denoised and the corresponding noise level estimation map simultaneously into the trained Swin Transformer-based denoising and image restoration subnetwork to obtain the denoised CT projection.
[0040] Specifically, the noise level estimation map obtained by the noise estimation subnetwork contains the main noise information. At the same time, due to the insufficient network depth, nonlinear characterization capability and loss constraints, the noise level estimation map also contains the texture information of the image.
[0041] Therefore, the low-dose CT denoising method based on Swin Transformer provided in an embodiment of the present invention first estimates and extracts the noise in the noisy CT projection image through a noise estimation subnetwork to obtain a noise level estimation map, and then the denoising and image restoration subnetwork based on Swin Transformer takes the noisy CT projection image and the noise level estimation map as input. The denoising and image restoration subnetwork can extract content features and potential texture features from the original low-dose CT projection image, as well as extract high-frequency texture features from the noise level estimation map, thereby performing global structural feature extraction and detail restoration, thereby improving the denoising accuracy of the output image.
[0042] Example 2
[0043] Based on the above embodiment, the embodiment of the present invention further provides a low-dose CT denoising method based on Swin Transformer, comprising the following steps:
[0044] S201: Constructing noise model in low-dose CT projections;
[0045] Specifically, the inventors believe that there are two essential reasons for the noise in LDCT: (1) There are interacting quantum fluctuations in the X-ray emission process, which leads to noise. This type of noise is called photon statistical noise, which usually obeys a compound Poisson distribution. Even in an ideal detection system, this noise is still unavoidable. (2) In practical applications, an ideal detection system does not exist. The electronic devices and systems themselves have noise, which in turn leads to noise in the projection data. This type of noise is usually called detection system thermal noise, which is generally assumed to obey a Gaussian distribution.
[0046] Based on the above analysis, the embodiment of the present invention comprehensively considers the statistical characteristics of photon statistical noise and system thermal noise, fully utilizes the statistical characteristics of projection data, and expresses the noise model in low-dose CT projection using formula (1):
[0047]
[0048] Where i represents the path of the X-ray, k represents the conversion gain of X-ray photons to electrons, N0 represents the initial incident intensity of the X-ray, and σ 2 represents the variance of Gaussian noise. i represents the original number of photons, Represents the number of photons after adding noise.
[0049] S202: adjusting the noise parameters in the noise model to construct a low-dose CT projection image set under different noise levels, recorded as a simulation data set;
[0050] Specifically, by modifying N0 and σ 2 These two noise parameters are used to adjust the intensity and complexity of the noise. The difficulty of noise removal increases with the increase of the intensity and complexity of the noise. In order to improve the generalization ability of the network, the embodiment of the present invention adjusts multiple sets of parameter values and simulates different noise levels, thereby constructing rich sample data. In this embodiment, a total of 50,000 training sample data and 5,000 verification sample data were produced.
[0051] S203: Construct a denoising deep network, the denoising deep network includes: Figure 2 The noise estimation subnetwork shown and Figure 3 The denoising and image restoration subnetwork based on Swin Transformer shown;
[0052] Specifically, the noise estimation subnetwork consists of four convolutional layers. The network estimates the noise level of the input projection image through the spatial distance constraint of the noise distribution.
[0053] The Swin Transformer-based denoising and image restoration subnetwork includes three feature information processing modules with the same structure, which are respectively recorded as the first feature information processing module, the second feature information processing module and the third feature information processing module; the first feature information processing module and the second feature information processing module respectively perform feature processing on the input CT projection to be denoised and the corresponding noise level estimation map, and the outputs of the two feature information processing modules are feature fused and input into the third feature information processing module, and the output of the third feature information processing module is the denoised CT projection.
[0054] Among them, Figure 3 As shown, each of the feature information processing modules includes: three convolutional layers and two Swin Transformer modules, which are respectively recorded as the first convolutional layer, the second convolutional layer, the third convolutional layer, the first Swin Transformer module and the second Swin Transformer module;
[0055] The feature information processing process of the feature information processing module specifically includes: the first convolution layer performs feature extraction on the input feature map, and records the extracted feature information as the first feature information; the first feature information is processed by the first SwinTransformer module and the second convolution layer in sequence, and the feature information output after processing is recorded as the second feature information; the first feature information and the second feature information are feature fused and then processed by the second Swin Transformer module and the third convolution layer in sequence, and the feature information output after processing is recorded as the third feature information; the first feature information, the second feature information and the third feature information are feature fused as the output of the feature information processing module.
[0056] The Swin Transforme module (STB) is used to split the input image into non-overlapping blocks of equal size, each of which is considered as a marker. If the size of the input image is H*W*C, the size of the blocks split by STB is P 2 *C, where H and W are the length and width of the image, C is the number of channels, and P is the size of the small block. Then the total number of decompositions is N = H*W / P 2In the embodiment of the present invention, given that the LDCT image size is 512*512*1, each input image is processed into 4096 image blocks, each block is flattened into a 128-dimensional identification vector, and the whole is a flattened 4096*128-dimensional sequence. The convolution layer is used to map a tensor of dimension 4096*128 to extract image features.
[0057] In order to further extract texture features in the noise, the intermediate features obtained by the first feature information processing module from the denoised CT projection are passed to the first Swin Transformer module in the second feature information processing module. Through this operation, the intermediate features of the original denoised CT projection image are passed to the feature processing process of the noise level estimation image, thereby realizing the recovery of texture information in the noise level image and improving the ability of the entire network to recover detailed texture information in the image.
[0058] exist Figure 3 Based on the structure of the feature information processing module shown, as an implementation method, the second feature information of the original CT projection image to be denoised is passed to the feature processing process of the noise level estimation map so as to be feature fused with the first feature information of the noise level estimation map.
[0059] Different from the traditional multi-head self-attention module, the SwinTransforme module (STB) is built based on the local attention and shift window mechanism. Figure 3 The lower part of the figure shows the architecture of the STB. The STB includes a window-based multi-head self-attention module and a shifted window-based multi-head self-attention module. When the window divisions of different layers are fixed, there is no new interaction between different local windows. Therefore, window division and window shifting are used for cross-window information interaction.
[0060] S204: Using a simulation data set to train the denoising deep network, to obtain a trained noise estimation subnetwork and a Swin Transformer-based denoising and image restoration subnetwork;
[0061] Specifically, when training a denoising deep network, it is critical to construct a loss function to guide the training process. The present invention uses projection denoising as an image restoration task. In order to improve the network's ability to remove noise and retain details, this embodiment designs a loss function consisting of a structural similarity loss and an L1 loss. The loss function is expressed by formula (2):
[0062] Loss = λ1L SL (x,y)+λ2L1(x,y) (2)
[0063] Among them, L SLrepresents the structural similarity loss, λ1 and λ2 represent the structural similarity loss L SL and the weight parameters of L1 loss, x is the output image predicted by the network, and y is the label image corresponding to x.
[0064] In this embodiment, the reason why the structural similarity loss L is introduced at the same time SL The reason for using the two losses is that the structural similarity (SSIM) loss is very insensitive to the uniform deviation of the image. When used alone as a loss function, it will cause color changes or brightness changes, but it can better preserve the contrast of high-frequency areas; when the L1 loss acts alone, it can preserve color and brightness, but it will cause local details to be blurred. Therefore, this embodiment combines the two losses, sets different weights, and further improves the quality of the restored image through overall optimization.
[0065] S205: inputting the CT projection to be denoised into the trained noise estimation subnetwork to obtain a noise level estimation map;
[0066] S206: Inputting the CT projection to be denoised and the corresponding noise level estimation map simultaneously into the trained Swin Transformer-based denoising and image restoration subnetwork to obtain the denoised CT projection.
[0067] Specifically, Figure 4 As shown, the CT projection to be denoised is input into the first feature information processing module, the corresponding noise level estimation map is input into the second feature information processing module, the features input from the two feature information processing modules are fused and input into the third feature information processing module to obtain the denoised CT projection.
[0068] The low-dose CT projection denoising method based on Swin Transformer provided in the embodiment of the present invention first characterizes the inherent statistical characteristics of the projection data noise by analyzing the projection generation process and the noise generation mechanism, and produces a matching training sample data set based on the projection image characteristics. By integrating the image feature information and the noise information, the network loss function is designed to improve the pertinence of noise removal and the effectiveness of detail retention. The noise under different scanning conditions is estimated and extracted by the noise estimation subnetwork to obtain the noise level estimation map of the projection image. The denoising and image restoration subnetwork based on the Swin Transformer block are used to extract deep features of the projection image and the noise level estimation map respectively. Content features and potential texture features are extracted from the original low-dose projection image, and high-frequency texture features are extracted from the noise level map. Finally, the features extracted by the two feature information processing modules are fused and further fed back to the last Swin Transformer block to realize image restoration, so as to improve the denoising accuracy of the output image; the network parameters are obtained after the network is trained, and the obtained parameters can be used to quickly and accurately complete the projection image noise removal.
[0069] In order to prove the effectiveness of the method of the present invention, the present invention also provides the following comparative experiment.
[0070] (1) Prepare simulation data sets and actual data sets
[0071] The construction process of the simulation data set has been given above and will not be repeated here.
[0072] Due to the complexity of the noise generation, it is difficult to accurately describe it using a simplified model. Only the main aspects can be simulated during the simulation process, and an image consistent with the real noise cannot be obtained. Therefore, in order to verify the denoising effect of the method proposed in this invention on real noise under actual conditions, an actual data set is further constructed to process the actual data.
[0073] Construction of actual data set: The thoracic membrane was scanned using a 16-slice medical CT (SOMATOM Sensation, Siemens). During the scanning process, different current values were set to obtain normal-dose and low-dose projection images. The tube voltage was 150 kVp, the normal-dose current was set to 120 mAs, and the tube current for LDCT was set to 30 mAs.
[0074] (2) Training and testing of denoising deep networks
[0075] The network training and testing were completed in the PyTorch (version 1.14.0) environment on the AMAX workstation. The two CPU models of the AMAX workstation used were Intel Xeon Gold 5118, with 64GB of available memory. The four computing graphics cards used for network training and testing were GeForce RTX A5000, each with 24GB of memory. The objective function optimizer selected for the network was Adam, and the learning rate was set to 1×10 -4 In the simulation experiment, the batch size is 32, the training process has 30 iterations, and the training of the entire network takes about 100 hours.
[0076] (3) Comparison network
[0077] DnCNN and CBDNet were selected as comparison networks.
[0078] (4) Experimental results
[0079] The denoising network of the present invention, the DnCNN network and the CBDNet network are used to perform denoising tests on the CT projections to be denoised in the actual data set. The test results are shown in Figure 5 shown.
[0080] Figure 5 The reconstructed images of low-dose abdominal CT are shown. Figure 5 It can be seen that the quality of CT images containing noise is seriously deteriorated, and some fine structures are blurred. All the applied methods have achieved image noise suppression to varying degrees. Although DnCNN and CBDNet can suppress the noise in CT images to a certain extent, there are still visible noise residues. Figure 5 In the results, the method of the present invention has further improved in suppressing artifact noise and retaining features, and is closest to normal-dose CT images in appearance. This is because, compared with other networks, the attention mechanism unique to the Transformer framework can effectively obtain global feature information, and the input of large quantities of training data provides a basis for giving full play to the network's effectiveness. The correction results obtained show that the low-dose CT projection denoising method based on Swin Transformer proposed in this method can fully extract the feature information of LDCT images and can effectively remove noise in CT images.
[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A low-dose CT denoising method based on Swin Transformer, characterized in that: include: Step 1: Build and train a denoising deep network, which includes a noise estimation subnetwork and a denoising and image restoration subnetwork based on SwinTransformer; Step 2: Input the CT projection to be denoised into the trained noise estimation subnetwork to obtain the noise level estimation map; Step 3: Input the CT projection to be denoised and the corresponding noise level estimation map simultaneously into the trained SwinTransformer-based denoising and image restoration subnetwork to obtain the denoised CT projection; The denoising and image restoration subnetwork based on Swin Transformer includes three feature information processing modules with the same structure, which are respectively recorded as the first feature information processing module, the second feature information processing module and the third feature information processing module; the first feature information processing module and the second feature information processing module respectively perform feature processing on the input CT projection to be denoised and the corresponding noise level estimation map, and the outputs of the two feature information processing modules are feature fused and input into the third feature information processing module, and the output of the third feature information processing module is the denoised CT projection; The feature information processing module specifically includes three convolutional layers and two Swin Transformer modules, which are respectively recorded as the first convolutional layer, the second convolutional layer, the third convolutional layer, the first Swin Transformer module and the second Swin Transformer module; The feature information processing process of the feature information processing module specifically includes: the first convolution layer extracts features from the input feature map, and records the extracted feature information as the first feature information; the first feature information is processed by the first Swin Transformer module and the second convolution layer in sequence, and the feature information output after processing is recorded as the second feature information; the first feature information and the second feature information are feature fused and then processed by the second Swin Transformer module and the third convolution layer in sequence, and the feature information output after processing is recorded as the third feature information; the first feature information, the second feature information and the third feature information are feature fused as the output of the feature information processing module; The intermediate features obtained by the first feature information processing module from the denoised CT projection processing are transmitted to the first Swin Transformer module in the second feature information processing module.
2. The low-dose CT denoising method based on Swin Transformer according to claim 1, characterized in that: The deep network for training denoising specifically includes: Step 1.1: Construct a noise simulation generative model in low-dose CT projections; Step 1.2: adjusting the noise parameters in the noise simulation generation model to construct a low-dose CT projection image set under different noise levels, recorded as a simulation data set; Step 1.3: Use the simulation data set to train the denoising deep network to obtain a trained noise estimation subnetwork and a Swin Transformer-based denoising and image restoration subnetwork.
3. The low-dose CT denoising method based on Swin Transformer according to claim 2, characterized in that: The noise model is expressed by formula (1): Where i represents the path of the X-ray, k represents the conversion gain of X-ray photons to electrons, N0 represents the initial incident intensity of the X-ray, and σ 2 represents the variance of Gaussian noise. i represents the original number of photons, Represents the number of photons after adding noise.
4. The low-dose CT denoising method based on Swin Transformer according to claim 1, characterized in that: In step 1, the loss function shown in formula (2) is used in the process of training the denoising deep network: Loss=λ1L SL (x,y)+λ2L1(x,y)(2) Among them, L SL represents the structural similarity loss, λ1 and λ2 represent the structural similarity loss L SL and the weight parameters of L1 loss, x is the output image predicted by the network, and y is the label image corresponding to x.
5. The low-dose CT denoising method based on Swin Transformer according to claim 1, characterized in that: The noise estimation subnetwork consists of four convolutional layers.