Low-dose CT image denoising method based on double-flow feature fusion

Abstract

The invention relates to the technical field of medical image processing, in particular to a low-dose CT image denoising method based on double-flow feature fusion, and the method comprises the steps: constructing a low-dose CT image denoising network based on double-flow feature fusion; a low-dose CT image denoising network based on double-flow feature fusion is trained, a trained CT image denoising model is obtained, and a segmented training strategy is adopted in the training process; and obtaining a to-be-processed low-dose CT image, and processing the low-dose CT image by using the CT image denoising model to obtain a denoised low-dose CT image. According to the method, the prior information of different scales is generated through training and integrated into the regression model, the prior information is dynamically aggregated in the feature extraction process, and the deep fusion of the prior features and the intermediate features is realized through the cross attention mechanism, so that the prior information is aggregated into the features of the regression model; and the generalization ability of the model in a complex noise scene is obviously enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and particularly to a low-dose CT image denoising method based on dual-stream feature fusion. Background Art

[0002] Computed tomography (CT) plays an important role in medical diagnosis. CT imaging with high-dose X-rays is clearer, but its ionizing radiation may pose potential hazards to human health. Low-dose X-ray (LDCT) technology reduces the risk by reducing the radiation dose. However, low-dose scanning leads to an increase in CT image noise and artifacts, affecting the diagnostic value. Image post-processing algorithms can effectively remove noise and retain important details by utilizing local or global characteristics of the image, providing a feasible solution for improving the quality of low-dose CT images.

[0003] In existing image post-processing technical solutions, methods using RED-CNN, EDCNN, CTformer, and Uformer regression models focus on accuracy and uniqueness and perform well in distortion metric indicators. However, there are problems such as simple output smoothing and insufficient restoration of high-frequency details. Methods using GAN, WGAN-VGG, and DU-GAN generative models can better retain image textures in the denoising task and have excellent detail restoration ability. However, there are problems such as large output randomness, poor stability, and high computational complexity.

[0004] Therefore, the existing two technical solutions still have defects in aspects such as the accuracy of noise modeling, stability, integrity of detail retention, restoration of data authenticity, and improvement of computational efficiency, and further solutions are urgently needed. Summary of the Invention

[0005] In view of this, the present invention discloses a low-dose CT image denoising method based on dual-stream feature fusion to solve the above problems, including:

[0006] S1. Construct a low-dose CT image denoising network based on dual-stream feature fusion;

[0007] S2. Train the low-dose CT image denoising network based on dual-stream feature fusion to obtain a trained CT image denoising model. The training process adopts a segmented training strategy, and the training data includes full-dose CT images and low-dose CT images;

[0008] S3. Obtain the low-dose CT image to be processed, and use the CT image denoising model to process the low-dose CT image to obtain a denoised low-dose CT image.

[0009] Furthermore, the low-dose CT image denoising network based on dual-stream feature fusion includes: a generative feature enhancement module, a multi-scale strategy module, a regression denoising module, and a feature fusion module;

[0010] The generative feature enhancement module is used to generate prior features according to the low-dose CT image to be processed; the regression denoising module is used to extract global features from the low-dose CT image to be processed for denoising, and perform deep fusion with the prior features using the feature fusion module to generate a denoised low-dose CT image.

[0011] The beneficial effects of the present invention include: generating prior information of different scales through training and integrating it into the regression model to enrich the information volume of the prior features, dynamically aggregating the prior information during the feature extraction process, and achieving deep fusion of the prior features and the intermediate features through the cross-attention mechanism, so as to aggregate the prior information into the features of the regression model, significantly enhancing the generalization ability of the model in complex noise scenarios, giving full play to the respective advantages of the regression model and the generative model, and enabling it to better adapt to the low-dose CT image denoising task with different noise levels; by adopting a multi-stage training strategy, ensuring that the generator can generate outputs highly consistent with the real prior features, thereby further improving the quality of the prior features and the denoising performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic structural diagram of the low-dose CT image denoising network based on dual-stream feature fusion in the present invention; Figure 1 In (a) is a schematic structural diagram of the regression denoising module, (b) is a schematic structural diagram of the generative feature enhancement module, and (c) is a schematic diagram of the multi-scale strategy module;

[0013] Figure 2 It is a schematic structural diagram of the prior feature encoder in the present invention;

[0014] Figure 3 It is a schematic structural diagram of the prior feature generative adversarial network in the present invention;

[0015] Figure 4 It is a schematic structural diagram of the feature fusion module in the present invention;

[0016] Figure 5 It is a schematic diagram of the segmented training strategy adopted in the present invention; Figure 5 In (a) is a schematic diagram of the first-stage training, (b) is a schematic diagram of the second-stage training, and (c) is a schematic diagram of the test stage. DETAILED DESCRIPTION OF THE INVENTION

[0017] In order to make the purpose, technical solutions, features, and advantages of the present invention clearer and more understandable, the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0018] This embodiment includes a low-dose CT image denoising method based on dual-stream feature fusion, including:

[0019] S1. Construct a low-dose CT image denoising network based on dual-stream feature fusion.

[0020] S2. Train the low-dose CT image denoising network based on dual-stream feature fusion to obtain a trained CT image denoising model; the training process adopts a segmented training strategy, and the training data includes full-dose CT images and low-dose CT images.

[0021] S3. Obtain the low-dose CT image to be processed, and use the CT image denoising model to process the low-dose CT image to obtain a denoised low-dose CT image.

[0022] Furthermore, as Figure 1 shown, the low-dose CT image denoising network based on dual-stream feature fusion includes: a generative feature enhancement module, a multi-scale strategy module, a regression denoising module, and a feature fusion module.

[0023] Furthermore, the generative feature enhancement module includes: a prior feature encoder (PFE) and a prior feature generative adversarial network (PF-GAN).

[0024] Specifically, the PFE is used to generate real prior features. As Figure 2 shown, the processing of data by the PFE includes:

[0025] Step 1. Reallocate the spatial dimension information of the full-dose CT image to the channel dimension through Pixel-Unshuffle and 3×3 convolution to obtain channel-reallocated features. While reducing the spatial resolution, the number of channels is increased, thereby retaining local detail information and reducing the computational amount.

[0026] Step 2. Use 5 residual blocks to further mine the channel-reallocated features to obtain deep context features. Each residual block contains two 3×3 convolutional layers and a ReLU activation function layer, and the ReLU activation function layer is set between the two convolutional layers to enhance the feature expression ability.

[0027] Step 3. Perform 3×3 convolution, global average pooling, and three reshaping-linear mapping operations on the deep context features to obtain real prior features Z.

[0028] The PFE designed in the present invention enables the dimension of the prior features to be much lower than the total number of pixels of the input image. Its rationality includes: by compressing the features into a low-dimensional latent space, the distortion problem caused by inaccurate generation of details by the generative adversarial network can be effectively avoided; it exhibits better generalization ability in complex noise scenarios and can adapt to various noise distributions and intensities; the generated latent features, as prior information, retain the structural and texture information of high-quality images, and significantly reduce the computational complexity and training difficulty through feature compression, playing a key role in subsequent denoising tasks.

[0029] Furthermore, PF-GAN uses the real prior features as a reference to generate high-quality prior features Z′ from low-dose CT images, thereby assisting the regression network to achieve precise denoising and reducing the dependence of the model on full-dose CT images. Its structure is as Figure 3 shown.

[0030] The generator is designed with a stacked structure to cover as large a receptive field as possible, so as to capture the global and local features of the input more comprehensively and enhance the integrity and diversity of feature extraction. Specifically, the generator consists of a convolutional network and a ReLU activation function. The convolutional layer uses a small 3×3 convolutional kernel, and the ReLU activation function is used after the convolutional operation to enhance the nonlinear expression ability of the model, so as to more effectively extract image features and generate latent prior features Z′.

[0031] The discriminator D is used to distinguish the latent prior features Z′ generated by the generator from the real prior features Z output by the prior feature encoder, and provides accurate guidance for the training of the generator by distinguishing the generated features from the real features. Specifically, the discriminator consists of a convolutional layer, a LeakyReLU activation function, and a linear mapping layer. The convolutional layer uses a small 3×3 convolutional kernel.

[0032] Furthermore, the present invention adopts the Wasserstein GAN (WGAN) as the training framework. WGAN uses the Wasserstein distance instead of the JS divergence to measure the difference between data distributions, thereby alleviating the common mode collapse and gradient vanishing problems in the original GAN training, and WGAN removes the logarithmic function in the loss function. WGAN trains the generator G and the discriminator D by solving the min-max problem. The formula of the WGAN training framework is:

[0033]

[0034] where represents the expected output of the real data x under the discriminator D, represents the expected output of the generated data G(z) under the discriminator D, z is the noise, is the gradient penalty term for network regularization, Points uniformly sampled from the straight line between the generated samples and the real samples; λ is the weight coefficient of the gradient penalty term, preferably 10 in this embodiment.

[0035] Furthermore, due to the high complexity and diversity of the noise in low-dose CT images, it is difficult for single-scale features to fully capture the variable characteristics of the noise. To enhance the diversity of features and the adaptability of the model to noise, the present invention designs a multi-scale strategy module. The multi-scale strategy module gradually generates multi-scale feature representations Z i , i ∈ {1, 2, 3, 4}, and introduces these features into the regression denoising module for fusion, thereby enhancing the network's ability to utilize multi-scale information. The schematic diagram of the multi-scale strategy is shown in Figure 1 (c) as shown.

[0036] Furthermore, as shown in Figure 1 , the regression denoising module adopts an "encoder-decoder" architecture. After the low-dose CT image to be processed enters the regression denoising module, it is processed by a 3×3 convolution and a ReLu activation function, and then processed by the encoder and decoder. After that, it is processed by a 3×3 convolution and a ReLu activation function again, and added element by element to the low-dose CT image to be processed to obtain the denoised low-dose CT image.

[0037] Specifically, the encoder consists of four layers of 5×5 convolutions. Before each layer of convolution, a feature fusion module (FIM) designed by the present invention is set, and a ReLu activation function is set after each layer of convolution; the decoder consists of four layers of 5×5 transposed convolutions. Before each layer of transposed convolution, a feature fusion module is set, and a ReLu activation function is set after each layer of transposed convolution.

[0038] The FIM is used to fuse the multi-scale feature representations and the intermediate features of the regression denoising module. The intermediate features are the output features of the previous level in the regression denoising module. The FIM dynamically aggregates prior information during the feature extraction process and realizes the deep fusion of prior features and intermediate features through a cross-attention mechanism, thereby aggregating the prior information into the features of the regression model. The processing of data by the FIM includes:

[0039] Step 1: Obtain the intermediate feature X i and the prior feature Z i generated by the multi-scale strategy module;

[0040] Step 2: Perform a reshaping operation on the intermediate feature X i to obtain the flattened sequence X r ;

[0041] Step 3: Map X r to the query vector Q through normalization and linear mapping, and map the prior feature Zi They are respectively projected into a key vector K and a value vector V, and attention weights Attention(Q, K, V) are calculated based on Q, K, and V;

[0042] Q = W Q X r ,

[0043] K = W K Z i

[0044] V = W V Z i

[0045]

[0046] Among them, W Q , W K , W V represent learnable parameters of a bias-free linear projection, and C represents the number of channels.

[0047] Step 4: Multiply the attention weights by the prior features, and perform linear mapping and reshaping operations, and finally add them to the intermediate feature X i to output the fused feature Out.

[0048] FIM can effectively inject prior information into the features of the regression model, thereby improving the denoising performance of the model, realizing the deep interaction between multi-scale feature representation and the intermediate features of the regression model, enabling prior information to dynamically guide the feature extraction process, and making the model show stronger robustness and generalization ability in complex noise scenarios, providing an important guarantee for high-quality denoising results.

[0049] Furthermore, as Figure 5 shown, the present invention adopts a segmented training strategy, including two-stage training and one-step testing, and ensures that the generator can generate an output highly consistent with the real prior features through an adversarial learning mechanism, thereby further improving the quality and denoising performance of the prior features.

[0050] Specifically, the first stage is as Figure 5As shown in (a), training is performed using both low-dose CT images and full-dose CT images. The input is divided into two branches: the first branch is similar to a conventional regression network, and the low-dose CT image generates a prediction result through the regression network; the second branch uses the full-dose CT image to generate prior features through a prior feature encoder. FIM integrates the prior features of the second branch into the regression network of the first branch. This stage mainly jointly trains the regression denoising module and the PFE. In the first-stage training, the PFE and the regression denoising module achieve synergy through joint optimization: the encoder provides high-quality prior features for the regression denoising module to improve the denoising effect, and at the same time, the regression denoising module guides the encoder to generate more discriminative prior features through its optimization process.

[0051] The second stage is as shown in Figure 5 (b). Training is performed using low-dose CT images, and the parameters of the prior feature encoder are not updated. Its output is only used to provide a supervision label for the generative adversarial network. The prior feature generative adversarial network learns to generate low-dimensional features consistent with the output of the prior feature encoder in the first stage and integrates the generated features into the regression network, thereby getting rid of the dependence on full-dose CT images for prior feature generation.

[0052] The testing stage is as shown in Figure 5 (c). Prior features are generated through the prior feature generative adversarial network and integrated with the output of the regression network to finally generate high-quality denoising results.

[0053] In the specific training process, the generator G and the discriminator D are updated through an alternating optimization method: fix the parameters of one network and update the parameters of the other network until the model converges.

[0054] The input of the multi-scale strategy module is different in different training stages: in the first stage of training, the input is the output Z of the prior feature encoder; in the second stage of training, the input is the output Z′ of the PF-GAN generator. Through multiple downsampling operations, prior features Z1, Z2, Z3, and Z4 of different scales are generated, and the multi-scale strategy module captures context information at different abstraction levels respectively.

[0055] Finally, it should be noted that the above only describes some embodiments of the present invention. For those skilled in the art, various changes, modifications, substitutions, and deformations of these embodiments can be conceived without departing from the principle and spirit of the present invention. The protection scope of the present invention is defined by the appended claims and their equivalents, and the above actions should all be covered within the protection scope of the present invention.

Claims

1. A low-dose CT image denoising method based on dual-stream feature fusion, characterized in that Including: S1. Construct a denoising network for low-dose CT images based on dual-stream feature fusion; S2. Train the denoising network for low-dose CT images based on dual-stream feature fusion to obtain a trained CT image denoising model; The training process adopts a segmented training strategy, and the training data includes full-dose CT images and low-dose CT images; S3. Obtain the low-dose CT image to be processed, and use the CT image denoising model to process the low-dose CT image to obtain a denoised low-dose CT image; The denoising network for low-dose CT images based on dual-stream feature fusion includes: a generative feature enhancement module, a multi-scale strategy module, a regression denoising module, and a feature fusion module; among them, the generative feature enhancement module is used to generate high-quality prior features, the multi-scale strategy module is used to generate prior features of different scales according to the high-quality prior features, the regression denoising module is used to extract global features from the low-dose CT image to be processed for denoising, and the feature fusion module is used to perform deep fusion with the prior features to obtain a denoised low-dose CT image.

2. The low-dose CT image denoising method based on dual-stream feature fusion according to claim 1, wherein The generative feature enhancement module includes: a prior feature encoder, a prior feature generative adversarial network; among them, the prior feature encoder is used to generate real prior features; the prior feature generative adversarial network takes the real prior features as a reference and generates high-quality prior features according to the low-dose CT image.

3. The low-dose CT image denoising method based on dual-stream feature fusion according to claim 2, wherein The processing of the prior feature encoder for data includes: reallocating the spatial dimension information of the full-dose CT image to the channel dimension through Pixel-Unshuffle and 3×3 convolution to obtain channel-reallocated features; using 5 residual blocks to further mine the channel-reallocated features to obtain deep context features, where the residual block contains two 3×3 convolutional layers and one ReLU activation function layer, and the ReLU activation function layer is set between the two convolutional layers; performing 3×3 convolution, global average pooling, and three reshaping-linear mapping operations on the deep context features to obtain real prior features.

4. The low-dose CT image denoising method based on dual-stream feature fusion according to claim 1, wherein The regression denoising module adopts an "encoder-decoder" architecture. After the low-dose CT image to be processed enters the regression denoising module, it is processed by a 3×3 convolution and a ReLu activation function, then processed by the encoder and decoder, and then processed by a 3×3 convolution and a ReLu activation function, and added element by element to the low-dose CT image to be processed to obtain a denoised low-dose CT image.

5. The low-dose CT image denoising method based on dual-stream feature fusion according to claim 3, wherein The encoder is composed of four 5×5 convolutions, a feature fusion module is set before each convolution, and a ReLu activation function is set after each convolution; the decoder is composed of four 5×5 transposed convolutions, a feature fusion module is set before each transposed convolution, and a ReLu activation function is set after each transposed convolution.

6. The low-dose CT image denoising method based on dual-stream feature fusion according to claim 4, characterized in that The processing of the feature fusion module for data includes: Step 1. Obtain the intermediate feature and the high-quality prior feature, and the intermediate feature is the output of the previous stage of the feature fusion module; Step 2. Perform a reshaping operation on the intermediate feature to obtain a flattened sequence; Step 3: Map it to a query vector through normalization and linear mapping, project the prior features into key vectors and value vectors respectively, and calculate the attention weights according to the query vector, key vector, and value vector; Step 4: Multiply the attention weights by the prior features, perform linear mapping and reshaping operations, and finally add them to the intermediate features to output the fused features.

7. The low-dose CT image denoising method based on dual-stream feature fusion according to claim 1, wherein The segmented training includes two-stage training and one-step testing.

8. The low-dose CT image denoising method based on dual-stream feature fusion according to claim 6, wherein In the first stage of training, the input of the multi-scale policy module is the true prior features; In the second stage of training, the input of the multi-scale policy module is the high-quality prior features.

Images