Low-dose CT image denoising method and device based on flow model

Abstract

The invention relates to the field of image denoising, in particular to a low-dose CT image denoising method and device based on a flow model. According to the method, a flow model used for denoising a low-dose CT image is mainly constructed, the flow model is composed of a set number of flow blocks, and each flow block is composed of a Haar layer and a set number of reversible neural network flow blocks; according to the method, the reverse process of the reversible neural network flow block is optimized, so that the flow model can effectively extract clean main structure information from a low-dose CT image, and meanwhile, the main structure information is systematically separated from specific detail textures of the image in a coarse-to-fine mode; according to the method, the problem caused by the ill-conditioned property of the denoising problem of the low-dose computed tomography CT image is relieved through the minimum loss function, and meanwhile the interpretability of the low-dose CT image is enhanced.

Description

Technical Field

[0001] The invention relates to the field of image denoising, and in particular to a low-dose CT image denoising method and device based on a flow model. Background Art

[0002] Low-dose CT scanning technology achieves the purpose of reducing radiation dose in CT examinations by reducing the X-ray tube current, and is widely used in clinical examinations and diagnosis. Although low-dose CT scanning technology has many clinical advantages, reducing the radiation dose of X-rays will cause more noise in the image, thereby reducing the contrast and clarity of the image.

[0003] Post-processing algorithms for low-dose CT images do not rely on projection data, but operate directly on reconstructed CT images. Most denoising methods based on CNN and encoder-decoder architectures rely on mean squared error to quantify the difference between denoised images and normal-dose CT images. This approach usually results in trained denoising networks that tend to predict the average value of the image. Despite showing satisfactory performance on some objective metrics, the generated images often appear blurry, overly smoothed, and lack texture details. Unsupervised generative adversarial networks (GANs) have great potential in generating realistic medical images. However, existing GAN frameworks still face some unresolved issues, including the challenge of distinguishing noise from artifacts, limited noise description capabilities, and instability during network training. Transformers use self-attention mechanisms to capture the interactions between global environments, thereby achieving robust performance. However, model training using mean squared error loss functions may lead to over-smoothing of images.

[0004] Many studies have utilized flow-based models for image restoration. NoiseFlow is a noise generation model based on conditional normalized flow, which aims to capture various noise components in the camera imaging pipeline. It uses latent variable sampling to represent the distribution of actual noise. However, the model's reliance on clean images as conditional priors prevents it from achieving full reversibility between clean and noisy images. SRFlow uses normalized flow to achieve perceptual image super-resolution. Although it has achieved certain results, it has a large number of parameters and slow inference speed. InvDN divides the noisy image into low-frequency and second components, discards the high-frequency part, and converts the noisy input into a low-resolution clear image and a latent representation containing noise, which may lead to information loss and over-smoothing. FDN regards image denoising as a task of learning distribution and disentanglement, and directly separates the noise in the latent space through normalized flow, resulting in a more complex network. FINO uses a reversible network to decompose image content and noise components in the latent space and reconstruct them in the image space to achieve lossless noise disentanglement, but this introduces a more complex dual model structure. Summary of the invention

[0005] In order to solve the problem in the background art when processing low-dose CT image noise, the present invention proposes a low-dose CT image denoising method and device based on a flow model.

[0006] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0007] On the one hand, the present invention proposes a low-dose CT image denoising method based on a flow model, characterized in that the low-dose CT image to be denoised needs to be input into the flow model, and the flow model outputs the denoised CT image;

[0008] The flow model is composed of a set number of flow blocks, each of which is composed of a Haar layer and a set number of reversible neural network flow blocks;

[0009] The optimization of the flow model specifically includes optimizing the reverse process of the reversible neural network flow block and optimizing the flow model by minimizing the loss function;

[0010] Establishing the minimization loss function requires that the pixel loss between the denoised image and the normal dose CT image Pixel loss between low-dose low-resolution images and normal-dose low-resolution images And the NLL loss in the reversible neural network flow block calculation process

[0011] Preferably, the low-dose CT image refers to a CT image whose radiation dose is 25% of the normal-dose CT image.

[0012] Preferably, a transformer-based attention denoising layer is used to optimize the reverse process of the reversible neural network flow block, and the attention denoising layer is specifically divided into frequency attention and spatial attention.

[0013] Further preferably, the optimization of the reverse process is specifically as follows: mapping the low-dose CT image x to Gaussian space y=f(x), using g(z) to estimate the deviation δ between the clean image and the noisy image, and using the corresponding bar flow block to map z to the image domain. In this process, performing operations on the low-resolution image obtained by downsampling the Haar layer is actually equivalent to performing operations on z; therefore, in the reverse process, multiple operations are performed on the low-resolution image obtained by downsampling the Haar layer, and these operations involve the attention denoising layer integrated into the reverse process.

[0014] Preferably, the paired low-dose CT image and the normal-dose CT image are normalized and input into the flow model to obtain the denoised image, and the pixel loss between the denoised image and the normal-dose CT image is calculated.

[0015] Further preferably, the normalization specifically refers to unifying the paired low-dose CT images and normal-dose CT images into 512×512 pixels; the low-dose CT image is recorded as x, the normal-dose CT image is recorded as y, and the paired low-dose CT image and normal-dose CT image are recorded as an image pair (x, y).

[0016] Preferably, the low-dose CT image and the normal-dose CT image are input into the flow block for Haar layer downsampling to obtain the low-dose low-resolution image and the normal-dose low-resolution image, and the pixel loss between the low-dose low-resolution image and the normal-dose low-resolution image is calculated.

[0017] Further preferably, the Haar layer downsampling is specifically performed by using a convolution kernel with a step size of 2 to perform Haar transform on the input image, and the original image size is (B, 1, H, W), where B is the batch size, 1 is the number of image channels, H is the image height, and W is the image width. After the transformation, the local area of ​​the original low-dose CT image is mapped to 4 frequency components, and the size of the original low-dose CT image is reduced to 1 / 4 of the original size, obtaining an image containing a first component and a second component, with a size of (B, 4, H / 4, W / 4), wherein the first component is the main structural information of the image, and the second component is the specific detail texture of the image.

[0018] Preferably, the low-dose low-resolution image and the normal-dose low-resolution image are calculated using a reversible neural network flow block to obtain the NLL loss in the calculation process of the reversible neural network flow block.

[0019] Further preferably, the core of the reversible neural network flow block in step S4 is an affine coupling layer, in which the low-dose low-resolution image is input, divided into two parts x1 and x2, and forward calculation and reverse calculation are performed, and the forward calculation process is as follows:

[0020]

[0021] The reverse calculation process is as follows:

[0022]

[0023] Further preferably, the reversible neural network flow block calculation process is used to obtain the The specific process is as follows: The target density p can be obtained by calculating the Jacobian determinant. y|a The expression form of , thereby deriving the log-likelihood function, and taking the negative logarithm of the log-likelihood function as the loss function The calculation formula is as follows:

[0024]

[0025] Where: is the conditional distribution of the normal dose CT image y; f θ is the flow model; y is the normal dose CT image; z is a variable in the latent space; a * is the information component a of the low-dose CT image x The information component a of the normal dose CT image y h is the output of z in the reversible layer of the flow model after reversible mapping; p z is the h from a standard normal distribution.

[0026] Further preferably, the variable z in the latent space is set to follow the Gaussian distribution p z (z)~N(0,I), using the variable transformation formula to derive the conditional distribution of the normal dose CT image y from the Gaussian distribution of the variable z in the latent space The specific formula is:

[0027]

[0028] Where: is the conditional distribution of the normal dose CT image y, p z is the distribution of variables in the latent space, f θ is the flow model, y is the normal dose CT image, z is the variable in the latent space, a * is the information component a of the low-dose CT image x The information component a of the normal dose CT image y The main information components that are consistent in the .

[0029] Preferably, using the Said and stated A minimization loss function is established, wherein the minimization loss function is specifically:

[0030]

[0031] Wherein: y is the normal dose image; is the denoised image; x lr is the low-dose and low-resolution image input; lr is the normal dose low resolution image; The final result obtained by minimizing the loss function; is the pixel loss; is the pixel loss between the denoised image and the normal-dose CT image; is the pixel loss between the low-dose low-resolution image and the normal-dose low-resolution image; The NLL loss during the calculation of the reversible neural network flow block λ1, λ2λ3 are the hyperparameters of loss weights. Specifically, λ1, λ2, λ3 are set to 16, 1, 1 respectively.

[0032] On the other hand, the present invention proposes a low-dose CT image denoising device based on a flow model, comprising at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a low-dose CT image denoising method based on a flow model.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] Compared with the prior art, the present invention proposes a low-dose CT image denoising method and device based on a flow model, which constructs a flow model, wherein the flow model is composed of a set number of flow blocks, each of which is composed of a Haar layer and a set number of reversible neural network flow blocks; the present invention optimizes the reverse process of the reversible neural network flow block so that the flow model can effectively extract clean main structural information from the low-dose CT image, and systematically separate the main structural information from the specific detail texture of the image in a coarse-to-fine manner; the present invention also alleviates the problems caused by the pathological nature of the low-dose computed tomography CT image denoising problem by minimizing the loss function, and at the same time enhances the interpretability of the low-dose CT image. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is an overall architecture diagram of the low-dose CT image flow model in Example 1;

[0036] Figure 2 is a flow chart of the reversible neural network flow block in Example 1;

[0037] Figure 3 This is a flowchart of the frequency attention layer in the attention module of Example 1.

[0038] Figure 4 This is a flowchart of the spatial attention layer in the attention module of Example 1. DETAILED DESCRIPTION

[0039] The present invention is further described in detail below in conjunction with the examples and specific implementation methods. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following examples, and all technologies realized based on the content of the present invention belong to the scope of the present invention.

[0040] Example 1

[0041] The present invention proposes a low-dose CT image denoising method based on a flow model, which is characterized in that a low-dose CT image to be denoised needs to be input into the flow model, and the flow model outputs a denoised CT image;

[0042] The flow model is composed of a set number of flow blocks, each of which is composed of a Haar layer and a set number of reversible neural network flow blocks;

[0043] The optimization of the flow model specifically includes optimizing the reverse process of the reversible neural network flow block and optimizing the flow model by minimizing the loss function;

[0044] Establishing the minimization loss function requires that the pixel loss between the denoised image and the normal dose CT image Pixel loss between low-dose low-resolution images and normal-dose low-resolution images And the NLL loss in the reversible neural network flow block calculation process

[0045] The low-dose CT image refers to a CT image whose radiation dose is 25% of the normal-dose CT image.

[0046] The reverse process of the reversible neural network flow block is optimized using a transformer-based attention denoising layer, where the attention denoising layer is specifically divided into frequency attention and spatial attention.

[0047] The optimization of the reverse process is specifically to map the low-dose CT image x to the Gaussian space y=f(x), use g(z) to estimate the deviation δ between the clean image and the noisy image, and use the corresponding bar flow block to map z to the image domain. In this process, the operation performed on the low-resolution image obtained by downsampling the Haar layer is actually equivalent to the operation performed on z; therefore, in the reverse process, multiple operations are performed on the low-resolution image obtained by downsampling the Haar layer, and these operations involve the attention denoising layer integrated into the reverse process.

[0048] The paired low-dose CT image and the normal-dose CT image are normalized and input into the flow model in the flow model to obtain a denoised image, and the pixel loss between the denoised image and the normal-dose CT image is calculated.

[0049] The normalization specifically refers to unifying the paired low-dose CT images and normal-dose CT images into 512×512 pixels; the low-dose CT image is recorded as x, the normal-dose CT image is recorded as y, and the paired low-dose CT image and normal-dose CT image are recorded as an image pair (x, y).

[0050] The low-dose CT image and the normal-dose CT image are input into the flow block for Haar layer downsampling to obtain the low-dose low-resolution image and the normal-dose low-resolution image, and the pixel loss between the low-dose low-resolution image and the normal-dose low-resolution image is calculated.

[0051] The Haar layer downsampling is specifically performed by using a convolution kernel with a step size of 2 to perform Haar transform on the input image. The original image size is (B, 1, H, W), where B is the batch size, 1 is the number of image channels, H is the image height, and W is the image width. After the transformation, the local area of ​​the original low-dose CT image is mapped to 4 frequency components, and the size of the original low-dose CT image is reduced to 1 / 4 of the original size, obtaining an image containing a first component and a second component, with a size of (B, 4, H / 4, W / 4), wherein the first component is the main structural information of the image, and the second component is the specific detail texture of the image.

[0052] The low-dose low-resolution image and the normal-dose low-resolution image are calculated using a reversible neural network flow block, and the NLL loss in the calculation process of the reversible neural network flow block is obtained.

[0053] The core of the reversible neural network flow block in step S4 is the affine coupling layer, in which the low-dose low-resolution image is input, divided into two parts x1 and x2, and forward calculation and reverse calculation are performed. The forward calculation process is as follows:

[0054]

[0055] The reverse calculation process is as follows:

[0056]

[0057] According to the reversible neural network flow block calculation process, the The specific process is as follows: The target density p can be obtained by calculating the Jacobian determinant. y|a The expression form of , thereby deriving the log-likelihood function, and taking the negative logarithm of the log-likelihood function as the loss function The calculation formula is as follows:

[0058]

[0059] Where: is the conditional distribution of the normal dose CT image y; f θ is the flow model; y is the normal dose CT image; z is a variable in the latent space; a * is the information component a of the low-dose CT image x The information component a of the normal dose CT image y h is the output of z in the reversible layer of the flow model after reversible mapping; p z is the h from a standard normal distribution.

[0060] Assume that the variable z in the latent space follows a Gaussian distribution p z (z)~N(0,I), using the variable transformation formula to derive the conditional distribution of the normal dose CT image y from the Gaussian distribution of the variable z in the latent space The specific formula is:

[0061]

[0062] Where: is the conditional distribution of the normal dose CT image y, p z is the distribution of variables in the latent space, f θ is the flow model, y is the normal dose CT image, z is the variable in the latent space, a * is the information component a of the low-dose CT image x The information component a of the normal dose CT image y The main information components that are consistent in the .

[0063] The normal dose image and the latent space variable also satisfy a bidirectional mapping relationship, which is specifically verified as follows:

[0064] S1. Decompose the image pair (x, y) through a reversible bijective transformation d to obtain the main component h, the information component a and the high-frequency details. The process is as follows:

[0065]

[0066] Where: x is the low-dose image, y is the normal-dose image, d is the reversible bijective change, h x is the main component of the low-dose CT image, h y is the main component of the normal dose CT image, a x is the information component of the low-dose CT image, a y is the information component of the normal dose CT image;

[0067] S2, the information component a of the low-dose CT image x The information component a of the normal dose CT image y The common and consistent main information component in the * , and analyze the main information components to obtain the normal dose CT image distribution, as follows:

[0068]

[0069] Where: p represents the distribution, y is the normal dose image, h y is the main component of the normal dose CT image, a y is the information component of the normal dose CT image;

[0070] At the same time, to ensure a x ,a y The consistency between the p(a y ) is expressed as the Dirac distribution δ(a y -a * ), the p(a y ) is approximated by an isotropic multivariate Gaussian distribution:

[0071]

[0072] Where: p represents the distribution, y is the normal dose image, h x is the main component of the low-dose CT image, h y is the main component of the normal dose CT image, a x is the information component of the low-dose CT image, a y is the information component of the normal dose CT image;

[0073] S3, mapping the normal dose CT image to a standard multivariate Gaussian distribution z~N(0,I), specifically:

[0074]

[0075] Where: Σ represents the covariance matrix, p represents the distribution, y is the normal dose image, a yis the information component of the normal dose CT image, a * is the information component a of the low-dose CT image x The information component a of the normal dose CT image y The main information components that are common and consistent in the

[0076] S4. Implementing a bidirectional mapping from the normal dose CT image y to the latent variable z through the flow model. The specific process is as follows:

[0077] z=f θ (y; a * )

[0078]

[0079] Where: f θ is the flow model, y is the normal dose CT image, z is the variable in the latent space, a * is the information component a of the low-dose CT image x The information component a of the normal dose CT image y The main information components are common and consistent in the .

[0080] Use the Said and stated A minimization loss function is established, wherein the minimization loss function is specifically:

[0081]

[0082] Wherein: y is the normal dose image; is the denoised image; x lr is the low-dose and low-resolution image input; lr is the normal dose low resolution image; The final result obtained by minimizing the loss function; is the pixel loss; is the pixel loss between the denoised image and the normal-dose CT image; is the pixel loss between the low-dose low-resolution image and the normal-dose low-resolution image; The NLL loss during the calculation of the reversible neural network flow block λ1, λ2λ3 are the hyperparameters of loss weights. Specifically, λ1, λ2, λ3 are set to 16, 1, 1 respectively.

[0083] Example 2

[0084] A low-dose CT image denoising device based on a flow model comprises at least one processor and a memory connected to the at least one processor in communication; the memory stores instructions executable by the at least one processor, the instructions are executed by the at least one processor so that the at least one processor can execute one of the aforementioned embodiments. The input and output interface may include a display, a keyboard, a mouse, and a USB interface for inputting and outputting data; the power supply is used to provide power to the electronic device.

[0085] Those skilled in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), disks or optical disks, etc. Various media that can store program codes.

[0086] When the above-mentioned integrated unit of the present invention is implemented in the form of a software functional unit and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

Claims

1. A low-dose CT image denoising method based on a flow model, characterized in that: Specifically, the low-dose CT image to be denoised needs to be input into the flow model, and the flow model outputs the denoised CT image; The flow model is composed of a set number of flow blocks, each of which is composed of a Haar layer and a set number of reversible neural network flow blocks; The optimization of the flow model specifically includes optimizing the reverse process of the reversible neural network flow block and optimizing the flow model by minimizing the loss function; Establishing the minimization loss function requires the pixel loss between the denoised image and the normal dose CT image Pixel loss between low-dose low-resolution images and normal-dose low-resolution images And the NLL loss in the reversible neural network flow block calculation process 2. The low-dose CT image denoising method based on flow model according to claim 1, characterized in that: The low-dose CT image refers to a CT image whose radiation dose is 25% of the normal-dose CT image.

3. A low-dose CT image denoising method based on a flow model as claimed in claim 1, characterized in that: The reverse process of the reversible neural network flow block is optimized using a transformer-based attention denoising layer, where the attention denoising layer is specifically divided into frequency attention and spatial attention.

4. The low-dose CT image denoising method based on flow model according to claim 1, characterized in that: The paired low-dose CT image and the normal-dose CT image are normalized and input into the flow model to obtain a denoised image, and the pixel loss between the denoised image and the normal-dose CT image is calculated.

5. The low-dose CT image denoising method based on flow model as claimed in claim 3, characterized in that: The normalization specifically refers to unifying the paired low-dose CT images and normal-dose CT images into 512×512 pixels; the low-dose CT image is recorded as x, the normal-dose CT image is recorded as y, and the paired low-dose CT image and normal-dose CT image are recorded as an image pair (x, y).

6. The low-dose CT image denoising method based on flow model according to claim 1, characterized in that: The low-dose CT image and the normal-dose CT image are input into the flow block for Haar layer downsampling to obtain the low-dose low-resolution image and the normal-dose low-resolution image, and the pixel loss between the low-dose low-resolution image and the normal-dose low-resolution image is calculated.

7. A low-dose CT image denoising method based on a flow model as claimed in claim 1, characterized in that: The low-dose low-resolution image and the normal-dose low-resolution image are calculated using a reversible neural network flow block, and the NLL loss in the calculation process of the reversible neural network flow block is obtained.

8. The low-dose CT image denoising method based on flow model according to claim 1, characterized in that: Use the Said and stated A minimization loss function is established, wherein the minimization loss function is specifically: Wherein: y is the normal dose image; is the denoised image; x lr is the low-dose and low-resolution image input; lr is the normal dose low resolution image; The final result obtained by minimizing the loss function; is the pixel loss; is the pixel loss between the denoised image and the normal-dose CT image; is the pixel loss between the low-dose low-resolution image and the normal-dose low-resolution image; The NLL loss during the calculation of the reversible neural network flow block λ1, λ2λ3 are the hyperparameters of loss weights. Specifically, λ1, λ2, λ3 are set to 16, 1, 1 respectively.

9. A low-dose CT image denoising device based on a flow model, characterized in that: comprising at least one processor, and a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a low-dose CT image denoising method based on a flow model according to any one of claims 1 to 8.

Images