Contrastive learning based low dose pet image denoising method

Abstract

The application discloses a low-dose PET image denoising method based on contrast learning, and uses the information of a low-dose PET image and a standard-dose PET image as negative samples and positive samples respectively. CR ensures that the denoised PET image is closer to the standard-dose PET image and farther away from the low-dose PET image in the representation space. In addition, considering the parameter and performance balance, a dense denoising network based on a class automatic encoder framework is developed, wherein an adaptive mixing Mixup operation can adaptively realize information flow and expand a receptive field in high and low resolution spaces of upsampling and downsampling, so that the conversion capability of the network is improved.

Description

Technical Field

[0001] This invention belongs to the field of biomedical image denoising technology, specifically relating to a low-dose PET image denoising method based on contrastive learning. Background Technology

[0002] Positron emission tomography (PET) is a nuclear medicine technique used for in vivo functional imaging. It can provide functional information about organs and lesions at the molecular level and plays an irreplaceable role in the diagnosis and treatment of heart disease, brain disease, and malignant tumors.

[0003] Computer vision tasks often involve image denoising, aiming to remove noise from damaged images and restore a true image. Denoising in medical imaging processing carries particular significance, as noise can hinder disease diagnosis and influence subsequent clinical decisions. Positron emission tomography (PET) is a molecular imaging modality used to provide metabolic and functional information. Denoising algorithms for this modality aim to overcome the trade-offs between scan time, radiation intensity, and image quality. In PET scans, it is also desirable to reconstruct a perfect image with a small number of coincidence events, minimizing ionizing radiation exposure.

[0004] A number of methods have been proposed to improve the image quality of PET images. These image restoration methods are implemented at various stages, including preprocessing of raw data before reconstruction, design of reconstruction algorithms, and post-processing after reconstruction. Image post-processing is performed directly on low-dose PET images and can be easily integrated into existing clinical procedures. Therefore, many researchers focus on addressing the denoising problem of low-dose PET images. In the field of PET image denoising, traditional image processing algorithms include NLM, BM3D, and Diffusion Filters. While these algorithms can reduce noise to varying degrees, they may lead to over-smoothing of the denoised images.

[0005] Recently, deep learning has demonstrated remarkable capabilities in PET imaging tasks. In the field of low-dose PET denoising, Sano et al. modified U-Net to reconstruct standard-dose PET images from low-dose PET images; Kim et al. optimized DnCNN to perform PET image denoising; Zhou et al. proposed a unified motion correction and denoising adversarial network (DPET) to simultaneously perform denoising and motion estimation on low-dose PET; in addition, there are some low-dose PET denoising works based on CycleGAN and Wasserstein GAN.

[0006] The methods described above have demonstrated their effectiveness in PET image denoising, but most focus on optimizing the network structure or designing different loss functions. They suffer from the following problems:

[0007] (1) It mainly uses the information of positive samples as a guide for denoising, while the information of negative samples is not fully utilized. Objectively, positive denoising alone results in poor denoising performance.

[0008] (2) Most existing methods typically use standard dose images (Ground Truth) as positive samples to guide the training of denoising networks using L1 / L2-based image reconstruction loss without any regularization. However, image reconstruction loss alone cannot effectively handle the details of the image and may lead to image color distortion.

[0009] (3) Most of the work focuses on increasing the depth and width of the denoising network, resulting in huge computational and memory requirements. Summary of the Invention

[0010] In view of this, this invention proposes a contrastive regularization (CR) based on contrastive learning, which uses information from low-dose PET images and standard-dose PET images as negative and positive samples, respectively. CR ensures that the denoised PET images are closer to standard-dose PET images and further away from low-dose PET images in the representation space. Furthermore, considering the balance between parameters and performance, a dense denoising network based on an autoencoder-like framework is developed, where an adaptive mixing operation can adaptively facilitate information flow and expand the receptive field in the upsampling and downsampling high- and low-resolution spaces to improve the network's conversion capability.

[0011] This invention is achieved through the following technical solution:

[0012] This invention discloses a low-dose PET image denoising method based on contrastive learning, comprising the following steps:

[0013] 1) Obtain the simulated standard dose Sinogram projection data, and use the Poisson distribution to downsample the standard dose Sinogram projection data to obtain low dose Sinogram projection data.

[0014] 2) The standard dose Sinogram projection data is reconstructed using the OSEM algorithm to obtain the corresponding standard dose PET image data; the low dose Sinogram projection data is reconstructed using the FBP algorithm to obtain the corresponding low dose PET image data.

[0015] 3) Obtain a large number of samples according to step 2), each sample including low-dose PET image data and standard-dose PET image data, and divide all samples into training set, validation set and test set;

[0016] 4) To balance denoising performance and model parameter count, a lightweight PET basic denoising network based on an autoencoder-like architecture was built. Low-dose PET image data was input into the denoising network, and the model output denoised PET images. To better recover the images, a new contrastive learning loss function was designed based on the method of constructing positive and negative samples using contrastive learning. Low-dose PET images and standard-dose PET images were used as positive and negative samples, respectively, and denoised PET images were used as anchor samples.

[0017] 5) During the training phase, input the low-dose PET image data from the training set into the network of step 4), and train it based on the contrastive learning loss function of step 4. With the principle of minimizing the loss function, repeat the forward propagation and backward differentiation, and continuously update the parameters until the value of the loss function is small enough and the model converges. At the same time, input the low-dose PET image data from the validation set into the model for validation, supervise the effectiveness of the model training, and adjust the parameters in a timely manner during the training phase.

[0018] 6) During the testing phase, low-dose PET image data from the test set are input into the trained PET base denoising network to directly obtain high-quality PET images.

[0019] As a further improvement, the downsampling in step 1) of the present invention is specifically implemented as follows: For standard dose Sinogram projection data, firstly, a random number matrix of the same size as its Sinogram matrix is ​​generated based on the Poisson distribution using Python's built-in library functions. By setting different normalization coefficients, the mean of the random number matrix can be set to different sizes. Then, through matrix multiplication, the mean of the Sinogram matrix is ​​reduced to one-nth of the original value, thereby obtaining low-dose Sinogram projection data, where n is the downsampling factor.

[0020] As a further improvement, the specific structure of the PET basic denoising network in step 4) of this invention is as follows: Inspired by the FABlock proposed in "FFA-Net: Feature Fusion Attention Network for Single Image Dehazing", four FABlocks are used as the basic blocks of the proposed denoising network. First, a 4x downsampling operation is used to enable the FABlocks to learn the feature representation in the low-resolution space. Then, a corresponding 4x upsampling operation and a convolution operation are used to recover the high-resolution image. Low-level features (such as edges and contours) are usually captured by the shallow layers of convolutional neural networks. However, as the number of layers increases, shallow feature pairs gradually degrade. To avoid the loss of shallow features, the model adds an adaptive mixing operation between the downsampling layer and the upsampling layer. This operation can be expressed as:

[0021] f ↑2 =Mix(f ↓1 f ↑1 )=σ(θ1)*f ↓1 +(1-σ(θ1))*f ↑1 ,

[0022] f ↑ =Mix(f ↓2 f ↑2 )=σ(θ2)*f ↓2 +(1-σ(θ2))*f ↑2 ,

[0023] Where f ↓i and f ↑i These are the feature maps of the i-th downsampling layer and the upsampling layer, respectively, f ↑ This is the final output, σ(θ) i ), i = 1, 2 is the i-th learnable factor that is fused from the i-th downsampling layer and the i-th upsampling layer and learned through the attention mechanism.

[0024] As a further improvement, step 4) of this invention uses the contrastive learning loss function, taking standard dose and low dose PET image data as positive and negative samples respectively, and the denoised PET image output by the PET basic denoising network as anchor samples. Contrast regularization (CR) is added to the L1 Loss to form ContrastLoss. For the latent feature space, common intermediate features are selected from the same fixed pre-trained model VGG19. End-to-end PET image denoising utilizes the contrastive learning loss function, expressed as:

[0025]

[0026] I represents a low-dose PET image, J represents a standard-dose PET image, φ(I, w) is the basic PET denoising network, w is the network model parameter, ||J-φ(I, w)|| is the data fidelity term image reconstruction loss L1 Loss, the β parameter is used to balance the data fidelity term and the regularization term, and ρ(φ(I, w)) is the regularization term, using contrastive regularization (CR), expressed as:

[0027]

[0028] G i ω represents the i-th hidden layer feature extracted from the pre-trained VGG19, specifically including only the 1st, 3rd, 5th, 9th, and 13th convolutional layers of VGG19 as the feature extraction part. i The weight parameters are respectively 1. D(x, y) represents the L1 distance between x and y.

[0029] As a further improvement, the specific process of training the network model in step 5) of this invention is as follows:

[0030] 6.1 Use Kaiming initialization to initialize the parameters in the denoising model and set the parameters of each layer;

[0031] 6.2 Input the low-dose PET image data from the training set into the denoising network model, train the built PET basic denoising network, and finally calculate the output of each layer through forward propagation to obtain the final output anchor sample of the network model.

[0032] 6.3 VGG19 extracts features from positive and negative samples and anchor samples respectively. Based on the principle of minimizing the loss function and learning the loss function through the above comparison, the partial derivative of the loss function is calculated, gradient backpropagation is performed, and the learnable parameters in the network model are updated through the Adam algorithm.

[0033] 6.4 Repeat steps 6.2 to 6.3 until the parameters of the entire network model converge.

[0034] The beneficial effects of this invention are as follows:

[0035] This invention discloses a contrastive learning-based method for denoising low-dose PET images, specifically a contrastive regularization (CR) approach that uses information from low-dose and full-dose PET images as negative and positive samples, respectively. CR ensures that the denoised PET images are closer to standard-dose PET images and further away from low-dose PET images in the representation space. Furthermore, considering the balance between parameters and performance, a lightweight denoising network based on an autoencoder-like framework is developed. An adaptive mixing operation adaptively facilitates information flow and expands the receptive field in the upsampling and downsampling high- and low-resolution spaces to improve the network's conversion capability. The model's performance has been validated on PET datasets, including both simulated and real datasets. Results show that this strategy outperforms state-of-the-art low-dose PET image denoising algorithms in terms of visual quality and quantitative metrics, and exhibits strong generalization ability. This simple yet effective strategy demonstrates promise for PET image denoising and may have significant clinical implications in the future. Attached Figure Description

[0036] Figure 1 This is a flowchart of the algorithm of the method of the present invention;

[0037] Figure 2 This is a schematic diagram of the structure of the network model CLPD-Net of this invention;

[0038] Figure 3 The low dose of the present invention in different frames 18 A comparative diagram of the denoised image results of the ablation experiments of each module of F-FDG;

[0039] Figure 4 The low dose of the present invention in different frames 18 A comparative diagram of the quality indicators of ablation experiments for each module of F-FDG;

[0040] Figure 5 Low dose for different methods in different frames 18 A comparative diagram of the denoised image results from the F-FDG comparative experiment;

[0041] Figure 6 Low dose for different methods in different frames 18 A comparative diagram of quality indicators in the F-FDG comparative experiment. Detailed Implementation

[0042] To describe the present invention in more detail, the technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0043] This invention relates to a method for denoising low-dose PET images based on contrastive learning. Figure 1 The algorithm flowchart of the method of the present invention includes the following steps:

[0044] 1) Obtain the simulated standard dose Sinogram projection data, and use the Poisson distribution to downsample the standard dose Sinogram projection data to obtain low dose Sinogram projection data.

[0045] 2) The standard dose Sinogram projection data is reconstructed using the OSEM algorithm to obtain the corresponding standard dose PET image data; the low dose Sinogram projection data is reconstructed using the FBP algorithm to obtain the corresponding low dose PET image data.

[0046] 3) Obtain a large number of samples according to step 2), each sample including low-dose PET image data and standard-dose PET image data, and divide all samples into training set, validation set and test set;

[0047] 4) To balance denoising performance and model parameter count, a lightweight PET basic denoising network based on an autoencoder-like architecture was built. Low-dose PET image data was input into the denoising network, and the model output denoised PET images. To better recover the images, a new contrastive learning loss function, ContrastLoss, was designed based on the method of constructing positive and negative samples using contrastive learning. Standard-dose PET images and low-dose PET images were used as positive and negative samples, respectively, and the denoised PET images were used as anchor samples.

[0048] 5) During the training phase, the low-dose PET image data from the training set is input into the PET basic denoising network described in step 4). The network is trained based on the contrastive learning loss function ContrastLoss described in step 4. The forward propagation and backward differentiation are repeated to continuously update the parameters, with the principle of minimizing the loss function, until the value of the loss function is small enough and the model converges. At the same time, the low-dose PET image data from the validation set is input into the model for validation to supervise the effectiveness of the model training and to adjust the parameters in a timely manner during the training phase.

[0049] 6) During the testing phase, low-dose PET image data from the test set are input into the trained PET base denoising network to directly obtain high-quality PET images.

[0050] Specifically, the steps include the following:

[0051] (1) Data collection.

[0052] The GATE toolkit was used to generate the simulation data required for the experiment. Monte Carlo simulation was used to generate standard-dose 3D dynamic sinogram and PET data (OSEM reconstruction). Poisson downsampling was performed on the standard-dose sinogram projection data using a Poisson distribution to obtain low-dose sinogram projection data. The low-dose sinogram projection data was then reconstructed using the FBP algorithm to obtain low-dose PET image data. This process was repeated more than 10 times to obtain a large number of samples, each including both low-dose and standard-dose PET image data.

[0053] (2) Loss function and sample form.

[0054] A contrastive learning loss function is employed, using standard-dose and low-dose PET image data as positive and negative samples, respectively, and the denoised PET image output from the basic PET denoising network as the anchor sample. Contrast regularization (CR) is added to the L1 Loss to form the contrastive learning loss function ContrastLoss. For the latent feature space, common intermediate features are selected from the same fixed pre-trained model VGG19. End-to-end PET image denoising using the contrastive learning loss function can be expressed as:

[0055]

[0056] I represents a low-dose PET image, J represents a standard-dose PET image, φ(I, w) is the basic PET denoising network, w is the network model parameter, ||J-φ(I, w)|| is the data fidelity term for image reconstruction loss (L1 Loss), and the β parameter is used to balance the data fidelity term and the regularization term. ρ(φ(I, w)) is the regularization term, here using contrastive regularization (CR), which can be expressed as:

[0057]

[0058] G i ω represents the i-th hidden layer feature extracted from the pre-trained VGG19, specifically including only the 1st, 3rd, 5th, 9th, and 13th convolutional layers of VGG19 as the feature extraction part. i The weight parameters are respectively 1, D(x,y) represents the L1 distance between x and y.

[0059] (3) Training phase.

[0060] Kaiming initialization was used to initialize the parameters in the denoising model and set the parameters of each layer. Low-dose PET image data from the training set were input into the denoising network model to train it. Finally, the output of each layer was calculated through forward propagation to obtain the final output anchor samples of the network model. Features of positive and negative samples and anchor samples were extracted by VGG19. Based on the principle of minimizing the loss function, the loss function was learned through the contrastive learning method described above. The partial derivative of the loss function was calculated, and the learnable parameters in the network were updated using the Adam algorithm. Forward propagation and backpropagation were repeated to continuously update the parameters until the value of the loss function was sufficiently small and the model converged. The model was trained for 300 epochs to allow the entire network to converge and obtain the final output of the network. In the validation phase, the trained model was validated every 5 epochs to monitor the effectiveness of the model training and to adjust the parameters in a timely manner.

[0061] (4) Testing phase.

[0062] The trained network is used to input low-dose PET images for denoising, and high-quality denoised images are obtained directly.

[0063] The following experiments, based on Monte Carlo simulation data, verify the effectiveness of this invention. The entire algorithm in this embodiment was tested on a PC system with a Core i9-10900k CPU and a GeForce RTX 3080 graphics card (10GB VRAM). In programming, PyTorch 1.10.0, a Python-based platform, was used to build the neural network, allowing for integration with multiple programming environments.

[0064] The simulated tracer is 18 The F-FDG phantom was a 3D brain phantom. The equipment used in the GATE simulation was a 655k brain scanner from Hamamatsu Photonics Co., Ltd., Japan. The simulation scan lasted 40 minutes with 18 time frames. The simulation data was randomly divided into a training set (1200 PETs), a validation set (300 PETs), and a test set (500 PETs).

[0065] Figure 2 The diagram shows the model structure. It includes the FA Block, Upsampling and Downsampling modules, and the Mixup module for adaptive mixing operations.

[0066] Figure 3 The image shows the low-dose performance of different modules of the denoising network in different frames. 18 The comparison of F-FDG reconstruction images, from left to right, shows the reconstruction image of this invention, the reconstruction image of the base module (Base), the reconstruction image of the base model plus adaptive mixing operation (Base+Mixup module), the reconstruction image of the base module plus contrast regularization (Base+CR module), and the ground truth image reconstructed by the OSEM algorithm; from top to bottom, the reconstruction images are frames 2, 8, and 14, respectively. As can be seen from the figures, the reconstruction image of this invention is rich in detail and has low noise, most closely resembling the ground truth image; with each additional module, the reconstruction effect increases with the gradient; therefore, the method of this invention can obtain high-quality PET reconstruction images, even under low-dose conditions, it can still reconstruct high-quality images. Figure 3 Low dose for different methods in different frames 18 The quantitative advantages of this invention can be clearly seen from the comparison metrics (SSIM and PSNR) of the F-FDG reconstructed images.

[0067] Figure 4 The image shows the low-dose performance of different modules of the denoising network in different frames. 18 The quantitative advantages of this invention can be clearly seen from the comparison metrics (SSIM and PSNR) of the F-FDG reconstructed images.

[0068] Figure 5 The figure shows the low-dose results of different methods in different frames. 18The comparison of F-FDG reconstructed images, from left to right, shows the reconstructed image from the FBP-Net algorithm, the reconstructed image from this invention, the reconstructed image from the DeepPET algorithm, the reconstructed image from U-Net, and the ground truth image reconstructed by the OSEM algorithm; from top to bottom, the reconstructed images are from frames 2, 8, and 14, respectively. As can be seen from the figures, the reconstructed image from this invention is rich in detail and has low noise, being closest to the ground truth image; the reconstructed images from U-Net and DeepPET algorithms show some non-existent structures at the edges, and the reconstructed image from FBP-Net is too smooth, losing some detail and making it difficult to handle low-dose reconstruction; therefore, the method of this invention can obtain high-quality PET reconstructed images, even under low-dose conditions, it can still reconstruct high-quality images.

[0069] Figure 6 The figure shows the low-dose results of different methods in different frames. 18 The quantitative advantages of this invention can be clearly seen from the comparison metrics (SSIM and PSNR) of the F-FDG reconstructed images.

[0070] The above description of the embodiments is provided to enable those skilled in the art to understand and apply the present invention. Those skilled in the art can readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made to the present invention by those skilled in the art based on the disclosure thereof should be within the scope of protection of the present invention.

Claims

1. A low-dose PET image denoising method based on contrastive learning, comprising the following steps: 1) Obtain the simulated standard dose Sinogram projection data, and use the Poisson distribution to downsample the standard dose Sinogram projection data to obtain low dose Sinogram projection data. 2) The standard dose Sinogram projection data is reconstructed using the OSEM algorithm to obtain the corresponding standard dose PET image data; the low dose Sinogram projection data is independently reconstructed using the standard FBP algorithm to obtain the corresponding complete low dose PET image data. 3) Obtain a large number of samples according to step 2), each sample including the complete low-dose PET image data and standard-dose PET image data, and divide all samples into training set, validation set and test set; 4) To balance denoising performance and model parameter count, a lightweight PET-specific denoising network based on an autoencoder-like framework and four FABlock basic units was constructed. The network incorporates an adaptive Mixup feature fusion operation between the upsampling and downsampling layers to preserve metabolic hotspots and organ boundary features of the PET images. The complete low-dose PET image data is input into the denoising network, and the model outputs a denoised PET image. To better recover key diagnostic details from the PET images, a novel contrastive learning loss function, ContrastLoss, is designed based on constructing positive and negative samples using contrastive learning. Standard-dose PET images and low-dose PET images are used as positive and negative samples, respectively, and the denoised PET image is used as the anchor sample. ContrastLoss consists of L1 loss and contrastive regularization (CR), and deep features are extracted using a pre-trained VGG19 model for contrastive constraints. 5) During the training phase, the complete low-dose PET image data from the training set is input into the PET-specific denoising network described in step 4). The network is trained based on the contrastive learning loss function ContrastLoss described in step 4. The forward propagation and backward differentiation are repeated to continuously update the parameters, with the principle of minimizing the loss function, until the value of the loss function is small enough and the model converges. At the same time as training, the complete low-dose PET image data from the validation set is input into the model for validation to supervise the effectiveness of the model training and to adjust the parameters in a timely manner during the training phase. 6) During the testing phase, the complete low-dose PET image data from the test set is input into the trained PET-specific denoising network to directly obtain high-quality PET images; The specific structure of the PET basic denoising network in step 4) is as follows: Four FABlocks are used as the basic blocks of the proposed denoising network. First, a 4x downsampling operation is applied to enable the FABlocks to learn feature representations in the low-resolution space. Then, a corresponding 4x upsampling operation and a convolution operation are used to recover the high-resolution image. Low-level features are captured by the shallow layers of the convolutional neural network; however, as the number of layers increases, shallow feature pairs gradually degrade. To avoid the loss of shallow features, an adaptive mixing operation is added between the downsampling and upsampling layers. This operation is expressed as: f ↑2 =Mix(f ↓1 ,f ↑1 )=σ(θ1)*f ↓1 +(1-σ(θ1))*f ↑1 , f ↑ =Mix(f ↓2 ,f ↑2 )=σ(θ2)*f ↓2 +(1-σ(θ2))*f ↑2 , Where f ↓i and f ↑i These are the feature maps of the i-th downsampling layer and the upsampling layer, respectively, f ↑ This is the final output, σ(θ) i ), i = 1, 2 is the i-th learnable factor that is learned through the attention mechanism by fusing the i-th downsampling layer and the i-th upsampling layer.

2. The low-dose PET image denoising method based on contrastive learning according to claim 1, characterized in that: The specific implementation of downsampling in step 1) is as follows: For standard dose Sinogram projection data, firstly, a random number matrix of the same size as the Sinogram matrix is ​​generated based on the Poisson distribution using Python's built-in library functions. By setting different normalization coefficients, the mean of the random number matrix can be set to different sizes. Then, through matrix multiplication, the mean of the Sinogram matrix is ​​reduced to one-nth of the original value, thereby obtaining low-dose Sinogram projection data, where n is the downsampling factor.

3. The low-dose PET image denoising method based on contrastive learning according to claim 1, characterized in that: Step 4) uses the contrastive learning loss function, taking standard-dose and low-dose PET image data as positive and negative samples respectively, and the denoised PET image output from the basic PET denoising network as anchor samples. Contrast regularization (CR) is added to the L1 Loss to form ContrastLoss. For the latent feature space, common intermediate features are selected from the same fixed pre-trained model VGG19. End-to-end PET image denoising utilizes the contrastive learning loss function, expressed as: I represents a low-dose PET image, J represents a standard-dose PET image, φ(I, w) is the basic PET denoising network, w is the network model parameter, ||J-φ(I, w)|| is the data fidelity term image reconstruction loss L1 Loss, the β parameter is used to balance the data fidelity term and the regularization term, and ρ(φ(I, w)) is the regularization term, using contrastive regularization (CR), expressed as: G i ω represents the i-th hidden layer feature extracted from the pre-trained VGG19, specifically including only the 1st, 3rd, 5th, 9th, and 13th convolutional layers of VGG19 as the feature extraction part. i The weight parameters are respectively D(x,y) represents the L1 distance between x and y.

4. The low-dose PET image denoising method based on contrastive learning according to claim 1, characterized in that: The specific training process in step 5) is as follows: 5.1 The parameters in the denoising model are initialized using Kaiming initialization, and the parameters of each layer are set. 5.2 Input the low-dose PET image data from the training set into the denoising network model, train the built PET basic denoising network, and finally calculate the output of each layer through forward propagation to obtain the final output anchor sample of the network model. 5.3 VGG19 extracts features from positive and negative samples and anchor samples respectively. Based on the principle of minimizing the loss function and after learning the loss function through the comparison described above, the partial derivative of the loss function is calculated, gradient backpropagation is performed, and the learnable parameters in the network model are updated through the Adam algorithm. 5.4 Repeat steps 5.2 to 5.3 until the parameters of the entire network model converge.

Images