A method for constructing a PET reconstruction model based on list data and deep learning

Abstract

The application discloses a PET reconstruction model construction method based on list data and deep learning, which combines list mode data and deep learning counting through a domain conversion function, and solves the problem that list mode data is difficult to be directly processed through a deep neural network. Through the domain conversion function, the application can convert list mode data of any format and any storage size from a measurement domain to an image domain, so that an end-to-end deep neural network method can be directly used for processing and learning. The application uses deep learning to reconstruct list mode data, and compared with other methods for reconstructing list mode data, the reconstruction speed is greatly improved; for a list mode MLEM algorithm, it takes tens of seconds to reconstruct a brain image, and the application only needs about one second to complete the reconstruction, so that the application has greater application potential in the fields of real-time reconstruction and short-time frame reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of PET imaging technology, specifically relating to a method for constructing a PET reconstruction model based on list data and deep learning. Background Technology

[0002] In PET imaging, the raw PET data must be reconstructed into images for further study. The most common method for PET image reconstruction is based on orthographic projection, which requires rearranging the raw measured data (list-mode data) into a projected format. However, the conversion from list-mode data to projected data often leads to a reduction in spatial and temporal sampling, inevitably resulting in a decrease in spatial resolution. Furthermore, the reconstruction time for sinusoidal data is roughly constant regardless of the number of counts or the acquisition time, limiting some potential applications of PET images, such as real-time PET imaging and short-frame reconstruction. Simultaneously, due to the increasing complexity of modern PET scanners, the number of response lines may exceed the number of detected events, and projected data inevitably stores some empty cells, which is wasteful of storage media. For example, a high-resolution research PET scan of 11C dynamics for 18 frames, with each frame lasting approximately 5 minutes, will generate 29GB of sinusoidal data, while list-mode data storage would only require 5-6GB of space.

[0003] The literature [Zhang Bin, Shan Baoci, Yun Mingkai, Zhao Shujun. Principle and implementation of PET tomographic reconstruction based on list-mode data [J]. Journal of Biomedical Engineering, 2012, 29(05): 814-819] proposes a PET image reconstruction algorithm based on list-mode data. This PET image reconstruction algorithm can process the annihilation time information one by one, and use the measured crystal physical position information for reconstruction. It can potentially obtain better spatial resolution, and the reconstruction speed is faster for low-count data or short-time frame data, making it more suitable for motion artifact correction. The reconstruction algorithm based on list data is represented by the list-mode MLEM algorithm, but the current algorithm is relatively time-consuming and the image quality needs to be further improved. High-level noise in the image is an urgent problem to be solved.

[0004] Currently, algorithms for direct reconstruction from list-mode data are mainly based on statistical iteration methods, including the aforementioned list-mode MLEM and subset partitioning acceleration algorithms. However, due to the lack of prior knowledge constraints, statistical iteration methods often suffer from high noise and long reconstruction times. Based on this, some scholars have proposed adding prior constraints to iterative methods. For example, the literature [Zhang Z, Ye J, Rose S, et al. Preliminary study of TV-constrained-likelihood-maximization image reconstruction from list-mode TOF-PET data[C] / / 2016 IEEE Nuclear Science Symposium, Medical Imaging Conference and Room-Temperature Semiconductor Detector Workshop(NSS / MIC / RTSD).IEEE,2016] proposes total variation constraints to improve the reconstruction of list-mode data. However, the parameters of these prior constraints often need to be determined manually, and the image quality still needs further improvement; the long reconstruction time remains a problem that urgently needs to be solved.

[0005] In recent years, deep learning methods have shown great application potential in medical image reconstruction. However, most of the deep learning methods currently used in PET image reconstruction start from projection domain data or image-based methods, ignoring the application potential of list data. Summary of the Invention

[0006] In view of the above, the present invention provides a PET reconstruction model construction method based on list data and deep learning, which can achieve high-quality reconstruction of list pattern data, and can achieve good performance in the recovery of structural information and noise suppression. Compared with traditional iterative methods, it significantly shortens the reconstruction time.

[0007] A method for constructing a PET reconstruction model based on list data and deep learning includes the following steps:

[0008] (1) Simulate projection of PET template images to obtain PET list data;

[0009] (2) Repeat step (1) to obtain a large number of samples, and divide all samples into training set, test set and validation set. Each set of samples contains PET list data and corresponding PET template image.

[0010] (3) Construct the ListmodeCNN model for list data reconstruction;

[0011] (4) The ListmodeCNN model is trained using training set samples, with PET list data as input and PET template images as labels;

[0012] (5) Input the PET list data in the test set sample directly into the trained model, and directly reconstruct the output PET image to test the performance of the model. After the test is completed, the model can be used to directly reconstruct the PET list data to obtain the PET image.

[0013] Furthermore, the ListmodeCNN model includes:

[0014] Domain transformation unit, used to transform the input PET list data from the measurement domain to the image domain;

[0015] The encoding / decoding unit is used to reconstruct the PET list data in the image domain to obtain the PET image.

[0016] Furthermore, the domain transformation unit uses a list data back projection algorithm or a list data iterative reconstruction algorithm with a single iteration to transform the PET list data from the measurement domain to the image domain.

[0017] Furthermore, the encoding / decoding unit is based on a U-shaped network design, and its structure includes an encoding unit, a gated attention unit, and a decoding unit, wherein:

[0018] The encoding unit consists of a convolution module C1, a dense module M1, a dense module M2, a downsampling module D1, a dense module M3, a dense module M4, a downsampling module D2, a dense module M5, a dense module M6, a downsampling module D3, a dense module M7, and a dense module M8 connected sequentially from input to output.

[0019] The decoding unit consists of, from input to output, an upsampling module U1, a convolution module C2, a dense module M9, a dense module M10, an upsampling module U2, a convolution module C3, a dense module M11, a dense module M12, an upsampling module U3, a convolution module C4, a dense module M13, a dense module M14, and a convolution module C5 connected in sequence.

[0020] The output of dense module M2 and the input of upsampling module U3 are fused through a gated attention unit, and the resulting output is concatenated with the output of upsampling module U3 as the input of convolutional module C4; the output of dense module M4 and the input of upsampling module U2 are fused through a gated attention unit, and the resulting output is concatenated with the output of upsampling module U2 as the input of convolutional module C3; the output of dense module M6 and the input of upsampling module U1 are fused through a gated attention unit, and the resulting output is concatenated with the output of upsampling module U1 as the input of convolutional module C2; ​​the output of convolutional module C5 is superimposed with the input of convolutional module C1 as the output of the encoding / decoding unit, which is the final output of the model.

[0021] Furthermore, the convolutional modules C1 to C5 adopt a dual convolutional layer structure, with each convolutional layer using a 3*3 kernel, and the convolution is processed by the PReLU activation function; the downsampling modules D1 to D3 use a 2*2 kernel with a stride of 2, and the convolution is processed by the PReLU activation function.

[0022] Furthermore, the dense modules M1 to M14 are composed of four convolutional layers connected sequentially from input to output. Each convolutional layer uses a 3*3 kernel and is processed by the PReLU activation function after convolution. For any convolutional layer, the input and output are concatenated to serve as the input of the next convolutional layer, and the output of the last convolutional layer is superimposed with the input of the first convolutional layer to serve as the output of the dense module.

[0023] Furthermore, the gated attention unit utilizes high-level features to guide low-level features to achieve attention to image detail features. It uses a 1*1 convolution kernel to convolve the output g of the corresponding dense module in the encoding unit and the input x of the corresponding upsampling module in the decoding unit. The convolution results are then superimposed and processed by the ReLU activation function, a 1*1 convolution kernel, batch normalization, and sigmoid activation to obtain the attention feature map α. Finally, the attention feature map α is multiplied by the input x of the corresponding upsampling module to obtain the output of the gated attention unit.

[0024] Furthermore, the process of training the ListmodeCNN model in step (4) is as follows:

[0025] 4.1 Initialize model parameters, including learnable parameters, convolutional layer parameters, learning rate, optimization algorithm, and maximum number of iterations;

[0026] 4.2 Input the PET list data from the training set samples into the ListmodeCNN model. The forward propagation transforms the data into the image domain through the domain transformation unit, and then the output result, i.e., the reconstructed PET image, is obtained through the encoding and decoding unit.

[0027] 4.3 Calculate the loss function between the PET image reconstructed by the model and the corresponding label, and update the model parameters by backpropagation using the gradient descent method based on the loss function;

[0028] 4.4 Following steps 4.2 and 4.3, the model parameters are iteratively updated using optimization algorithms until the loss function is minimized and converged or the maximum number of iterations is reached, at which point training is complete.

[0029] This invention combines list-pattern data with deep learning counting through a domain transformation function, solving the problem that list-pattern data is difficult to process directly using deep neural networks. Through the domain transformation function, this invention can transform list-pattern data of any format and storage size from the measurement domain to the image domain, thereby enabling direct processing and learning using end-to-end deep neural network methods.

[0030] Furthermore, this invention proposes a novel network structure based on U-shaped network design (i.e., residual attention-dense U-shaped network) to learn and reconstruct list pattern data after domain transformation. Compared with other U-shaped network methods and traditional list pattern reconstruction methods, this invention can obtain better reconstructed image quality, especially for small and difficult-to-distinguish simulated tumor blocks with excellent recovery effect.

[0031] This invention utilizes deep learning for list-pattern data reconstruction, significantly improving reconstruction speed compared to other methods used for list-pattern data reconstruction. For the list-pattern MLEM algorithm, reconstructing a brain image takes tens of seconds, while this invention can complete the reconstruction in only about 1 second, showing greater application potential in real-time reconstruction and short-time-frame reconstruction. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the PET reconstruction model construction method of the present invention.

[0033] Figure 2 This is a schematic diagram of the overall structure of the ListmodeCNN model of the present invention.

[0034] Figure 3 This is a structural diagram of a dense module.

[0035] Figure 4 This is a schematic diagram of the gating attention unit.

[0036] Figure 5This is a comparison chart of the reconstruction results of the PET template image and the list mode data by different methods; rows 1 to 3 correspond to cross-sectional, sagittal and coronal slice images respectively, and columns 1 to 5 correspond to the reconstruction results of the PET template image, MLEM, U-net, RDUnet and the reconstruction results obtained by the ListmodeCNN of this invention respectively. Detailed Implementation

[0037] To describe the present invention in more detail, the technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] like Figure 1 As shown, the PET reconstruction model construction method based on list data and deep learning of this invention includes:

[0039] Training phase

[0040] (1) Simulate projection of PET template image to obtain PET list data.

[0041] (2) Repeat step (1) to obtain a large amount of sample data and divide it into training set, test set and validation set. Each set of samples contains PET list data and corresponding template image.

[0042] (3) Determine the detailed structure of the model based on the list data reconstruction. The input of the model is the PET list data and the initial image, and the output is the reconstructed PET image. This model is called ListmodeCNN and includes a domain transformation unit and an encoding / decoding unit, such as... Figure 2 As shown:

[0043] The domain transformation unit is used to transform the original list data from the measurement domain to the image domain. This transformation can be achieved using a list data back projection algorithm or a list data iterative reconstruction algorithm with a single iteration. The encoding and decoding unit is used to reconstruct PET images while reducing noise. The encoder-decoder unit is based on a U-shaped network design and includes an encoding unit on the left, a decoding unit on the right, and a gated attention unit in the middle.

[0044] The encoding unit on the left includes a convolutional module, a dense module, and a downsampling module. Each convolutional module uses two 3x3 kernels, and after each convolution, it passes through a parameterized modified linear unit activation function (PReLU). The dense module reuses the feature map output from each convolutional layer and connects it to the current output feature map to enhance the network's learning ability. Its structure is as follows: Figure 3 As shown, the downsampling module consists of a 2*2 kernel with a stride of 2 and a parameterized modified linear unit activation function (PReLU).

[0045] The decoding unit on the right includes a convolutional module, a dense module, and an upsampling module. The convolutional module and the dense module have the same settings, but the number of input and output features differs. The upsampling module is completed using transposed convolution and PReLU, with a 2*2 convolutional kernel.

[0046] The central gating attention unit utilizes high-level features to guide low-level features, enabling attention to detailed image features, such as... Figure 4 As shown, it uses the low-level feature map in the left encoding unit and the high-level feature map in the right decoding unit before upsampling to perform convolution with a kernel size of 1*1 and then adds them together. Then, through 1*1 convolution, batch normalization and sigmoid activation, attention feature map α is obtained. The attention feature map α is multiplied with the low-level feature map to obtain the output of the gated attention module. Finally, the output is concatenated with the high-level feature map and then upsampled.

[0047] (4) During the training phase, the reconstruction model is trained using PET template images as labels and list-pattern data as input. The specific training process is as follows:

[0048] 4.1 Initialize model parameters. Use a random normal distribution to initialize model parameters, including weight matrix terms, bias terms, etc.

[0049] 4.2 Input the list pattern data into the reconstruction model, and forward propagation transforms the data into the image domain through the domain transformation unit. Then, the output result, i.e., the reconstructed PET image, is obtained through the encoding and decoding unit.

[0050] 4.3 Calculate the loss function between the labels and the model output, as well as the gradient of the loss function with respect to each variable. Use the Adam optimizer to update all learnable parameters in the model until the value of the loss function remains basically unchanged, at which point the training ends. In this example, the L1 norm loss function is used.

[0051] 4.4 The model is validated using the validation set samples, and the model that performs best on the validation set is selected as the final reconstruction model.

[0052] Inference phase

[0053] (1) Obtain PET list pattern data by measurement or simulation (test set).

[0054] (2) Input the list-mode data into the trained reconstruction model, and the model directly outputs the PET reconstruction results.

[0055] The following experiments, based on simulated list pattern data, verify the effectiveness of this invention. The simulated dataset used includes 25 human brain template data samples, each sample comprising 390 training data pairs, i.e., template images and corresponding list pattern data; the simulated tracer is...18 F-FDG, the count of the simulated list data is 1×10. 6 20 samples were used as training data, totaling 7800 sets of training data. 2 samples were used as test data, totaling 780 sets of data. 3 samples were used as test data, totaling 1170 sets of data.

[0056] ListmodeCNN was implemented using PyTorch 1.7.0 and trained on Ubuntu 18.0 with TiTanX. The optimizer was Adam, the initial learning rate was 0.0001, the batch size was 4, and a total of 200 epochs were trained. The domain transformation module used the EM algorithm for list data in one iteration. The epoch with the best performance on the validation set was used to evaluate the test set.

[0057] We used structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) to evaluate the experimental results. Table 1 shows the evaluation metrics for different methods.

[0058] Table 1

[0059]

[0060] As shown in the table, the SSIM and PSNR values ​​obtained from the listmodeCNN reconstruction are the best among the four methods. Figure 5 The experimental results show that the model of the present invention has advantages in solving the problem of reconstructing list-pattern data, based on the reconstruction results of different methods.

[0061] The above description of the embodiments is provided to enable those skilled in the art to understand and apply the present invention. Those skilled in the art can readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made to the present invention by those skilled in the art based on the disclosure thereof should be within the scope of protection of the present invention.

Claims

1. A method for constructing a PET reconstruction model based on list data and deep learning, comprising the following steps: (1) Simulate projection of the PET template image to obtain PET list data; (2) Repeat step (1) to obtain a large number of samples, and divide all samples into training set, test set and validation set. Each set of samples contains PET list data and corresponding PET template image. (3) Construct the ListmodeCNN model for list data reconstruction, which includes: Domain transformation unit, used to transform the input PET list data from the measurement domain to the image domain; The encoding / decoding unit is used to reconstruct the PET list data in the image domain to obtain the PET image; The encoding / decoding unit is based on a U-shaped network design, and its structure includes an encoding unit, a gated attention unit, and a decoding unit, wherein: The encoding unit consists of a convolution module C1, a dense module M1, a dense module M2, a downsampling module D1, a dense module M3, a dense module M4, a downsampling module D2, a dense module M5, a dense module M6, a downsampling module D3, a dense module M7, and a dense module M8 connected sequentially from input to output. The decoding unit consists of, from input to output, an upsampling module U1, a convolution module C2, a dense module M9, a dense module M10, an upsampling module U2, a convolution module C3, a dense module M11, a dense module M12, an upsampling module U3, a convolution module C4, a dense module M13, a dense module M14, and a convolution module C5 connected in sequence. The output of dense module M2 and the input of upsampling module U3 are fused through a gated attention unit, and the resulting output is then concatenated with the output of upsampling module U3 as the input of convolutional module C4. Similarly, the output of dense module M4 and the input of upsampling module U2 are fused through a gated attention unit and concatenated with the output of upsampling module U2 as the input of convolutional module C3. The output of dense module M6 and the input of upsampling module U1 are fused through a gated attention unit and concatenated with the output of upsampling module U1 as the input of convolutional module C2. Finally, the output of convolutional module C5 is superimposed with the input of convolutional module C1, and this superposition serves as the output of the encoding / decoding unit, i.e., the final output of the model. The convolutional modules C1 to C5 adopt a dual convolutional layer structure, with each convolutional layer using a 3*3 kernel. After convolution, the kernels are processed by the PReLU activation function. The downsampling modules D1 to D3 use a 2*2 kernel with a stride of 2. After convolution, the kernels are processed by the PReLU activation function. The dense modules M1~M14 are composed of four convolutional layers connected sequentially from input to output. Each convolutional layer uses a 3*3 kernel and is processed by the PReLU activation function after convolution. For any convolutional layer, the input and output are concatenated to serve as the input of the next convolutional layer. The output of the last convolutional layer is superimposed with the input of the first convolutional layer to serve as the output of the dense module. The gated attention unit utilizes high-level features to guide low-level features, thereby achieving attention to image detail features. It employs 1x1 convolutional kernels to process the outputs of corresponding dense modules in the encoding unit. g and the input of the corresponding upsampling module in the decoding unit x Convolution is performed on both, and the results of the convolutions are then superimposed and processed by ReLU activation, a 1x1 convolution kernel, batch normalization, and sigmoid activation to obtain the attention feature map. α Finally, the attention feature map α Input of the corresponding upsampling module x The result of multiplication is the output of the gated attention unit; (4) The ListmodeCNN model is trained using training set samples, with PET list data as input and PET template images as labels; (5) Input the PET list data in the test set sample directly into the trained model, and directly reconstruct the output PET image to test the performance of the model. After the test is completed, the model can be used to directly reconstruct the PET list data to obtain the PET image.

2. The PET reconstruction model construction method of claim 1, wherein: The domain transformation unit uses a list data back projection algorithm or a list data iterative reconstruction algorithm with a single iteration to transform PET list data from the measurement domain to the image domain.

3. The PET reconstruction model construction method of claim 1, wherein: The process of training the ListmodeCNN model in step (4) is as follows: 4.1 Initialize model parameters, including learnable parameters, convolutional layer parameters, learning rate, optimization algorithm, and maximum number of iterations; 4.2 Input the PET list data from the training set samples into the ListmodeCNN model. The forward propagation transforms the data into the image domain through the domain transformation unit, and then the output result, i.e., the reconstructed PET image, is obtained through the encoding and decoding unit. 4.3 Calculate the loss function between the PET image reconstructed by the model and the corresponding label, and update the model parameters by backpropagation using the gradient descent method based on the loss function; 4.4 Following steps 4.2 and 4.3, the model parameters are iteratively updated using optimization algorithms until the loss function is minimized and converged or the maximum number of iterations is reached, at which point training is complete.

Images