A low-dose PET image enhancement method and system

Abstract

The present application relates to a kind of low-dose PET image enhancement method and system, belong to medical and industrial field positron emission tomography (PET) imaging, for solving prior art using anatomical structure information MR image as prior information when low-dose PET image data is generated normal dose PET image, effect is not good.This scheme is obtained from MR image and spatial anatomical arrangement information through spatial transformation anatomical structure spatial characteristics, by first using anatomical structure spatial characteristics with low-dose PET image characteristics to realize the adaptive enhancement of spatial region information and different modal fusion, to improve the anatomical structure display quality of low-dose image data, improve anatomical structure definition, provide effective reference for clinical diagnosis, reduce tracer usage, reduce economic cost, reduce patient radiation burden.

Description

Technical Field

[0001] This case relates to positron emission tomography (PET) imaging in the medical and industrial fields, and more particularly to a low-dose PET image enhancement method and system. Background Technology

[0002] Most current deep learning methods construct end-to-end networks, directly generating normal-dose PET images from low-dose brain PET imaging data. MR images, which provide anatomical information, are also often introduced as prior information into the reconstruction process. However, these methods all neglect the spatial anatomical arrangement information between the imaging data and the anatomical structures. Summary of the Invention

[0003] To address the aforementioned problems in existing technologies, the present invention aims to propose a low-dose PET image enhancement method. This method fully utilizes the spatial anatomical arrangement information between MR data and actual anatomical structures. By adaptively enhancing regional information and fusing different modalities in low-dose images, it improves the display quality of low-dose images, thereby reducing the amount of radioactive tracer required for PET scans and alleviating patient radiation exposure. Furthermore, it enhances the clarity of anatomical structures, providing a valuable reference for clinical diagnosis.

[0004] Firstly, this case proposes a low-dose whole-brain image enhancement method, which is based on low-dose PET images, MR images, and spatial brain anatomical alignment information, and uses a trained 3D neural network to obtain standard-dose PET enhanced images.

[0005] The 3D neural network acquires multi-scale features of low-dose PET images, MR images, and spatial anatomical arrangement information. For the low-dose PET image features at each scale, spatial adjustment is performed using the anatomical structure spatial features obtained from the MR images and spatial anatomical arrangement information to obtain PET spatial enhancement images corresponding to each scale. Using each PET spatial enhancement image, scale-wise fusion prediction is performed to obtain standard-dose PET enhancement images. The spatial anatomical arrangement information is obtained from T1-weighted MR images.

[0006] In one embodiment of the above technical solution: the 3D neural network includes an encoder and a decoder; the encoder includes J encoding units of different scales, the input of the j-th encoding unit is downsampled to obtain the input of the (j+1)-th encoding unit, j=1, 2, ..., J-1; the j-th encoding unit takes its corresponding size low-dose PET image, MR image and spatial anatomical arrangement information as input, and outputs the j-th PET spatial enhancement image, j=1, 2, ..., J; the decoder includes J-1 decoding units, wherein: the input of the (J-1)-th decoding unit is the upsampled image of the j-th PET spatial enhancement image and the (J-1)-th PET spatial enhancement image, and the output is the (J-1)-th PET fusion prediction image; the j-th decoding unit takes the upsampled image of the (j+1)-th PET fusion prediction image and the j-th PET spatial enhancement image as input to obtain the j-th PET fusion prediction image, j=J-2, J-3, ..., 2, 1, and uses the j-th PET fusion prediction image as the standard dose PET enhancement image.

[0007] In one embodiment of the above technical solution: the spatial transformation step includes: based on spatial anatomical arrangement information Obtaining Affine Parameters , H represents the affine transformation; based on affine parameters To obtain image features of MR images Scaled feature image , And obtain the affine parameters of the scaling feature. , Q represents the affine transformation; based on affine parameters Acquire image features of low-dose PET images Feature images after scaling and translation , ,Will As a PET spatial enhancement image.

[0008] In one embodiment of the above technical solution: the 3D neural network uses the MAE function as the loss function, specifically:

[0009]

[0010] In the formula: G represents the mapping relationship, This represents the network parameters, where n is the total number of training samples. This is a sample of low-dose PET image data. Low-dose PET image data samples Corresponding MR image data samples, Low-dose PET image samples Corresponding spatial anatomical arrangement information, Low-dose PET image data samples Corresponding standard dose PET image samples.

[0011] In one embodiment of the above technical solution: the decoding unit concatenates the input upsampled image and the feature fusion image, and then processes them using convolutional layers and transposed convolutional layers.

[0012] Based on the above-mentioned methods and technical solutions, this case proposes a corresponding low-dose whole-brain image enhancement system.

[0013] The beneficial technical effects of this case are:

[0014] (1) By obtaining spatial anatomical arrangement information from T1-weighted MR images, and then obtaining anatomical spatial features from MR images and spatial anatomical arrangement information through spatial transformation, the spatial features of anatomical structures can be achieved by first utilizing the spatial features of anatomical structures in low-dose PET images to achieve adaptive enhancement of spatial region information and fusion of different modalities. This is beneficial to improve the anatomical structure display quality of low-dose image data, reduce the amount of tracer used, reduce economic costs, and alleviate the radiation burden on patients.

[0015] (2) By fusing and predicting PET spatial enhancement images at various scales, we can obtain PET images with better image quality and more accurate anatomical location, reduce the radiation burden on patients, improve the clarity of anatomical structures, and provide effective reference for clinical diagnosis. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 , one Schematic diagram of the implementation framework;

[0018] Figure 2 , one Low-dose brain PET images in one embodiment;

[0019] Figure 3 , Figure 2 The corresponding standard reference image;

[0020] Figure 4 , Figure 2 The corresponding standard dose brain PET images generated. Detailed Implementation

[0021] While PET / MRI holds promise for improving our understanding of neurological diseases, it still has some limitations. For example, patients need to receive an injection of a radiopharmaceutical tracer before a PET scan, which increases their radiation risk. Furthermore, reduced injection doses can lead to increased noise in PET images, complicating clinical diagnosis. On the other hand, PET scans are relatively expensive, increasing the financial burden on patients; this high cost is a recognized disadvantage of PET / MRI.

[0022] Most current deep learning methods construct end-to-end networks, directly generating normal-dose PET images from low-dose PET image data. MR images, which provide anatomical information, are also often introduced as prior information into the reconstruction process. However, these methods all neglect the spatial anatomical arrangement information between the image data and the anatomical structures.

[0023] Based on this, this approach utilizes deep learning technology to introduce spatial anatomical arrangement information into PET-MR images to ensure the consistency and accuracy of anatomical structural spatial location information with tissue imaging data. By implementing spatial transformations, adaptive enhancement and modal fusion of spatial location information of various tissues in low-dose PET images are achieved, thereby improving PET image quality and supporting the study of the functional and metabolic characteristics of imaged tissues. The method in this study can be used to improve low-dose whole-brain PET images, whole-body PET images, and cardiac PET images, etc.

[0024] The following, in conjunction with the accompanying drawings, will clearly and completely describe how the technical solution of this application is implemented, based on the process of obtaining a standard-dose whole-brain PET image from a low-dose whole-brain PET image. Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0025] In this implementation, the matrix size of the T1-weighted MR image is 256×256×256, and the voxel size is 1×1×1 mm. The matrix size of the PET image is also 256 × 256 × 256, and the voxel size is 1×1×1 mm.

[0026] like Figure 1As shown in Figure A, precise spatial anatomical arrangement information of the whole brain is first obtained from T1-weighted MR images. In this embodiment, spatial anatomical arrangement information refers to the positional features of various brain tissues. To achieve this, real T1 MR images are collected from PET / MRI integrated equipment scans. The obtained images are used for head surface reconstruction, skull dissection, brain volume and intensity normalization, surface image registration, surface extraction and reconstruction, structural labeling (Gyral Labling), and arrangement of different brain regions to obtain spatial anatomical arrangement information. These steps can be performed using the FreeSurfer toolkit, which has been proven in previous studies to provide accurate and reliable results. By using FreeSurfer and according to the brain region partitioning requirements, spatial brain anatomical arrangement information can be directly obtained. For example, in this embodiment, 45 different brain regions (such as the cortex and cerebellum) were obtained.

[0027] See Figure 1 In B, MR images, spatial anatomical arrangement information, and low-dose PET images are used as inputs to a trained 3D neural network to obtain standard-dose PET output.

[0028] Figure 1 The diagram (C) illustrates the specific 3D neural network structure used in this scheme. It is a U-shaped structure including an encoder and a decoder. The encoder includes encoding units for acquiring PET spatially enhanced images at various scales. The decoder includes decoding units for fusing the PET spatially enhanced images scale-wise and making predictions, ultimately outputting a synthesized standard-dose PET image. The number of encoding and decoding units is determined based on the desired image sharpness and available hardware computing power. As shown in the diagram, the encoder has five encoding units, which, combined with downsampling operations, transmit feature information at different scales. The decoder includes four decoding units for progressively recovering the size of the standard-dose PET enhancement features.

[0029] Each coding unit aims to acquire anatomical spatial features from MR images and spatial anatomical arrangement information through spatial transformation, and to adjust low-dose PET image features using these anatomical spatial features, thereby achieving modal fusion. For example... Figure 1 As shown in Figure D, when faced with multimodal information input, three-dimensional convolution is first used to convert the spatial anatomical arrangement information into affine parameters. The convolution kernel size used is 3×3×3 with a stride of 1, and the activation function chosen is the ReLU function. Affine parameters are then utilized. MR features at the j-th scale Scaling operations on a channel can be specifically represented as follows:

[0030]

[0031]

[0032] in: Represented as an affine transformation, affine parameters MR features at the j-th scale Initial scaling is performed to obtain features .

[0033] Next, a similar operation is performed to transform the MR features at the j-th scale. Continue with the affine transformation. Use a 3×3×3 kernel with a stride of 1 and a ReLU activation function to obtain the affine parameters. Then use affine parameters PET feature image at scale j Scaling and translation operations on a channel can be specifically represented as follows:

[0034]

[0035]

[0036] in: Represented as an affine transformation, This represents the PET features after affine transformation.

[0037] The scaling operation in the above process is a pixel-by-pixel multiplication operation, and the translation operation is a pixel-by-pixel addition operation.

[0038] Spatial transformation allows for adaptive affine transformation of the channel dimensions of intermediate feature maps and fusion of different modalities, thereby enhancing the spatial features of anatomical structures in low-dose PET images, improving the display quality of anatomical structures in low-dose image data, and increasing the clarity of anatomical structures.

[0039] Each decoding unit includes a 3 × 3 × 3 convolutional layer (span = 1) and a 3 × 3 × 3 transposed convolutional layer (span = 2), with instance normalization layers and activation function layers inserted. The feature map obtained by the decoding unit using skip connections in the encoding part is copied to the feature map obtained in the decoding part, thereby improving the accuracy of standard dose PET image prediction. Specifically, taking the figure as an example, the upsampled PET spatial enhancement image output by the 5th encoding unit and the PET spatial enhancement image output by the 4th encoding unit are the inputs to the 4th decoding unit. The decoding unit concatenates them and then processes them using convolutional and transposed convolutional layers to output the 4th PET fusion prediction image. The next 3rd decoding unit takes the upsampled image of the 4th PET fusion prediction image and the 3rd PET spatial enhancement image as input to output the 3rd PET fusion prediction image. The 2nd and 1st decoding units operate similarly to the 3rd decoding unit, using the PET fusion prediction image output by the 1st decoding unit as the standard dose PET enhancement image.

[0040] Let the training dataset for the 3D neural network be D, where D = {( , , , ), ( , , , ),...,( , , )},in, These are samples from a low-dose PET image dataset. These are samples from an MR image dataset. This is the spatial anatomical arrangement information extracted from T1-weighted MR images, where n is the total number of training samples. It is a set of data samples of standard dose PET enhanced images predicted by the network.

[0041] During training, the MAE function is selected as the loss function, and the training loss can be expressed as:

[0042]

[0043] Where G represents the mapping relationship. This represents network parameters.

[0044] Next, a 3D neural network can be trained using the dataset and optimized using the Adam optimizer. Low-dose PET images of the whole brain will be used. MR images and spatial anatomical arrangement information As network input, images enhanced with standard dose PET. The network is trained using ground truth as the reference data for prediction.

[0045] Once training is complete, a trained 3D neural network can be obtained. Figure 2 This illustrates a low-dose PET image of the brain. Figure 3 For its corresponding PET prediction reference image, Figure 4 Standard-dose PET images output by a trained 3D neural network. Figure 3 and Figure 4 As can be seen, the predicted image and the reference image are very close, and by Figure 2 and Figure 4 The comparison shows that the generated standard-dose PET images have a significant quality improvement compared to low-dose PET images, with excellent spatial consistency and anatomical accuracy, which is beneficial for the detection and analysis of neurogenic diseases.

[0046] When low-dose PET images, MR images, and spatial anatomical arrangement information of the same tissue are input, standard-dose PET-enhanced images can be directly obtained for clinical diagnosis or research. These standard-dose PET-enhanced images provide adaptive enhancement of spatial region information compared to low-dose PET images, improving the display quality and clarity of anatomical structures and helping to reduce radiation burden on patients. This approach can integrate multimodal information and, in addition to being applied to PET / MR equipment, can also be applied to PET / CT equipment after appropriate modification to provide a comprehensive view of pathophysiological processes.

[0047] In other embodiments, the above method is implemented as a corresponding system, including a preprocessing module and a 3D neural network module. The preprocessing module is configured to obtain spatial anatomical arrangement information from T1-weighted MR images. The 3D neural network module is configured to acquire multi-scale features of low-dose PET images, MR images, and spatial anatomical arrangement information; for each scale of low-dose PET image features, spatial adjustments are made using the anatomical structure spatial features obtained from the MR images and spatial anatomical arrangement information to obtain PET spatially enhanced images corresponding to each scale; using each PET spatially enhanced image, scale-wise fusion prediction is performed to obtain a standard-dose PET enhanced image. The 3D neural network module includes an encoder and a decoder. The features of the encoder and decoder, as well as the loss function of the 3D neural network module, are described in the same way as in the above method.

[0048] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods or systems disclosed herein can be implemented using software plus necessary general-purpose hardware, or they can be implemented using dedicated hardware including dedicated integrated circuits, dedicated CPUs, dedicated memory, dedicated components, etc. Generally, any function performed by a computer program can be easily implemented using corresponding hardware, and the specific hardware structure used to implement the same function can be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the purposes of this disclosure, software program implementation is more often a preferred implementation method.

[0049] Although embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the specific embodiments and application fields described above. The specific embodiments described above are merely illustrative and instructive, and not restrictive. Those skilled in the art can make many other forms based on the guidance of this specification and without departing from the scope of protection of the claims of the present invention, and all of these are within the scope of protection of the present invention.

Claims

1. A low-dose PET image enhancement method, characterized in that: The method is based on low-dose PET images, MR images, and spatial brainanatomical alignment information, and uses a trained 3D neural network to obtain standard-dose PET enhanced images. The 3D neural network acquires multi-scale features of low-dose PET images, MR images, and spatial anatomical arrangement information, respectively. For the low-dose PET image features at each scale, spatial adjustment is performed using the anatomical spatial features obtained from MR images and spatial anatomical arrangement information to obtain the corresponding PET spatially enhanced image at each scale; using each PET spatially enhanced image, scale-wise fusion prediction is performed to obtain the standard-dose PET enhanced image. The spatial anatomical arrangement information was obtained from T1-weighted MR images; The 3D neural network includes an encoder and a decoder. The encoder includes J encoding units of different scales. The input of the j-th encoding unit is downsampled to obtain the input of the (j+1)-th encoding unit, j=1, 2, ..., J-1. The j-th encoding unit takes its corresponding size low-dose PET image, MR image, and spatial anatomical arrangement information as input and outputs the j-th PET spatial enhancement image, j=1, 2, ..., J. The decoder includes J-1 decoding units, where: the input of the (J-1)-th decoding unit is the upsampled image of the j-th PET spatial enhancement image and the (J-1)-th PET spatial enhancement image, and the output is the (J-1)-th PET fusion prediction image; the j-th decoding unit takes the upsampled image of the (j+1)-th PET fusion prediction image and the j-th PET spatial enhancement image as input to obtain the j-th PET fusion prediction image, j=J-2, J-3, ..., 2, 1, and uses the first PET fusion prediction image as the standard dose PET enhancement image. The step of spatial adjustment using spatial features of anatomical structures obtained from MR images and spatial anatomical arrangement information includes: Based on spatial anatomical arrangement information Obtaining Affine Parameters , H represents the affine transformation; Based on affine parameters Obtaining image features from MR images Scaled feature image , And obtain the affine parameters of the scaling feature. , Q is an affine transformation; Based on affine parameters Acquire image features of low-dose PET images Feature images after scaling and translation , ,Will As a PET spatial enhancement image.

2. The method according to claim 1, characterized in that: 3D neural networks use the MAE function as the loss function, specifically: In the formula: G represents the mapping relationship, This represents the network parameters, where n is the total number of training samples. This is a sample of low-dose PET image data. Low-dose PET image data samples Corresponding MR image data samples, Low-dose PET image samples Corresponding spatial anatomical arrangement information, Low-dose PET image data samples Corresponding standard dose PET image samples.

3. The method according to claim 1, characterized in that: The decoding unit stitches the two input images together, and then processes them using convolutional layers and transposed convolutional layers to output a PET fusion prediction image.

4. A low-dose PET image enhancement system, characterized in that: The system includes a preprocessing module and a 3D neural network module; The preprocessing module is configured to obtain spatial anatomical arrangement information from T1-weighted MR images; The 3D neural network module is configured to acquire multi-scale features of low-dose PET images, MR images, and spatial anatomical arrangement information, respectively. For the low-dose PET image features at each scale, spatial adjustment is performed using the anatomical spatial features obtained from MR images and spatial anatomical arrangement information to obtain the corresponding PET spatially enhanced image at each scale; using each PET spatially enhanced image, scale-wise fusion prediction is performed to obtain the standard-dose PET enhanced image. The 3D neural network includes an encoder and a decoder. The encoder includes J encoding units of different scales. The input of the j-th encoding unit is downsampled to obtain the input of the (j+1)-th encoding unit, j=1, 2, ..., J-1. The j-th encoding unit takes its corresponding size low-dose PET image, MR image, and spatial anatomical arrangement information as input and outputs the j-th PET spatial enhancement image, j=1, 2, ..., J. The decoder includes J-1 decoding units, where: the input of the (J-1)-th decoding unit is the upsampled image of the j-th PET spatial enhancement image and the (J-1)-th PET spatial enhancement image, and the output is the (J-1)-th PET fusion prediction image; the j-th decoding unit takes the upsampled image of the (j+1)-th PET fusion prediction image and the j-th PET spatial enhancement image as input to obtain the j-th PET fusion prediction image, j=J-2, J-3, ..., 2, 1, and uses the first PET fusion prediction image as the standard dose PET enhancement image. The step of spatial adjustment using spatial features of anatomical structures obtained from MR images and spatial anatomical arrangement information includes: Based on spatial anatomical arrangement information Obtaining Affine Parameters , H represents the affine transformation; Based on affine parameters Obtaining image features from MR images Scaled feature image , And obtain the affine parameters of the scaling feature. , Q is an affine transformation; Based on affine parameters Acquire image features of low-dose PET images Feature images after scaling and translation , ,Will As a PET spatial enhancement image.

5. The system according to claim 4, characterized in that: The 3D neural network module uses the MAE function as the loss function, specifically: In the formula: G represents the mapping relationship, This represents the network parameters, where n is the total number of training samples. This is a sample of low-dose PET image data. Low-dose PET image data samples Corresponding MR image data samples, Low-dose PET image samples Corresponding spatial anatomical arrangement information, Low-dose PET image data samples Corresponding standard dose PET image samples.

6. The system according to claim 4, characterized in that: The decoding unit stitches the two input images together, then processes them using convolutional and transposed convolutional layers to output a PET fusion prediction image.

Images