Low-dose pet image enhancement method and system

Abstract

The present application relates to positron emission tomography (PET) imaging in the medical and industrial fields, and discloses a low-dose PET image enhancement method and system, used for solving the problem in the prior art of having poor effect when an MR image of anatomical structure information is used as prior information to generate a normal-dose PET image from low-dose PET image data. According to the solution, spatial features of an anatomical structure are obtained from an MR image and spatial anatomical alignment information, and the spatial features of the anatomical structure are used to spatially adjust low-dose PET image features to achieve adaptive enhancement of spatial regional information and fusion of different modalities, so as to improve the display quality of the anatomical structure of low-dose image data, improve the definition of the anatomical structure, provide effective reference for clinical diagnosis, reduce the use amount of a tracer, reduce economic costs, and lessen the radiation load on patients.

Description

A low-dose PET image enhancement method and system Technical Field

[0001] This case involves positron emission tomography (PET) imaging in the medical and industrial fields, and in particular, a low-dose PET image enhancement method and system. Background Art

[0002] Most current deep learning methods construct end-to-end networks, directly generating normal-dose PET images from low-dose brain PET data. MR images, which provide anatomical information, are often incorporated into the reconstruction process as prior information. However, these methods ignore the spatial anatomical arrangement between the image data and the anatomical structures.

[0003] Summary of the Invention

[0004] To address the aforementioned issues in the prior art, the present invention aims to propose a low-dose PET image enhancement method that fully utilizes the spatial anatomical arrangement information between MR data and actual anatomical structures. By adaptively enhancing regional information and integrating different modalities in low-dose images, the method improves the display quality of low-dose images, thereby reducing the amount of radioactive tracer required for PET scans and alleviating patient radiation exposure. It also enhances the clarity of anatomical structures, providing an effective reference for clinical diagnosis.

[0005] First, this case proposes a low-dose whole-brain image enhancement method. The method uses a trained 3D neural network to obtain standard-dose PET enhanced images based on low-dose PET images, MR images, and the spatial brain anatomical alignment information.

[0006] The 3D neural network separately obtains multi-scale features of low-dose PET images, MR images, and spatial anatomical arrangement information; spatially adjusts the low-dose PET image features at each scale using the anatomical structure spatial features obtained from the MR images and spatial anatomical arrangement information to obtain PET spatially enhanced images corresponding to each scale; and uses each PET spatially enhanced image to perform scale-by-scale fusion prediction to obtain a standard-dose PET enhanced image; the spatial anatomical arrangement information is obtained from the T1-weighted MR image.

[0007] In one embodiment of the above technical solution: the 3D neural network includes an encoder and a decoder; the encoder includes J encoding units of different scales, the input of the j-th encoding unit is downsampled to obtain the input of the j+1-th encoding unit, j=1, 2, ..., J-1; the j-th encoding unit takes its corresponding size low-dose PET image, MR image and spatial anatomical arrangement information as input, and outputs the j-th PET spatially enhanced image, j=1, 2, ..., J; the decoder includes J-1 decoding units, wherein: the input of the J-1-th decoding unit is the upsampled image of the J-th PET spatially enhanced image and the J-1-th PET spatially enhanced image, and the output is the J-1-th PET fusion prediction image; the j-th encoding unit takes the upsampled image of the j+1-th PET fusion prediction image and the j-th PET spatially enhanced image as input to obtain the j-th PET fusion prediction image, j=J-2, J-3, ..., 2, and uses the last PET fusion prediction image as the standard metrology PET enhanced image.

[0008] In one embodiment of the above technical solution: the spatial transformation step includes: based on the spatial anatomical arrangement information f s Get the affine parameter γ1, γ1 = H(f s ), H is the affine transformation; based on the affine parameter γ1, the image feature f of the MR image is obtained m The scaled feature image And get the affine parameters (α, δ) of the scaled feature, Q is the affine transformation; based on the affine parameters (α, δ), the image feature f of the low-dose PET image is obtained p The zoomed and translated feature image Will As PET spatial enhancement image.

[0009] In one embodiment of the above technical solution: the 3D neural network uses the MAE function as the loss function, specifically:

[0010] Where: G represents the mapping relationship, θ i represents the network parameters, n is the total number of training samples, x i is a low-dose PET image data sample, y i is the low-dose PET image data sample x i The corresponding MR image data sample, z i For the low-dose PET image sample x i The corresponding spatial anatomical arrangement information, w i is the low-dose PET image data sample x iCorresponding standard-dose PET image samples.

[0011] In one implementation of the above technical solution: the decoding unit splices the input upsampled image and the feature fusion image, and then processes them using a convolution layer and a transposed convolution layer.

[0012] Based on the above method and technical solution, this case proposes a corresponding low-dose whole-brain image enhancement system.

[0013] Beneficial technical effects of this case:

[0014] (1) By obtaining spatial anatomical arrangement information from T1-weighted MR images, and then obtaining anatomical structure spatial features from MR images and spatial anatomical arrangement information through spatial transformation, the low-dose PET image features are first used to utilize the anatomical structure spatial features to achieve adaptive enhancement of spatial region information and fusion of different modalities, which is beneficial to improve the anatomical structure display quality of low-dose imaging data, reduce the use of tracers, reduce economic costs, and alleviate the radiation burden on patients.

[0015] (2) By fusing and predicting PET spatial enhancement images at various scales, PET images with better image quality and more accurate anatomical location can be obtained, which can reduce the patient's radiation burden, improve the clarity of the anatomical structure, and provide an effective reference for clinical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 is a schematic diagram of an implementation framework;

[0018] FIG2 is a low-dose brain PET image according to one embodiment;

[0019] Figure 3. Reference standard images corresponding to Figure 2;

[0020] Figure 4. Standard-dose brain PET images generated corresponding to Figure 2. DETAILED DESCRIPTION

[0021] While PET / MRI holds promise for advancing our understanding of neurological diseases, it still has several limitations. For example, prior to a PET scan, patients must receive an injection of a radiopharmaceutical tracer, increasing their radiation exposure. Reducing the injected dose can increase PET image noise, complicating clinical diagnosis. Furthermore, PET scans are relatively expensive, increasing the cost to patients and a recognized disadvantage of PET / MRI.

[0022] Most current deep learning methods construct end-to-end networks, directly generating normal-dose PET images from low-dose PET imaging data. MR images, which provide anatomical structural information, are often incorporated into the reconstruction process as prior information. However, these methods ignore the spatial anatomical arrangement between the imaging data and the anatomical structures.

[0023] Based on this, this approach leverages deep learning technology to introduce spatial anatomical alignment information into PET-MR images, ensuring consistency and accuracy between the spatial position of anatomical structures and tissue imaging data. By implementing spatial transformations, adaptive enhancement of spatial position information of various tissue regions in low-dose PET images and fusion of different modalities are achieved, improving PET image quality and supporting research on the functional and metabolic characteristics of imaged tissues. This method can be used to improve low-dose whole-brain PET, whole-body PET, and cardiac PET images, among other applications.

[0024] The following, combined with the accompanying drawings, provides a clear and complete description of how the technical solution of this application is implemented, illustrating the process of obtaining a standard-dose whole-brain PET image based on a low-dose whole-brain PET image. Obviously, the described embodiments represent only a portion of the embodiments of this application, and not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of this application without inventive effort are also within the scope of protection of this application.

[0025] In this implementation, the matrix size of T1-weighted MR images was 256 × 256 × 256, and the voxel size was 1 × 1 × 1 mm. The matrix size of PET images was also 256 × 256 × 256, and the voxel size was 1 × 1 × 1 mm.

[0026] As shown in A in Figure 1, accurate spatial anatomical arrangement information about the whole brain is first obtained from the T1-weighted MR image. In the present embodiment, the spatial anatomical arrangement information is about the positional characteristics of each brain tissue. To achieve this purpose, real T1MR images are collected from PET / MRI integrated equipment scans, and the obtained images are used to perform head surface reconstruction, skull stripping, brain volume and intensity normalization, surface image registration, surface extraction and reconstruction, structural labeling (Gyral Labeling) and arrangement of various parts of the brain to obtain spatial anatomical arrangement information. These steps can be performed using the FreeSurfer toolkit, which has been proven to provide accurate and reliable results in previous studies. By using FreeSurfer, and according to the requirements of brain region partitioning, spatial brain anatomical arrangement information output can be directly obtained. Exemplarily, 45 different brain regions (such as cortex and cerebellum) are obtained in the present embodiment.

[0027] Referring to B in FIG1 , MR images, spatial anatomical arrangement information, and low-dose PET images are used as inputs to the trained 3D neural network to obtain standard-dose PET outputs.

[0028] Figure 1, C, illustrates the specific 3D neural network architecture used in this approach. It is a U-shaped structure consisting of an encoder and a decoder. The encoder includes encoding units for acquiring PET spatially enhanced images at various scales, while the decoder includes decoding units for fusing these images scale by scale and performing predictions, ultimately outputting a synthesized standard-dose PET image. The number of encoding and decoding units is determined by the desired image clarity and the available hardware computing power. As shown in the figure, the encoder has five encoding units, which incorporate downsampling to convey feature information at different scales. The decoder includes four decoding units for progressively restoring the size of standard-dose PET enhancement features.

[0029] Each encoding unit is designed to obtain anatomical spatial features from MR images and spatial anatomical arrangement information through spatial transformation, and use the anatomical spatial features to adjust the low-dose PET image features and achieve different modal fusion. As shown in Figure 1D, facing the input of multimodal information, the spatial anatomical arrangement information is first converted into affine parameters γ1 using three-dimensional convolution, where the convolution kernel size used is 3×3×3, the step size is 1, and the activation function is the ReLU function. Then, the affine parameters γ1 are used to transform the MR features of the j-th scale. Scaling operations are performed on channels, which can be expressed as follows:

[0030] Where: H represents the affine transformation, and the affine parameter γ1 is the MR feature of the jth scale Perform preliminary scaling to obtain features

[0031] Then, similar operations are performed to transform the MR features of the jth scale into Continue with affine transformation. The convolution kernel size used is 3×3×3, the step size is 1, and the activation function is the ReLU function. The affine parameters (α, δ) are obtained and then the affine parameters (α, δ) are used to transform the PET feature image of the jth scale. Zooming and panning operations are performed on the channel, which can be expressed as follows:

[0032] Where: Q represents an affine transformation, represents the PET features after affine transformation.

[0033] The scaling in the above process is a pixel-by-pixel multiplication operation on the pixels in the image, and the translation operation is a pixel-by-pixel addition operation on the pixels in the image.

[0034] Spatial transformation can allow adaptive affine transformation of the channel dimension of the intermediate feature map and fuse different modalities, thereby enhancing the anatomical spatial characteristics of low-dose PET images, improving the anatomical display quality of low-dose imaging data, and enhancing the anatomical clarity.

[0035] Each decoding unit consists of a 3×3×3 convolutional layer (span = 1) and a 3×3×3 transposed convolutional layer (span = 2), interleaved with instance normalization and activation function layers. The feature maps obtained by the encoding part of the decoding unit are copied to the feature maps obtained by the decoding part using skip connections, thereby improving the accuracy of standard-dose PET image prediction. For example, the upsampled PET spatial enhancement image output by the fifth encoding unit and the PET spatial enhancement image output by the fourth encoding unit serve as inputs to the fourth decoding unit. The decoding unit concatenates these images and processes them using convolutional and transposed convolutional layers, outputting the fourth PET fusion prediction image. The third decoding unit then takes the upsampled image of the fourth PET fusion prediction image and the third PET spatial enhancement image as inputs and outputs the third PET fusion prediction image. The second and first decoding units operate similarly to the third decoding unit, using the PET fusion prediction image output by the first decoding unit as the standard-dose PET enhancement image.

[0036] The training data set of the 3D neural network is set as D, D = {(x1, y1, z1, w1), (x2, y2, z2, w2), ..., (x n ,y n , z n , w n)}, where x={x1, x2, ..., x n} is a sample in the low-dose PET image dataset, y = {y1, y2, ..., y n} is a sample from the MR image dataset, z = {z1, z2, ..., z n} is the spatial anatomical arrangement information extracted from T1-weighted MR images, n is the total number of training samples, w = {w1, w2, ..., w n} is the data sample set of standard dose PET enhanced images predicted by the network.

[0037] During training, the MAE function is selected as the loss function, and the training loss can be expressed as:

[0038] Where G represents the mapping relationship, θ i Indicates network parameters.

[0039] Next, we can use the dataset to train the 3D neural network and use the Adam optimizer to optimize it. i , MR image y i and spatial anatomical arrangement information z i As network input, the standard dose PET enhanced image w i As prediction reference data (ground truth) to train the network.

[0040] At the end of training, a trained 3D neural network is obtained. Figure 2 illustrates a low-dose PET image of the brain, Figure 3 shows the corresponding predicted PET reference image, and Figure 4 shows the standard-dose PET image output by the trained 3D neural network. As can be seen in Figures 3 and 4, the predicted and reference images are very close. Comparing Figures 2 and 4 shows that the generated standard-dose PET images are significantly improved in quality compared to low-dose PET images, with excellent spatial consistency and anatomical accuracy, which is beneficial for the detection and analysis of neurological diseases.

[0041] By inputting low-dose PET and MR images of the same tissue, along with spatial anatomical arrangement information, standard-dose PET-enhanced images can be directly obtained for clinical diagnosis or research. Compared to low-dose PET images, these standard-dose PET-enhanced images adaptively enhance spatial regional information, improving the display quality and clarity of anatomical structures, and reducing radiation burden on patients. This solution, capable of fusing multimodal information, can be applied not only to PET / MR equipment but, after appropriate modification, to PET / CT equipment, providing a comprehensive view of pathophysiological processes.

[0042] In some other embodiments, the above method is implemented as a corresponding system, including a preprocessing module and a 3D neural network module. The preprocessing module is configured to obtain spatial anatomical arrangement information from T1-weighted MR images. The 3D neural network module is configured to respectively obtain multi-scale features of low-dose PET images, MR images, and spatial anatomical arrangement information; for the low-dose PET image features of each scale, spatial adjustments are performed using the anatomical structure spatial features obtained from the MR images and the spatial anatomical arrangement information to obtain PET spatial enhanced images corresponding to each scale; and each PET spatial enhanced image is used to perform scale-by-scale fusion prediction to obtain a standard-dose PET enhanced image. The 3D neural network module includes an encoder and a decoder. The features of the encoder and decoder and the loss function of the 3D neural network module are all the same as those described in the above method.

[0043] Through the above description of the embodiments, those skilled in the art will clearly understand that the disclosed method or system can be implemented by means of software plus necessary general-purpose hardware. Of course, it can also be implemented by means of dedicated hardware, including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. In general, any function performed by a computer program can be easily implemented by the corresponding hardware. Moreover, the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the present disclosure, software program implementation is often the preferred embodiment.

[0044] Although the embodiments of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to the above-mentioned specific embodiments and application fields. The above-mentioned specific embodiments are merely illustrative and instructive, and are not restrictive. A person skilled in the art, guided by this specification and without departing from the scope of protection of the claims of the present invention, may also devise various forms, all of which fall within the scope of protection of the present invention.

Claims

1. A low-dose PET image enhancement method, characterized in that: The method is based on low-dose PET images, MR images and the spatial brain anatomical alignment information, and uses a trained 3D neural network to obtain standard-dose PET enhanced images; The 3D neural network respectively acquires multi-scale features of low-dose PET images, MR images, and spatial anatomical arrangement information; For each scale of low-dose PET image features, spatial adjustment is performed using the anatomical structure spatial features obtained from MR images and spatial anatomical arrangement information to obtain the PET spatial enhanced images corresponding to each scale; using each PET spatial enhanced image, scale-by-scale fusion prediction is performed to obtain the standard-dose PET enhanced image; The spatial anatomical arrangement information is obtained from T1-weighted MR images.

2. The method according to claim 1, characterized in that: The 3D neural network includes an encoder and a decoder; The encoder includes J encoding units of different scales, the input of the jth encoding unit is downsampled to obtain the input of the j+1th encoding unit, j=1, 2, ..., J-1; the jth encoding unit takes the low-dose PET image, MR image and spatial anatomical arrangement information of its corresponding size as input, and outputs the jth PET spatial enhanced image, j=1, 2, ..., J; The decoder comprises J-1 decoding units, wherein: the input of the J-1 decoding unit is the upsampled image of the J-th PET spatial enhancement image and the J-1-th PET spatial enhancement image, and the output is the J-1-th PET fusion prediction image; the j-th encoding unit takes the upsampled image of the j+1-th PET fusion prediction image and the j-th PET spatial enhancement image as input to obtain the j-th PET fusion prediction image, j=J-2, J-3, ..., 2, and The final PET fusion prediction image is used as the standard metrological PET enhanced image.

3. The method according to claim 1, characterized in that The spatial adjustment is performed using the anatomical structure spatial features obtained from the MR image and the spatial anatomical arrangement information, and the steps include: Based on spatial anatomical arrangement information s Get the affine parameter γ1, γ1 = H(f s ), H is the affine transformation; Based on the affine parameter γ1, the image feature f of the MR image is obtained m The scaled feature image And get the affine parameters (α, δ) of the scaled feature, Q is the affine transformation; Based on the affine parameters (α, δ), the image features f of the low-dose PET image are obtained p The zoomed and translated feature images Will As PET spatial enhancement image.

4. The method according to claim 1, characterized in that: The 3D neural network uses the MAE function as the loss function, specifically: Where: G represents the mapping relationship, θ i represents the network parameters, n is the total number of training samples, x i is a low-dose PET image data sample, y i is the low-dose PET image data sample x i The corresponding MR image data sample, z i For the low-dose PET image sample x i The corresponding spatial anatomical arrangement information, w i is the low-dose PET image data sample x i Corresponding standard-dose PET image samples.

5. The method according to claim 2, characterized in that: The decoding unit concatenates the two input images, and then processes them using convolutional layers and transposed convolutional layers to output a PET fusion prediction image.

6. A low-dose PET image enhancement system, characterized in that: The system includes a preprocessing module and a 3D neural network module; The preprocessing module is configured to obtain spatial anatomical arrangement information from the T1-weighted MR image; The 3D neural network module is configured to respectively acquire multi-scale features of low-dose PET images, MR images, and spatial anatomical arrangement information; For the low-dose PET image features of each scale, spatial adjustment is performed using the anatomical structure spatial features obtained from MR images and spatial anatomical arrangement information to obtain the PET spatial enhanced images corresponding to each scale; using each PET spatial enhanced image, scale-by-scale fusion prediction is performed to obtain the standard-dose PET enhanced image.

7. The system according to claim 6, characterized in that: The 3D neural network module includes an encoder and a decoder; The encoder includes J encoding units of different scales, the input of the jth encoding unit is downsampled to obtain the input of the j+1th encoding unit, j=1, 2, ..., J-1; the jth encoding unit takes the low-dose PET image, MR image and spatial anatomical arrangement information of its corresponding size as input, and outputs the jth PET spatial enhanced image, j=1, 2, ..., J; The decoder includes J-1 decoding units, wherein: the input of the J-1 decoding unit is the upsampled image of the J-1 PET spatial enhancement image and the J-1 PET spatial enhancement image, and the output is the J-1 PET fusion prediction image; the j-th encoding unit takes the upsampled image of the j+1 PET fusion prediction image and the j-th PET spatial enhancement image as input to obtain the j-th PET fusion prediction image, j=J-2, J-3, ..., 2, and takes the last PET fusion prediction image as the standard metrology PET enhanced image.

8. The system according to claim 6, characterized in that The spatial adjustment is performed using the anatomical structure spatial features obtained from the MR image and the spatial anatomical arrangement information, and the steps include: Based on spatial anatomical arrangement information s Get the affine parameter γ1, γ1 = H(f s ), H is the affine transformation; Based on the affine parameter γ1, the image feature f of the MR image is obtained m Zoomed feature image And get the affine parameters (α, δ) of the scaled feature, Q is the affine transformation; Based on the affine parameters (α, δ), the image features f of the low-dose PET image are obtained p The zoomed and translated feature images Will As PET spatial enhancement image.

9. The system according to claim 6, characterized in that: The 3D neural network module uses the MAE function as the loss function, specifically: Where: G represents the mapping relationship, θ i represents the network parameters, n is the total number of training samples, x i is a low-dose PET image data sample, y i is the low-dose PET image data sample x i The corresponding MR image data sample, z i For the low-dose PET image sample x i The corresponding spatial anatomical arrangement information, w i is the low-dose PET image data sample x i Corresponding standard-dose PET image samples.

10. The system according to claim 7, characterized in that: The decoding unit concatenates the two input images, processes them using convolutional layers and transposed convolutional layers, and outputs a PET fusion prediction image.

Images