Method and system for generating PET image based on MRI image

Abstract

The invention relates to a medical PET / MRI imaging technology, in particular to a method and a system for generating a PET image based on an MRI image, which are used for solving the problems of blurring, artifacts, detail missing, functional information missing and the like existing in generation from the MRI image to the PET image. According to the scheme, an anatomical structure feature image is obtained based on an MRI image of a patient; and generating a PET image of the patient based on the anatomical structure feature image and the coding control random noise image. According to the scheme, the MRI image is adopted as a structure guiding condition, and the functional PET image highly consistent with the original MRI in anatomical structure is effectively guided to be generated, so that a non-invasive, low-cost and high-reliability functional imaging substitution scheme is realized on the premise of no radioactive tracer agent.

Description

Technical Field

[0001] This disclosure relates to medical PET / MRI imaging technology, and more particularly to a method and system for generating PET images based on MRI images. Background Technology

[0002] Clinical diagnosis of neurodegenerative diseases (such as AD and PD) typically relies on the comprehensive analysis of multimodal medical imaging. Among these, MRI provides high-resolution anatomical information and is widely used for assessing changes in brain structure; while functional PET provides key functional information such as glucose metabolism or dopaminergic system activity and is currently recognized as an important tool for the early identification of pathological changes in neurodegenerative diseases.

[0003] However, in real-world clinical settings, functional PET imaging relies on specific radioactive tracers (such as […]). 18 F]FDG、[ 11 C]CFT、[ 18 PET scans (such as FP-CIT) not only pose radiation exposure risks but also require specialized radiopharmaceutical synthesis and imaging equipment. This makes it difficult to obtain complete functional imaging data from patients in many primary healthcare institutions or follow-up settings. Furthermore, the high cost and complex procedures of PET scans limit their widespread application in large-scale screening and long-term follow-up.

[0004] Therefore, exploring a technology for automatically generating PET functional images based on MRI images has become a research hotspot in the field of intelligent multimodal image synthesis in recent years. This "modal conversion" method can synthesize realistic functional images without the need for tracer injection, which reduces patient risk and has potential cost-effectiveness and scalability.

[0005] Currently, some studies have attempted to use deep learning techniques to synthesize PET images from MRI images to address the shortcomings of clinical functional imaging. Representative methods include modality transfer techniques based on generative adversarial networks (GANs) and their variants.

[0006] For example, Chinese patent application CN202011435776A proposes a "Method and System for MRI-PET Image Modal Conversion Based on Cyclic Generative Adversarial Networks". This method is based on a Cycle-GAN architecture and designs a bidirectional generative network to achieve mutual conversion between MRI and PET images without registration constraints. Although this method achieves the mapping relationship between image modalities to a certain extent, because its generation mechanism fails to fully constrain the consistency of image structure, the generated PET images often have problems such as blurring, artifacts, and missing details, making it difficult to provide reliable support in complex clinical anatomical scenarios.

[0007] In addition, Zhenrong Shen et al. published a paper in the *European Journal of Nuclear Medicine and Molecular Imaging* in 2025 entitled "Cross-modality PET image synthesis for Parkinson's Disease diagnosis: a leap from [ 18 F]FDG to [ 11 In C]CFT, a scheme based on cross-tracer PET image synthesis was proposed. This study utilizes deep neural networks to [ 18 F]FDG image generation[ 11 C]CFT images have improved the accuracy of Parkinson's disease diagnosis. However, this method only applies to the conversion between functional PET tracers and does not cover the cross-modal conversion needs from structural MRI to functional PET images, thus failing to address the lack of functional information in structural images. Summary of the Invention

[0008] The purpose of this disclosure is to propose a method and system for generating PET images based on MRI images, aiming to generate synthetic PET images with high resolution, high anatomical consistency and high functional expression capabilities, overcome the limitations of existing methods in converting MRI images to PET images with low imaging accuracy and poor interpretability, and promote the non-invasive and intelligent development of nuclear medicine imaging technology.

[0009] To achieve the above objectives, this disclosure proposes a method for generating PET images based on MRI images. The method includes the following steps: obtaining anatomical feature images based on the patient's MRI images; and generating PET images of the patient based on the anatomical feature images by encoding and controlling random noise images.

[0010] In one embodiment of the above technical solution, the step of acquiring the anatomical structure feature image includes: extracting the region of interest (ROI) mask image based on the patient's MRI image; and extracting features through multiple convolutional layers based on the MRI image and the ROI mask image to acquire the anatomical structure feature image.

[0011] In one embodiment of the above technical solution, the encoding control includes the following steps: obtaining encoder E1 by copying encoder E2 of the Stable Diffusion model; constructing a ControlNet by adding a zero convolutional layer after encoder E1; and making skip connections between its output and the layers corresponding to decoder D of the Stable Diffusion model; if the termination condition is not met, repeatedly executing the process of using the fused image of the anatomical structure feature image and the random noise image as input to the ControlNet, while using the random noise image as input to encoder E2, and updating the random noise image using the output of decoder D; if the termination condition is met, decoding the output of decoder D to obtain a PET image corresponding to the MRI image.

[0012] In one embodiment of the above technical solution, the decoding of the output of decoder D is performed using an image decoder, which includes several residual blocks.

[0013] To achieve the above objectives, this disclosure also proposes a system for generating PET images based on MRI images. The system includes a first generation module and a second generation module. The first generation module is configured to acquire anatomical feature images based on the patient's MRI images. The second generation module is configured to generate PET images of the patient based on the anatomical feature images by encoding and controlling random noise images.

[0014] In one embodiment of the above technical solution, the first generation module includes a first extraction unit and a second extraction unit; the first extraction unit is configured to extract the region of interest (ROI) based on the patient's MRI image to obtain an ROI image; the second extraction unit is configured to perform feature extraction through multiple convolutional layers based on the MRI image and the ROI mask image to obtain an anatomical structure feature image.

[0015] In one embodiment of the above technical solution, the second generation module includes a denoising unit; the denoising unit is configured to obtain encoder E1 by copying encoder E2 of the Stable Diffusion model, construct a ControlNet by adding a zero convolutional layer after encoder E1, and make skip connections between its output and the layers corresponding to decoder D of the Stable Diffusion model; if the termination condition is not met, the fusion image of the anatomical structure feature image and the random noise image is repeatedly used as the input of ControlNet, while the random noise image is used as the input of encoder E2, and the output of decoder D is used to update the random noise image; if the termination condition is met, the output of decoder D is decoded to obtain a PET image corresponding to the MRI image.

[0016] The beneficial technical effects of this disclosure are as follows: by making full use of the anatomical information of MRI images, anatomical structural feature images are obtained based on the patient's MRI images, and based on the encoding of the anatomical structural feature images, functional PET images that are highly consistent with the original MRI in terms of anatomical structure are generated; key details are preserved, excessive image smoothing is avoided, and image quality is improved; and a non-invasive, low-cost, and highly reliable functional imaging alternative for the early identification of neurodegenerative diseases is provided without the need for radioactive tracers. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the model architecture in one implementation method.

[0019] Figure 2 This is a comparison chart of the average PSNR for each method in one implementation method.

[0020] Figure 3 This is a comparison chart of the average SSIM values ​​for each method in one implementation method. Detailed Implementation

[0021] Early diagnosis of neurodegenerative diseases such as Parkinson's disease (PD) and Alzheimer's disease (AD) heavily relies on the combined analysis of functional nuclear medicine imaging (such as FP-CIT PET and FDG PET) and structural magnetic resonance imaging (MRI). PET provides functional information on brain metabolism and neurotransmitter systems, exhibiting high sensitivity and specificity in early disease identification; however, its reliance on radioactive tracers makes PET examinations costly, poses radiation exposure risks, and has a complex procedure, limiting its widespread use in large-scale screening and follow-up scenarios. In contrast, MRI offers advantages such as high spatial resolution, no radiation, and broad applicability, and is widely used in clinical practice; however, MRI cannot reflect the activity of the brain's dopamine system or the state of glucose metabolism, making it insufficient to meet the functional diagnostic needs of neurodegenerative diseases.

[0022] To overcome the aforementioned problems, previous studies have attempted to use methods such as Generative Adversarial Networks (GANs) to achieve image modality conversion from MRI to PET. However, these methods suffer from issues such as image blurring, significant artifacts, loss of detail, and poor consistency in anatomical structures. The generated images also show deviations in metabolic distribution in key brain regions compared to real PET images, exhibiting anatomical misalignment or functional distortion, making it difficult to meet the clinical usability requirements of functional images. Therefore, a new image conversion technology is urgently needed to synthesize PET images with high functional expressiveness and anatomical consistency without the use of radioactive tracers, thereby improving the early non-invasive diagnostic capabilities for neurodegenerative diseases.

[0023] Based on this, this disclosure proposes a technical solution for generating PET images based on MRI images. In this solution, anatomical structural feature images are obtained from the patient's MRI images. Based on these anatomical feature images, random noise images are encoded to generate PET images of the same patient, thereby effectively guiding the generation of functional PET images that are highly consistent with the original MRI in terms of anatomical structure. Specifically, this solution introduces conditional coding to fuse anatomical feature information from MRI during diffusion modeling, achieving pixel-level functional image reconstruction. This method not only improves the spatial resolution of the generated images but also preserves key metabolic features and tissue boundary details, effectively improving the problems of image blurring and information loss in traditional methods.

[0024] This method can generate high-quality, structurally consistent, and functionally accurate PET images without the need for injecting radioactive tracers, thus providing reliable imaging support for the early non-invasive diagnosis of various neurodegenerative diseases such as Alzheimer's disease and Parkinson's disease.

[0025] The following description, in conjunction with the accompanying drawings, clearly and completely describes how the technical solution of this case is implemented. Obviously, the described embodiments are only a part of the embodiments of this case, and not all of them. Based on the embodiments in this case, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0026] (a) Data preprocessing First, data acquisition is performed. Specifically, PET / MR scan data from Parkinson's disease patients are used to acquire paired [data / images / images]. 18 F]FP-CIT PET (dopamine transporter imaging) and T1-weighted MRI images were used as a sample. All image data were ensured to be time-synchronized during acquisition and spatially aligned through rigid registration.

[0027] Secondly, image preprocessing is performed. This preprocessing includes resampling and normalization. Resampling involves resampling the PET and MRI images to ensure they have the same size and spatial resolution, guaranteeing spatial consistency between images. Normalization involves normalizing the MRI and PET images on a patient-by-patient basis to eliminate brightness differences between different image layers, preventing inconsistent brightness during image generation.

[0028] Furthermore, to more accurately reproduce the details of regions of interest (ROIs) when generating PET images, thereby improving the clinical diagnostic value of the generated images, regions of interest (ROIs) are extracted from MRI images. In machine vision and image processing, the ROI is the area to be processed outlined from the image being processed using rectangles, circles, ellipses, irregular polygons, etc. For example, the FSL segmentation tool is used to extract ROIs from the MRI image dataset as a mask. A mask is a template used to occlude or protect specific areas of an image or material surface. The mask can be used to occlude the image, thereby controlling the area or process of image processing. In this scheme, the ROI mask clearly indicates the areas that the model needs to focus on during training and generation. By using the ROI mask as additional input information, the model can better learn the characteristics of these areas and more accurately reproduce the details of these areas when generating PET images, thereby improving the clinical diagnostic value of the generated images.

[0029] (II) Design of Condition-Guided Mechanism Designed as follows Figure 1 The model architecture shown is based on the Stable Diffusion architecture and introduces conditional coding through the ControlNet mechanism to control the image generation process of Stable Diffusion, thereby achieving high-quality PET image generation. Specifically, it includes a Feature Extractor, a ControlNet module, a Stable Diffusion module, and an Image Decoder.

[0030] (2.1) Condition preprocessing The input MRI image and ROI mask image are dimensionality-reduced using a feature extractor, and their size is reduced to match the input space size of the Stable Diffusion model, ultimately yielding an anatomical structure feature image. This anatomical structure feature image is a structure-guided condition obtained based on the MRI image, and its features are called conditional features. .

[0031] To obtain sufficient structural detail features from MRI images, the feature extractor consists of multiple convolutional layers and downsampling layers. The number of convolutional layers is at least 3.

[0032] One implementation method, see Figure 1 The module representation shows that the feature extractor's specific structure from input to output is: CCD-CD-CD-Z. Here, C represents a convolutional layer, D represents downsampling, and Z represents a zero convolutional layer. The zero convolutional layer initializes the weights of the conditional inputs to zero. This design ensures that in the early stages of training, the influence of the conditional inputs on the generation process is controllable and does not interfere with the basic Stable Diffusion model structure. As training progresses, the weights of the zero convolutional layer are gradually updated, allowing the model to effectively utilize the conditional inputs to improve the generated results.

[0033] (2.2) Encoding control of target image generation Encoder E1 is obtained by copying encoder E2 from the U-Net architecture of the Stable Diffusion model in this application. A zero-convolutional layer is then added after encoder E1 to obtain the ControlNet module. Its output is then skip-connected to the layers corresponding to the decoder D of the Stable Diffusion model. These zero-convolutional layers are initialized with zero weights to ensure that the influence of conditional inputs on the generation process is controllable in the early stages of training and does not interfere with the basic Stable Diffusion model structure. As training progresses, the weights of the zero-convolutional layers are gradually updated, enabling the model to effectively utilize conditional inputs to improve the generated results.

[0034] The ControlNet module encodes a fused image of anatomical structural features and random noise. ControlNet essentially introduces additional conditional information into the Stable Diffusion U-Net architecture, improving the controllability and quality of the generated images. The random noise image is generated during the training phase from PET images, which, along with MRI images, form a sample set.

[0035] Since encoder E1 and encoder E2 have the same structure and the same initial weight parameters, the conditional control vector output by encoder E1 and the feature vector output by encoder E2 after downsampling are the same in number and the same in dimension.

[0036] This invention employs Stable Diffusion as the backbone network for image generation, consisting of an encoder E2 and a decoder D. During the back-diffusion process, encoder E2 receives a random noise image as input, and decoder D outputs a one-step denoising result. Specifically, during the upsampling process of decoder D, each upsampled feature layer is not only added and fused with the corresponding layer's encoding vector output by ControlNet, but also skipped connections with the downsampled features of the same layer in encoder E2, thereby fusing the anatomical structural features of the MRI image. The image generation process of the Stable Diffusion model is controlled by the conditional encoding of the corresponding layer output by ControlNet. This conditional guidance allows the model to simultaneously consider the structural information of the MRI image and the detailed features of the Region of Interest (ROI) during image generation, generating PET images with consistent structure and ROI details, achieving pixel-level functional image reconstruction. In this way, ControlNet continuously adjusts the generation weights of the Stable Diffusion module, ensuring that the generated PET images are highly consistent with the conditional features in terms of structure and detail, significantly improving the quality of the generated images and their clinical diagnostic value.

[0037] In one implementation, the above configuration is used as a denoising unit. The denoising unit is configured to obtain encoder E1 by copying encoder E2 of the Stable Diffusion model, construct a ControlNet by adding a zero convolutional layer after encoder E1, and then skip-connect its output to the layers corresponding to decoder D of the Stable Diffusion model.

[0038] The generation process of the target image is an iterative denoising process. In one implementation, a preset number of denoising steps T is set. If the termination condition is not met, the process is repeated, using the fused image of the anatomical structure feature image and the random noise image as input to ControlNet, while using the random noise image as input to encoder E2, and updating the random noise image using the output of decoder D. When the termination condition is met, the output of decoder D is decoded to obtain the PET image corresponding to the MRI image.

[0039] (2.3) Target Image Generation The final PET image is obtained by decoding the output of decoder D using an image decoder. The image decoder progressively improves the spatial resolution of the image through multiple deconvolution operations, obtaining a PET image that is highly consistent with the input MRI and ROI mask images in terms of structure and detail.

[0040] use Figure 1The module identifiers in the code are as follows: The specific structure of the decoder from input to output is: C-RRRU-RRRU-RRRU-RRR-GSC. Here, R represents the ResNet Block, U represents upsampling, G represents group normalization, and S represents the nonlinear transformation function (Swish).

[0041] The residual block consists of two concatenated convolutional modules, with the input and output directly added together for residual learning. Each convolutional module sequentially performs group normalization, calculates a nonlinear transformation function based on the input, and then convolves the results.

[0042] (III) Model Training and Optimization Process The training objective of the model is to optimize the generation process by learning how to recover high-quality PET images from noise. The training process mainly includes forward diffusion and backward diffusion.

[0043] Forward diffusion process: The standard diffusion model forward process is used to process the original image. A progressive Gaussian noise addition process is applied, causing the image to gradually lose detail and eventually become pure noise. The forward diffusion process can be represented by the following formula:

[0044] in, In time step The image, It is the noise attenuation coefficient. It is standard Gaussian noise.

[0045] Backdiffusion process: Starting with pure noise, the noise is gradually removed to recover a clear image. The backdiffusion process can be represented by the following formula:

[0046] in, It is the noise increase factor. It is noise in the model prediction.

[0047] The formula for generating the entire image is:

[0048] in, It is a target-synthesized PET image. It is input noise. and It is a zero convolutional layer. It is the feature extraction process. and It is the UNet encoder part in the reverse diffusion process. It is the UNet decoder part in the reverse diffusion process. It is the image decoding process. These are the parameters for generating the image model.

[0049] To optimize the model's generation performance, an L2-based noise prediction loss function is adopted, with the goal of minimizing the mean squared error (MSE) between the predicted noise and the actual noise.

[0050] in, It is the initial noise. It is a time step. It is a conditional feature. It is standard Gaussian noise. It is noise in the model prediction. The noise at time step t.

[0051] (iv) Model Inference and Performance Verification The trained model performs inference by taking a new MRI image and its corresponding ROI mask as input, and combining it with random noise images to generate a corresponding PET image. The generated image is then compared with a real PET image to evaluate the model's performance.

[0052] The quality of the generated images was evaluated using PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity) metrics. To validate the model's generalization ability, we used a separate test set to ensure that the model could generate high-quality PET images on unseen MRI data.

[0053] For quantitative evaluation, the image quality of the generated delayed-phase PET images can be assessed using two metrics: PSNR and SSIM. Validation was performed on a large patient dataset, and the quantitative metric test results are shown in Table 1 below.

[0054] Table 1. Comparison of quantitative indicators between the method of this application and existing methods on the test set.

[0055] As shown in Table 1, the method provided in this application (Ours) generates images with the highest peak signal-to-noise ratio (PSNR) and the highest structural similarity index (SSIM). In other words, it outperforms other comparative models in all image quality evaluation metrics, especially in terms of PSNR, which indicates that the generated images have higher fidelity in terms of structural consistency and metabolic reduction.

[0056] (V) Output Results and Clinical Application Prospects Through the above technical process, this invention enables high-quality synthesis of functional PET images without invasiveness, low cost, or radiation exposure. This method offers significant advantages in terms of structure preservation, functional representation, and diagnostic readability, and has broad application potential in the early screening and auxiliary diagnosis of neurodegenerative diseases such as Alzheimer's and Parkinson's.

[0057] Through the above description of the embodiments, those skilled in the art can clearly understand that, based on the above content, a method for generating PET images based on MRI images can be implemented. The steps of the method include: acquiring anatomical structure feature images based on the patient's MRI images; and generating PET images of the patient by encoding and controlling random noise images based on the anatomical structure feature images. The step of acquiring the anatomical structure feature images includes: extracting regions of interest (ROIs) from the patient's MRI images to obtain ROI mask images; and extracting features from the MRI images and ROI mask images through multiple convolutional layers to obtain the anatomical structure feature images.

[0058] A corresponding system can be implemented based on the method for generating PET images from MRI images. For example, a system for generating PET images from MRI images includes a first generation module and a second generation module; wherein: the first generation module is configured to acquire anatomical structure feature images based on a patient's MRI image; the second generation module is configured to generate a PET image of the patient by encoding and controlling random noise images based on the anatomical structure feature images. The first generation module includes a first extraction unit and a second extraction unit; the first extraction unit is configured to extract regions of interest (ROIs) from the patient's MRI image to obtain ROI images; the second extraction unit is configured to perform feature extraction through multiple convolutional layers based on the MRI image and the ROI mask image to acquire the anatomical structure feature images.

[0059] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods and / or systems of this disclosure can be implemented by means of software plus necessary general-purpose hardware, or by special-purpose hardware including application-specific integrated circuits, dedicated CPUs, dedicated memory, dedicated components, etc. Generally, any function performed by a computer program can be easily implemented using corresponding hardware, and the specific hardware structure used to implement the same function can be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the purposes of this disclosure, software program implementation is more often a preferred implementation method.

[0060] Although the embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this disclosure is not limited to the specific embodiments and application fields described above. The specific embodiments described above are merely illustrative and instructive, and not restrictive. Those skilled in the art can make many other forms based on the guidance of this specification and without departing from the scope of protection of the claims of this disclosure, and all of these are within the scope of protection of this disclosure.

Claims

1. A method for generating PET images based on MRI images, characterized in that, The steps of the method include: Based on the patient's MRI images, obtain images of anatomical features; Based on the anatomical feature images, the PET images of the patient are generated by encoding and controlling random noise images.

2. The method according to claim 1, characterized in that, The steps for acquiring the anatomical structure feature image include: Based on the patient's MRI images, the region of interest (ROI) is extracted to obtain a ROI mask image; Based on the MRI images and ROI mask images, feature extraction is performed through multiple convolutional layers to obtain anatomical structure feature images.

3. The method according to claim 1, characterized in that, The encoding control includes the following steps: Encoder E1 is obtained by copying encoder E2 of the Stable Diffusion model. After encoder E1, a zero convolutional layer is added to construct ControlNet, and its output is skipped to the layers corresponding to decoder D of the Stable Diffusion model. If the termination condition is not met, the process is repeated, using the fused image of the anatomical feature image and the random noise image as input to ControlNet, while using the random noise image as input to encoder E2, and updating the random noise image using the output of decoder D. When the termination condition is met, the output of decoder D is decoded to obtain a PET image corresponding to the MRI image.

4. The method according to claim 3, characterized in that, The decoding of the output of decoder D is performed using an image decoder, which includes several residual blocks.

5. A system for generating PET images based on MRI images, characterized in that, The system includes a first generation module and a second generation module; wherein: The first generation module is configured to acquire anatomical feature images based on the patient's MRI images; The second generation module is configured to generate the patient's PET image based on the anatomical feature image and encoded control random noise image.

6. The system according to claim 5, characterized in that, The first generation module includes a first extraction unit and a second extraction unit; The first extraction unit is configured to extract the region of interest (ROI) image based on the patient's MRI image; The second extraction unit is configured to extract features from the MRI image and the ROI mask image through multiple convolutional layers to obtain anatomical structure feature images.

7. The system according to claim 5, characterized in that, The second generation module includes a noise reduction unit; The denoising unit is configured to obtain encoder E1 by copying encoder E2 of the Stable Diffusion model, add a zero convolutional layer after encoder E1 to construct ControlNet, and make skip connections between its output and the layers corresponding to decoder D of the Stable Diffusion model. If the termination condition is not met, the process is repeated, using the fused image of the anatomical feature image and the random noise image as input to ControlNet, while using the random noise image as input to encoder E2, and updating the random noise image using the output of decoder D. When the termination condition is met, the output of decoder D is decoded to obtain a PET image corresponding to the MRI image.

8. The system according to claim 7, characterized in that, The decoding of the output of decoder D is performed using an image decoder, which includes several residual blocks.

Images