Multi-tracer universal PET-MR image correction method and storage medium

Abstract

The invention provides a multi-tracer universal PET-MR image correction method and a storage medium, and the method comprises a first image acquisition step, a first image preprocessing step, an encoder construction step, a first image reconstruction step, a second image reconstruction step, a second image acquisition step, and a second image preprocessing step. The method comprises the following steps: a first multi-tracer universal PET-MR shooting correction framework is provided, feature structure item guidance containing anatomy participation information is added in a PET-MR correction process, attenuation knowledge and anatomy reference information are migrated from CT to MR, and a multi-scale pixel-level adaptive expert module is provided. Conflicting tasks are distributed through a multi-scale path to reduce interference among tracers, and cross-modal quantitative alignment of PET-MR and PET-CT is achieved through a Unet network. The multi-tracer PET-MR image correction method has the advantages that the method is suitable for multi-tracer PET-MR image correction, the quantization difference between the multi-tracer PET-MR image and the PET-CT image is reduced, and the correction quality of the PET-MR image is remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of machine learning, and in particular to a multi-tracer universal PET-MR image correction method. Background Art

[0002] The fusion of positron emission tomography (PET) and magnetic resonance imaging (MR) (PET-MR) represents a revolutionary advancement in clinical molecular imaging, attracting extensive research attention and expanding clinical applications. This hybrid system offers unique advantages, such as the superior soft tissue contrast provided by MRI, reduced ionizing radiation exposure, and the ability to simultaneously acquire metabolic function and anatomical information.

[0003] In clinical practice, radiologists often assume that quantitative results from PET-MR and PET-coupled computed tomography (PET-CT) are equivalent. Consequently, they often directly apply quantitative criteria established based on PET-CT to interpret PET-MR images. PET-CT remains the gold standard for molecular imaging due to its high sensitivity and reliable quantitative assessment of physiological and pathological processes, particularly in neurology. However, systematic clinical evaluations have demonstrated significant discrepancies between standardized uptake values ​​(SUVRs) between PET-MR and PET-CT for different tracers. This quantitative discrepancy may stem from inherent limitations of MR-based attenuation correction (AC) methods. In PET-CT systems, the X-ray attenuation coefficient of CT is directly related to tissue electron density, ensuring accurate attenuation correction and reliable PET reconstruction. In contrast, MR signal intensity lacks a direct correlation with electron density or atomic composition, forcing AC algorithms to estimate attenuation properties through segmentation or atlas-based templates, an approximation that introduces systematic quantification errors. These discrepancies are often overlooked in clinical practice, and direct application of PET-CT quantitative criteria may lead to diagnostic bias in PET-MR. Therefore, there is an urgent need to develop methods that can correct PET-MR images across tracers to align PET-CT.

[0004] Deep learning-based image restoration (IR) technology can transform low-quality medical images into high-quality images, providing a viable solution to the aforementioned problems. In the field of PET imaging, studies have successfully utilized IR technology to reconstruct standard-dose images from low-dose PET. However, these methods primarily rely on single PET data and ignore the anatomical prior information in structural images. Furthermore, most PET reconstruction methods are designed for a single tracer, and their performance significantly degrades in cross-tracer scenarios due to differences in uptake distribution caused by the biochemical properties of different tracers. Given that the same PET system often needs to scan multiple tracers in clinical practice, it is inefficient to deploy and maintain a separate model for each tracer. Therefore, the development of a universal correction algorithm for multiple tracers is of great clinical significance. Summary of the Invention

[0005] This application provides a universal multi-tracer PET-MR image correction method to address the problem that existing PET reconstruction can only target a single tracer, cannot reduce the quantitative difference between multi-tracer PET-MR images and PET-CT images, and thus affects the correction quality of PET-MR images.

[0006] The present application proposes a universal multi-tracer PET-MR image correction method, which specifically includes a first image acquisition step, a first image preprocessing step, an encoder construction step, a first image reconstruction step, a second image reconstruction step, a second image acquisition step, a second image preprocessing step, a neural network construction step, and an image input step.

[0007] The first image acquisition step is used to acquire a head CT image and a head MR image of a subject; the first image preprocessing step is to preprocess the head CT image and the head MR image; the encoder construction step is to construct a first encoder and a second encoder, and the first encoder is trained using a pixel-level L2 loss function, and the second encoder is trained using a contrastive learning paradigm; the first image reconstruction step is to input the preprocessed head CT image into the trained first encoder, extract and encode the anatomical structure information in the preprocessed head CT image, and obtain CT features; the second image reconstruction step is to input the preprocessed head MR image into the trained second encoder, and convert the preprocessed head MR image into MR features aligned with the CT features; the second image acquisition step is to obtain paired PET-MR images and PET-CT images of multiple tracers based on the obtained mutually aligned CT features and MR features; the second image preprocessing step is to preprocess the PET-MR images and the PET-CT; the neural network construction step is to initialize a Unet network specifically for PET-MR image correction, the Unet network introduces a channel attention mechanism, selects different channel weights for different tracers, and uses a cross-attention mechanism to train the Unet network based on the obtained MR features; the image input step is to input the PET-MR image into the trained Unet network to obtain a corrected PET-MR image.

[0008] Furthermore, the first image preprocessing step specifically includes an image quality control step and an image normalization step.

[0009] The image quality control step is used to check various image indicators and eliminate images with unqualified quality; the image normalization step is to adjust the pixel values ​​of qualified images to a preset threshold range.

[0010] Furthermore, the first encoder and the second encoder are autoencoders based on the Vision Transformer architecture.

[0011] Furthermore, in the encoder construction step, specific steps of training the second encoder include a first encoder freezing step, a feature alignment step, and an information overfitting offset step.

[0012] The first encoder freezing step is used to freeze the first encoder; the feature alignment step is based on obtaining the CT feature and using a positive loss function to align the MR feature with the patch at the same position in the CT feature at the pixel level; the information overfitting offset step uses a negative loss function to offset the overfitting of semantic information caused by further alignment between patches with non-corresponding positions.

[0013] Furthermore, the second image preprocessing step specifically includes an image quality control step and an image processing step.

[0014] The image quality control step is used to check various image indicators and eliminate images with unqualified quality; the image processing step is used to pre-process qualified images and normalize the images.

[0015] Furthermore, the Unet network includes a multi-scale pixel-level expert model, which can assign each pixel of the input PET-MR image to the most appropriate expert network for processing based on the routing instructions at the original image scale.

[0016] Furthermore, in the neural network construction step, specific steps of training the Unet network include a second encoder freezing step, an output interaction step, a contribution degree calculation step, and a feature fusion step.

[0017] The second encoder freezing step is used to freeze the second encoder; the output interaction step is based on the obtained MR features, and uses the cross-attention mechanism to embed the output of the second encoder into the bottom layer of the Unet network, so that the MR features output by the second encoder interact with the input features of the bottom layer of the Unet network, and the input features of the bottom layer of the Unet network are PET-MR images; the contribution degree calculation step is to determine the contribution degree of the MR features in the input features of the bottom layer of the Unet network by calculating the attention weight, and the formula is: ; Among them, Q represents the input features of the Unet bottom layer, K represents the learnable projection of MR features, To prevent the scaling factor from vanishing gradients, Softmax represents the activation function, and Attention scores represent the degree of contribution. The feature fusion step is to fuse the weighted MR features with the input features of the bottom layer of the Unet network to obtain a corrected PET-MR image.

[0018] The present invention also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are read by at least one processor, the at least one processor is caused to execute at least one step of the multi-tracer universal PET-MR image correction method.

[0019] The present application provides a universal multi-tracer PET-MR image correction method and storage medium, innovatively proposing the first universal multi-tracer PET-MR uptake correction framework, adding characteristic structure terms containing anatomical reference information to guide the PET-MR correction process, migrating attenuation knowledge and anatomical reference information from CT to MR, and providing a multi-scale pixel-level adaptive expert module. By allocating conflicting tasks through multi-scale paths to reduce interference between tracers, the Unet network is used to achieve cross-modal quantitative alignment of PET-MR and PET-CT, thus resolving the problem that existing PET reconstruction can only be performed for a single tracer and is not suitable for multi-tracer PET-MR image correction. The method reduces the quantitative difference between multi-tracer PET-MR images and PET-CT images, significantly improving the correction quality of PET-MR images. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of the multi-tracer universal PET-MR image correction method described in an embodiment of the present application; Figure 2 is a flowchart of the first image preprocessing step described in an embodiment of the present application; Figure 3 is a schematic diagram of a multi-tracer universal PET-MR image correction method according to an embodiment of the present application; Figure 4 is a flowchart of the encoder construction steps described in an embodiment of the present application; Figure 5 is a flowchart of the second image preprocessing step described in an embodiment of the present application; Figure 6 is a flowchart of the steps of constructing a neural network according to an embodiment of the present application; Figure 7 This is a comparison chart of the results of the multi-tracer universal PET-MR image correction method described in the embodiment of the present application and the experimental results of other methods; Figure 8 It is a schematic diagram of the storage medium described in an embodiment of the present application.

[0022] Description of reference numerals: 100 storage medium, 110 processor. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application. The present application provides one, which is described in detail below. It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments of the present application. In the following embodiments, the description of each embodiment has its own emphasis. For the parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0024] like Figure 1 As shown, the present application provides a multi-tracer universal PET-MR image correction method, which specifically includes step S1) a first image acquisition step, step S2) a first image preprocessing step, step S3) an encoder construction step, step S4) a first image reconstruction step, step S5) a second image reconstruction step, step S6) a second image acquisition step, step S7) a second image preprocessing step, step S8) a neural network construction step, and step S9) an image input step.

[0025] Step S1) The first image acquisition step is to acquire a head CT image and a head MR image of the subject.

[0026] Step S2) A first image preprocessing step is to preprocess the head CT image and the head MR image.

[0027] like Figure 2 As shown, step S2) the first image preprocessing step specifically includes step S21) an image quality control step and step S22) an image normalization step.

[0028] Step S21) Image quality control step: Check various image indicators and eliminate images with unqualified quality. When performing quality control on the subject's head CT images and head MR images, the operation is consistent. That is, the image clarity, contrast, noise level and other indicators are carefully checked, and images with artifacts, blur or other quality issues are eliminated to ensure the accuracy and reliability of subsequent analysis.

[0029] Step S22) Image normalization is to adjust the pixel values ​​of the qualified image to within a preset threshold range to facilitate subsequent calculation and analysis. The threshold range is set according to requirements.

[0030] Step S3) An encoder construction step is performed to construct a first encoder and a second encoder. The first encoder is trained using a pixel-level L2 loss function. The L2 loss function can measure the difference between the reconstructed image and the original image at each pixel. By minimizing this difference, the autoencoder can better learn the features and structure of the CT image, thereby improving the performance and accuracy of the encoder. The second encoder is trained using a contrastive learning paradigm. The first encoder and the second encoder are autoencoders based on the Vision Transformer architecture.

[0031] like Figure 3 As shown, in this embodiment, the Vision Transformer architecture has powerful feature extraction and representation capabilities, capable of capturing the complex structures and features in CT images. By training the autoencoder to reconstruct CT images, in each iteration, the autoencoder attempts to learn the inherent features and structure of the input CT image and reconstruct an image as similar as possible to the original image based on this learned information. This results in a Vision Transformer encoder with CT anatomical representation capabilities, which can effectively extract and encode anatomical structure information in CT images.

[0032] like Figure 4 As shown, in the step S3) encoder construction step, the specific steps of training the second encoder in step S3) include step S31) first encoder freezing step, step S32) feature alignment step and step S33) information overfitting offset step.

[0033] Step S31) A first encoder freezing step is to freeze the first encoder and retain the output features of the first encoder.

[0034] Step S32) Feature alignment step: Based on the obtained CT features, a positive loss function is used to align the MR features with the patch at the same position in the CT features at the pixel level.

[0035] Step S33) Information overfitting offset step, using a negative loss function to offset the semantic information overfitting caused by further alignment between patches that do not correspond to each other in position.

[0036] In this embodiment, a trained CT encoder is used to generate anatomical reference features, and then the trained CT encoder is frozen. The same architecture is used to train the MR encoder to convert the MR image into MR features that are aligned with the CT anatomical reference features. The training goal is to align the output of the MR encoder with the output of the frozen CT anatomical reference feature encoder. During the training process, a contrastive learning paradigm is used to achieve patch-level feature alignment. In the contrast loss, the positive loss function encourages pixel-level alignment of MR features with patches at the same position in the CT features, while the negative loss function offsets the overfitting of semantic information caused by further alignment between patches with non-corresponding positions. Through such training, although the MR encoder uses MR features as output, its output features can learn the attenuation knowledge and anatomical reference features of the CT encoder.

[0037] Step S4) a first image reconstruction step, inputting the pre-processed head CT image into the trained first encoder, extracting and encoding the anatomical structure information in the pre-processed head CT image to obtain CT features.

[0038] Step S5) The second image reconstruction step is to input the preprocessed head MR image into the trained second encoder, and convert the preprocessed head MR image into MR features aligned with the CT features.

[0039] Step S6) a second image acquisition step of acquiring a paired PET-MR image and a PET-CT image of the multi-tracer based on the obtained aligned CT features and MR features; Step S7) The second image preprocessing step is to preprocess the PET-MR image and the PET-CT image.

[0040] like Figure 4 As shown, step S7) the second image preprocessing step specifically includes step S71) an image quality control step and step S72) an image processing step.

[0041] Step S71) Image quality control step, checking various image indicators and eliminating images with unqualified quality.

[0042] Step S72) Image processing step: pre-processing the qualified image, including skull peeling, normalization, and slicing, and normalizing the image.

[0043] Step S8) A neural network construction step is performed to initialize a Unet network specifically for PET-MR image correction. The Unet network introduces a channel attention mechanism, selects different channel weights for different tracers, and trains the Unet network using a cross-attention mechanism based on the obtained MR features.

[0044] Another example Figure 2 As shown in this embodiment, the Unet network, a convolutional neural network with an encoder-decoder structure, has a unique architecture that effectively captures image context and spatial features. To mitigate cross-tracer interference in shared network parameters, a channel attention mechanism is introduced into the Unet network. This mechanism allows the network to flexibly select different channel weights for different tracers. During training and calibration of the Unet network, the cross-attention mechanism is used to embed the output of the MR encoder—the MR features that have learned CT attenuation knowledge—into the bottom layer of the Unet network. As a method for establishing associations between different feature representations, the cross-attention mechanism can adaptively adjust the feature representations at the bottom layer of the Unet network based on the importance of MR features.

[0045] Furthermore, the Unet network includes a multi-scale pixel-level expert model, which can assign each pixel of the input PET-MR image to the most appropriate expert network for processing based on the routing instructions at the original image scale.

[0046] In this embodiment, a multi-scale pixel-level expert model employs a Mixture of Experts (MoE) design approach through a multi-scale pixel routing (MsPR) module. This approach aims to mitigate cross-tracer interference in shared network parameters through a routing instruction assignment mechanism. Considering the fine-grained nature of the tracer uptake correction task, the MsPR module assigns each pixel of the input PET-MR image to the most appropriate expert network for processing based on the routing instructions. Specifically, the MsPR module processes the residual feature map via n parallel multi-scale expert networks. The multi-scale pixel-level expert model generates routing instructions through the following steps: first, the PET-MR image is encoded into an n-channel feature map and a softmax activation function is applied along the channel dimension. A top-K selection mechanism (K=4) is then applied to each spatial location in the feature map, masking out NK channel values ​​along the channel dimension. After top-K selection, the feature maps of different channels are assigned as gating maps to the corresponding expert networks. This adaptive routing strategy enables the network to dynamically integrate effective features from different spatial locations to generate optimal residual images for different tracers, significantly improving uptake correction accuracy.

[0047] like Figure 6 As shown, in step S8) the neural network construction step, the specific steps of training the Unet network include step S81) a second encoder freezing step, step S82) an output interaction step, step S83) a contribution degree calculation step, and step S84) a feature fusion step.

[0048] Step S81) A second encoder freezing step is to freeze the second encoder and retain the output features of the second encoder.

[0049] Step S82) Output interaction step: Based on the obtained MR features, the output of the second encoder is embedded into the bottom layer of the Unet network using a cross-attention mechanism, so that the MR features output by the second encoder interact with the input features of the bottom layer of the Unet network, and the input features of the bottom layer of the Unet network are PET-MR images.

[0050] Step S83) Contribution calculation step: Determine the contribution of the MR feature to the input feature at the bottom layer of the Unet network by calculating the attention weight. The formula is: ; Among them, Q represents the input features of the Unet bottom layer, K represents the learnable projection of MR features, To prevent the gradient from disappearing, Softmax represents the activation function, and Attention scores represent the contribution.

[0051] Step S84) Feature fusion step, fusing the weighted MR features with the input features of the bottom layer of the Unet network to obtain a corrected PET-MR image.

[0052] Step S9) Image input step: input the PET-MR image into the trained Unet network to obtain a corrected PET-MR image.

[0053] like Figure 7 As shown in this example, the rectified multi-tracer head PET-MR image is subtracted from the corresponding PET-CT image to generate a residual map, where positive values ​​are represented by dark gray and negative values ​​by black. The data distribution of the residual maps generated by different models is plotted. It can be observed that compared with the residual maps generated by other models, the residual map of this model is the lightest, and the overall variance and mean of the pixels approach 0. This indicates that the residual between the PET-MR image rectified by this model and its corresponding PET-CT image is minimized, indicating the highest correction quality. In a quantitative evaluation, we used two commonly used image quality assessment metrics, peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), to assess the PET-MR correction quality of this model. In comparative experiments, this model achieved the highest PSNR and SSIM, demonstrating the effectiveness of this method.

[0054] like Figure 8As shown, the present invention further provides a storage medium 100 storing computer-readable instructions. When the computer-readable instructions are read by at least one processor 110, the at least one processor 110 executes at least one step of the multi-tracer universal PET-MR image correction method.

[0055] The present application provides a universal multi-tracer PET-MR image correction method and storage medium, innovatively proposing the first universal multi-tracer PET-MR uptake correction framework. Feature structure terms containing anatomical reference information are added to guide the PET-MR correction process. Attenuation knowledge and anatomical reference information are transferred from CT to MR. A multi-scale pixel-level adaptive expert (MsPR) module is provided. Inter-tracer interference is reduced by allocating conflicting tasks through multi-scale paths. A Unet network is used to achieve cross-modal quantitative alignment of PET-MR and PET-CT. This solves the problem that existing PET reconstruction can only target a single tracer and cannot reduce the large quantitative differences between multi-tracer PET-MR images and PET-CT images, significantly improving the correction quality of PET-MR images.

[0056] The above describes in detail the present application's provision of a universal multi-tracer PET-MR image correction method and storage medium. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The descriptions of the above embodiments are intended only to facilitate understanding of the present application's method and core concepts. Furthermore, those skilled in the art will appreciate that variations in the specific implementation methods and scope of application may occur based on the concepts of the present application. In summary, the contents of this specification should not be construed as limiting the present application.

Claims

1. A universal multi-tracer PET-MR image correction method, characterized in that: The specific steps include: A first image acquisition step is to acquire a head CT image and a head MR image of the subject; A first image preprocessing step is to preprocess the head CT image and the head MR image; An encoder construction step, constructing a first encoder and a second encoder, training the first encoder using a pixel-level L2 loss function, and training the second encoder using a contrastive learning paradigm; A first image reconstruction step, inputting the preprocessed head CT image into a trained first encoder, extracting and encoding anatomical structure information in the preprocessed head CT image to obtain CT features; a second image reconstruction step, inputting the preprocessed head MR image into a trained second encoder, converting the preprocessed head MR image into MR features aligned with the CT features; a second image acquisition step of acquiring a paired PET-MR image and a PET-CT image of the multiple tracers based on the obtained aligned CT features and MR features; a second image preprocessing step of preprocessing the PET-MR image and the PET-CT image; A neural network construction step is to initialize a Unet network specifically for PET-MR image correction. The Unet network introduces a channel attention mechanism, selects different channel weights for different tracers, and trains the Unet network using a cross-attention mechanism based on the obtained MR features; as well as The image input step inputs the PET-MR image into the trained Unet network to obtain a corrected PET-MR image.

2. The multi-tracer universal PET-MR image correction method according to claim 1, wherein: The first image preprocessing step specifically includes the following steps: Image quality control steps, checking various image indicators and eliminating images with substandard quality; and The image normalization step adjusts the pixel values ​​of the qualified image to within a preset threshold range.

3. The multi-tracer universal PET-MR image correction method according to claim 1, wherein: The first encoder and the second encoder are autoencoders based on the Vision Transformer architecture.

4. The multi-tracer universal PET-MR image correction method according to claim 1, wherein: In the encoder construction step, the specific steps of training the second encoder include: a first encoder freezing step of freezing the first encoder; a feature alignment step, based on the obtained CT features, using a positive loss function to align the MR features with the patch at the same position in the CT features at a pixel level; and In the information overfitting offset step, the negative loss function is used to offset the semantic information overfitting caused by further alignment between patches that do not correspond to each other.

5. The multi-tracer universal PET-MR image correction method according to claim 1, wherein: The second image preprocessing step specifically includes the following steps: Image quality control steps, checking various image indicators and eliminating images with substandard quality; and The image processing step preprocesses the qualified images and normalizes the images.

6. The multi-tracer universal PET-MR image correction method according to claim 1, wherein: The Unet network includes A multi-scale pixel-level expert model can assign each pixel of the input PET-MR image to the most appropriate expert network for processing based on routing instructions at the original image scale.

7. The multi-tracer universal PET-MR image correction method according to claim 1, wherein: In the neural network construction step, the specific steps of training the Unet network include: a second encoder freezing step of freezing the second encoder; an output interaction step, based on the obtained MR features, using a cross-attention mechanism to embed the output of the second encoder into the bottom layer of the Unet network, so that the MR features output by the second encoder interact with the input features of the bottom layer of the Unet network, where the input features of the bottom layer of the Unet network are PET-MR images; Contribution calculation step: The contribution of the MR feature to the input feature at the bottom layer of the Unet network is determined by calculating the attention weight. The formula is: ； Among them, Q represents the input features of the Unet bottom layer, K represents the learnable projection of MR features, To prevent the gradient from vanishing, the scaling factor, Softmax represents the activation function, and Attention scores represent the contribution; and In the feature fusion step, the weighted MR features are fused with the input features of the bottom layer of the Unet network to obtain the corrected PET-MR image.

8. A storage medium storing computer-readable instructions, which, when read by at least one processor, causes the at least one processor to execute at least one step of the multi-tracer universal PET-MR image correction method according to any one of claims 1 to 7.

Images