A method and device for synthesizing medical images from MRI to PET

Abstract

The present invention discloses a method and apparatus for MRI-to-PET medical image synthesis. The method comprises: constructing a forward modal attribute decoupling mechanism: decoupling the corresponding appearance vectors and anatomical structure vectors from MRI and PET modal images, respectively, and synthesizing the PET modal image using the appearance vectors of the PET modal image and the anatomical structure vectors of the MRI modal image; constructing a backward modal attribute decoupling mechanism: using Gaussian vectors to learn the distribution of the appearance vectors of the PET modal image, and using the Gaussian vectors as the appearance vectors of the PET modal image during the testing phase to guide MRI-to-PET medical image synthesis; and designing a diagnosis-oriented modal attribute optimization strategy: by effectively aggregating complementary disease-related information between MRI and PET modal images, more accurate diagnostic information is predicted to guide the network to synthesize high-quality PET modal images. The apparatus comprises: a processor and a memory. The present invention fully exploits the modal attribute information contained in different imaging modalities to guide the network to synthesize high-quality PET modal images.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and medical image synthesis, and in particular to a method and apparatus for MRI-to-PET medical image synthesis. Background Art

[0002] With advancements in medical imaging technology, magnetic resonance imaging (MRI) and positron emission tomography (PET) have played crucial roles in the diagnosis of brain diseases. Considering that MRI modal images can capture structural information of the brain while PET modal images provide functional information, combining MRI and PET modal images can provide complementary multimodal information for the diagnosis of brain diseases such as Alzheimer's Disease (AD), thereby improving diagnostic accuracy. However, in clinical applications, the high radiation risk and cost of PET imaging, coupled with the frequent occurrence of missing PET modal images, limit the performance of combining MRI and PET modal images for AD diagnosis. Against this backdrop, researching an MRI-to-PET medical image synthesis method is of significant research importance, as it can assist physicians in more efficient AD diagnosis in clinical applications.

[0003] In recent years, thanks to the powerful feature extraction and learning capabilities of convolutional neural networks (CNNs), the synthesis of MRI-to-PET medical images using CNNs has gradually become a popular research direction. Hu et al. proposed a bidirectional mapping generative adversarial network (GAN), which uses PET modality information to guide the conversion of MRI modality images to PET modality images. To further improve the quality of synthesized PET modality images in disease-related regions, Pan et al. used a pre-trained disease diagnosis network to provide the synthesis network with prior disease knowledge, effectively assisting the synthesis network in synthesizing PET modality images containing more disease-related information. Although the above methods improve the quality of synthesized PET modality images, two problems still exist. First, MRI and PET modality images are obtained by different instruments from the same part of the subject, and they have anatomical structural consistency and appearance differences. Existing methods do not fully explore the modal attribute information contained in different imaging modalities to assist image synthesis, limiting the quality of synthesized PET modality images. In addition, although the above methods use diagnostic information to guide the synthesis network and improve the synthesis quality of disease-related regions, existing methods ignore the differences in the contribution of different modality images to the disease diagnosis task, and cannot provide accurate diagnostic information to the synthesis network, thus making it difficult to guide the network to synthesize high-quality PET modality images. Summary of the Invention

[0004] This invention provides a method and apparatus for synthesizing MRI-to-PET medical images. This invention comprehensively utilizes the consistency of anatomical structures and the differences in appearance between MRI and PET modal images to improve the quality of synthesized PET modal images. Furthermore, by effectively aggregating complementary disease-related information between MRI and PET modal images, this invention predicts more accurate diagnostic information to guide the network in synthesizing high-quality PET modal images, as detailed below:

[0005] A method for synthesizing MRI-to-PET medical images, the method comprising:

[0006] Construct a forward modal attribute decoupling mechanism: Decouple the corresponding appearance vector and anatomical structure vector from MRI and PET modal images respectively, and synthesize the PET modal image using the appearance vector of the PET modal image and the anatomical structure vector of the MRI modal image;

[0007] Constructing a backward modal attribute decoupling mechanism: using Gaussian vectors to learn the distribution of appearance vectors in PET modal images, and using Gaussian vectors as appearance vectors in PET modal images to guide the synthesis of MRI to PET medical images during the testing phase;

[0008] Design a diagnostic-oriented modal attribute optimization strategy: Utilize complementary disease-related information between MRI and PET modal images to predict diagnostic information to optimize the modal attributes of MRI and PET modal images, and then synthesize PET modal images.

[0009] Specifically, the decoupling of the corresponding appearance vector and anatomical structure vector from MRI and PET modal images involves:

[0010] MRI modal images were acquired using an appearance vector encoder and an anatomical structure vector encoder, respectively. MRI and PET modal images I PET The appearance vector and anatomical structure vector are as follows:

[0011]

[0012] in, and These represent anatomical structure vector encoders for MRI and PET modal images, respectively, consisting of three stacked convolutional layers. and These represent the appearance vector encoders for MRI and PET modal images, respectively, consisting of four stacked convolutional layers, two fully connected layers, and reparameterization operations. MRI and a PET s represents the anatomical structure vectors obtained after decoupling the MRI modality image and the PET modality image, respectively. MRI and sPET These represent the appearance vectors obtained after decoupling the MRI modal image and the PET modal image, respectively.

[0013] Specifically, the synthesis of a PET modality image using the appearance vector of the PET modality image and the anatomical structure vector of the MRI modality image involves:

[0014] a MRI and s PET G as a PET mode generator P Input to synthesize PET modal images s MRI and a PET As an MRI modality generator G M Input to synthesize MRI modal images as follows:

[0015]

[0016] Among them, GP and G M These represent generators for MRI and PET modal images, respectively, consisting of three stacked convolutional layers and an adaptive instance normalization layer.

[0017] a MRI and s MRI As an MRI modality generator G M Input to synthesize MRI modal images a PET and SP ET As input to the PET modality generator GP to synthesize PET modal images The formula is expressed as follows:

[0018]

[0019] Specifically, the method of learning the distribution of appearance vectors in PET modal images using Gaussian vectors, and using Gaussian vectors as appearance vectors in PET modal images to guide the synthesis of MRI to PET medical images during the testing phase, involves the following:

[0020] The anatomical structure vector a MRI And a randomly sampled Gaussian vector N(z) as the PET mode generator G P Input to predict PET modal images as follows:

[0021]

[0022] Synthesized PET modal images The appearance vector is obtained as input to the PET modal image appearance vector encoder GP.

[0023] Specifically, the method of utilizing complementary disease-related information between MRI and PET modal images to predict diagnostic information to optimize the modal properties of MRI and PET modal images, and then synthesizing PET modal images, involves:

[0024] The appearance vector s of the MRI modality image MRI and the appearance vectors of PET modal images PET Enhanced s were obtained by enhancing disease-related features through self-attention mechanisms. MRI and s PET Using mutual attention mechanism to enhance s MRI and s PET The complementary appearance vectors sa are obtained by fusion. ll The formula is expressed as follows:

[0025] s all = cross_att(self_att(s) MRI ), self_att(s PET ))

[0026] Here, self_att(·) represents the self-attention mechanism, and cross_att(·) represents the mutual attention mechanism, based on complementary appearance vectors s all and MRI anatomical structure vector a MRI The disease diagnostic features F are generated by using adaptive instance normalization, and the disease diagnostic features F are used as input to the disease classifier to achieve accurate AD diagnosis.

[0027] The method further includes: a training mechanism for training an MRI-to-PET medical image synthesis network;

[0028] The first stage trains the forward modal attribute decoupling mechanism and the backward modal attribute decoupling mechanism to synthesize PET modal images. The overall loss function for this stage is:

[0029]

[0030] Where α1, α2, α3, and α4 represent the weight coefficients of different loss functions;

[0031] The second stage trains a diagnostic-oriented modal attribute optimization strategy to improve the quality of synthesized PET modal images. The overall loss function for this stage is:

[0032] L second =β1L D

[0033] Where β1 represents the weight coefficient of the loss function;

[0034] The third stage trains the forward modal attribute decoupling mechanism and the backward modal attribute decoupling mechanism. The overall loss function for this stage is L. all =L first +L second .

[0035] The method further includes: using MRI modal images as anatomical structure vector encoders. The anatomical structure vectors of the MRI modal images are obtained from the input; the randomly sampled Gaussian vectors and the anatomical structure vectors of the MRI modal images are used as the PET modality generator G. P The input is used to synthesize the final PET modal image.

[0036] A second aspect is an MRI-to-PET medical image synthesis apparatus, the apparatus comprising: a processor and a memory, the memory storing program instructions, the processor calling the program instructions stored in the memory to cause the apparatus to perform the method described in any of the first aspects.

[0037] Third aspect, a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described in any one of the first aspects.

[0038] The beneficial effects of the technical solution provided by this invention are:

[0039] 1. This invention constructs a forward modal attribute decoupling mechanism, which improves the synthesis quality of PET modal images by mining consistent anatomical structure information and different appearance information between different modal images; in addition, a backward modal attribute decoupling mechanism is also constructed, which learns the appearance information of PET modal images by using Gaussian vectors during the training phase, so as to guide the synthesis of MRI to PET medical images during the testing phase.

[0040] 2. This invention designs a diagnosis-oriented modal attribute optimization strategy, which comprehensively utilizes the complementary disease-related information between MRI and PET modal images to predict more accurate diagnostic information, thereby optimizing the modal attributes of MRI and PET modal images and improving the quality of the synthesized PET modal images.

[0041] 3. This invention improves the performance of medical image synthesis by combining forward modal attribute decoupling mechanism, backward modal attribute decoupling mechanism and diagnosis-oriented modal attribute optimization strategy. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the workflow of an MRI-to-PET medical image synthesis method.

[0043] Figure 2 This is a quantitative comparison chart of our method with other advanced methods. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below.

[0045] Example 1

[0046] To overcome the shortcomings of existing technologies, embodiments of the present invention provide a method for synthesizing MRI-to-PET medical images, the method comprising:

[0047] I. Constructing a forward modal attribute decoupling mechanism

[0048] Considering that MRI and PET modal images are obtained from the same part of a subject using different instruments, they exhibit both anatomical structural consistency and appearance differences. Utilizing both the anatomical structural consistency and appearance differences between MRI and PET modal images helps in synthesizing high-quality PET modal images. This invention proposes a forward modal attribute decoupling mechanism. This mechanism decouples the corresponding appearance vectors and anatomical structure vectors from MRI and PET modal images, respectively, and then synthesizes the PET modal image by using the appearance vector from the PET modal image and the anatomical structure vector from the MRI modal image. Here, anatomical structure is a well-known technical term in the art, and the anatomical structure vector is the vectorized expression of anatomical structure information; this invention will not elaborate further on this aspect.

[0049] Specifically, MRI modal images (IMs) are first acquired using an appearance vector encoder and an anatomical structure vector encoder, respectively. R I and PET modal images, P ET The appearance vector and anatomical structure vector are expressed by the following formula:

[0050]

[0051] in, and These represent anatomical structure vector encoders for MRI and PET modal images, respectively, consisting of three stacked convolutional layers. and These represent the appearance vector encoders for MRI and PET modal images, respectively, consisting of four stacked convolutional layers, two fully connected layers, and reparameterization operations. MRI and a PET s represents the anatomical structure vectors obtained after decoupling the MRI modality image and the PET modality image, respectively. MRI and s PETThese represent the appearance vectors obtained after decoupling the MRI modal image and the PET modal image, respectively.

[0052] Furthermore, a MRI and s PET G as a PET mode generator P Input to synthesize PET modal images At the same time, s MRI and a PET As an MRI modality generator G M Input to synthesize MRI modal images The formula is expressed as follows:

[0053]

[0054] Among them, GP and G M These represent generators for MRI and PET modal images, respectively, consisting of three stacked convolutional layers and an adaptive instance normalization layer.

[0055] Furthermore, a MRI and s MRI As an MRI modality generator G M Input to synthesize MRI modal images At the same time, a PET and s PET G as a PET mode generator P Input to synthesize PET modal images The formula is expressed as follows:

[0056]

[0057] To constrain the synthesized PET modal images And real PET modal images I PET Consistency between them and the synthesized MRI modal images and real MRI modal images I MRI The consistency between them introduces reconstruction loss. and The formula is expressed as follows:

[0058]

[0059] Where E represents the expected value, and ||·||1 represents the mean absolute error.

[0060] Meanwhile, to ensure the consistency of anatomical structure vectors extracted from MRI modal images and PET modal images, an anatomical structure consistency loss L was constructed. a The difference between the anatomical structure vectors in constrained MRI modal images and PET modal images is expressed by the following formula:

[0061]

[0062] Among them, D a Represents an anatomical structure identification device.

[0063] Furthermore, embodiments of the present invention also introduce adversarial loss. and and L1 loss and Used to ensure the synthesis of MRI modal images and synthesized PET modal images The authenticity is expressed by the following formula:

[0064]

[0065] Among them, D P Represents PET modal image discriminator, D M This represents an MRI modality image discriminator.

[0066] II. Constructing a backward modal attribute decoupling mechanism

[0067] Considering that real PET modal images cannot be obtained during the testing phase PET This results in the inability to obtain the appearance vectors of the true PET modal image. PET To achieve forward modal attribute decoupling, a backward modal attribute decoupling mechanism was constructed, which utilizes Gaussian vectors to learn the PET modal image I... PET The distribution of appearance vectors was used to guide the synthesis of MRI-to-PET medical images during the testing phase by utilizing Gaussian vectors as appearance vectors for PET modal images.

[0068] Specifically, the anatomical structure vector a of MRI modal images is obtained by utilizing the forward modal attribute decoupling mechanism. MRI Then, a MRI And a randomly sampled Gaussian vector N(z) as the PET mode generator G P Input to predict PET modal images The formula is expressed as follows:

[0069]

[0070] Furthermore, in order to ensure the quality of the synthesized PET modal images... The authenticity of the synthesized PET modal images The appearance vector is obtained as input to the PET modal image appearance vector encoder GP. Introducing the latent vector loss L latent Constrained appearance vector The consistency with the Gaussian vector N(z) is expressed by the following formula:

[0071]

[0072] To constrain the appearance vectors of PET modal images to follow a Gaussian distribution prior, a KL divergence constraint function L is introduced. KL Constrained Gaussian vector N(z) and appearance vector s of PET modal image PET Consistency of distribution. The formula is as follows:

[0073] L KL =E[D KL (N(z)||s PET )]

[0074] Among them, D KL This represents the KL divergence.

[0075] Finally, adversarial losses were introduced. and L1 loss function Ensure the synthesis of PET modal images And real PET modal images, P ET The consistency between them is expressed by the following formula:

[0076]

[0077] III. Designing Diagnostic-Oriented Modal Attribute Optimization Strategies

[0078] Considering that incorporating disease-related information during the synthesis process can help improve the synthesis quality of disease-related regions in PET modal images, this invention proposes a diagnosis-oriented modal attribute optimization strategy. This strategy comprehensively utilizes complementary disease-related information between MRI and PET modal images to predict more accurate diagnostic information, thereby optimizing the modal attributes of MRI and PET modal images and improving the quality of the synthesized PET modal images.

[0079] Specifically, firstly, the appearance vector s of the MRI modality image is... MRI and the appearance vectors of PET modal images PET Enhanced s were obtained by enhancing disease-related features through self-attention mechanisms. MRI and s PET Subsequently, the enhanced s was used with a mutual attention mechanism. MRI and s PET The complementary appearance vectors s are obtained by fusion. all The formula is expressed as follows:

[0080] s all = cross_att(self_att(s) MRI), self_att(s PET ))

[0081] Here, self_att(·) represents the self-attention mechanism, and cross_att(·) represents the cross-attention mechanism.

[0082] Based on complementary appearance vectors s all and MRI anatomical structure vector a MRI The disease diagnostic features F are generated using adaptive instance normalization. These features F are then used as input to a disease classifier to achieve accurate AD diagnosis.

[0083] Finally, cross-entropy loss L is introduced. D The training process of the diagnostic-oriented modal attribute optimization strategy is constrained to optimize the modal attributes of MRI and PET modal images. The formula is expressed as follows:

[0084]

[0085] Where N represents the total number of subjects in the training set, F i Let D(·) represent the disease diagnostic features of the i-th subject, and let D(F) represent the disease classifier. i ) represents the predicted diagnostic result for the i-th subject, y i This represents the true diagnostic result of the i-th subject.

[0086] IV. Training an MRI-to-PET medical image synthesis network

[0087] The training of this network in this embodiment of the invention is divided into three stages. The first stage trains the forward modal attribute decoupling mechanism and the backward modal attribute decoupling mechanism to synthesize PET modal images. The overall loss function for this stage is:

[0088]

[0089] Wherein, α1, α2, α3, and α4 represent the weight coefficients of different loss functions. In this invention, α1 is set to 8, α2 to 1, α3 to 0.1, and α4 to 0.01.

[0090] The second stage trains a diagnostic-oriented modal attribute optimization strategy to improve the quality of synthesized PET modal images. The overall loss function for this stage is:

[0091] L second =β1L D

[0092] Wherein, β1 represents the weight coefficient of the loss function. In this invention, β1 is set to 1.

[0093] The third stage trains the forward modal attribute decoupling mechanism and the backward modal attribute decoupling mechanism. The overall loss function for this stage is L. all =L first +L second Ultimately, high-quality PET modal images can be synthesized.

[0094] V. Testing the MRI-to-PET Medical Image Synthesis Network

[0095] During testing, MRI modal images from the test set and randomly sampled Gaussian vectors were used as inputs to a backward modal attribute decoupling mechanism to synthesize PET modal images. Specifically, the MRI modal images were first used as anatomical structure vector encoders. The anatomical structure vectors of the MRI modal images are obtained from the input. Then, the randomly sampled Gaussian vectors and the anatomical structure vectors of the MRI modal images are used as the PET modality generator G. P The input is used to synthesize the final PET modal image.

[0096] Example 2

[0097] An MRI-to-PET medical image synthesis apparatus includes a processor and a memory, the memory storing program instructions, the processor calling the program instructions stored in the memory to cause the apparatus to perform the following method steps in Embodiment 1:

[0098] Construct a forward modal attribute decoupling mechanism: Decouple the corresponding appearance vector and anatomical structure vector from MRI and PET modal images respectively, and synthesize the PET modal image using the appearance vector of the PET modal image and the anatomical structure vector of the MRI modal image;

[0099] Constructing a backward modal attribute decoupling mechanism: using Gaussian vectors to learn the distribution of appearance vectors in PET modal images, and using Gaussian vectors as appearance vectors in PET modal images to guide the synthesis of MRI to PET medical images during the testing phase;

[0100] Design a diagnostic-oriented modal attribute optimization strategy: Utilize complementary disease-related information between MRI and PET modal images to predict diagnostic information to optimize the modal attributes of MRI and PET modal images, and then synthesize PET modal images.

[0101] Specifically, the decoupling of the corresponding appearance vector and anatomical structure vector from MRI and PET modal images is as follows:

[0102] MRI modal images (IM) were acquired using an appearance vector encoder and an anatomical structure vector encoder, respectively. RI And PET modal images, P ET The appearance vector and anatomical structure vector are as follows:

[0103]

[0104] in, and These represent anatomical structure vector encoders for MRI and PET modal images, respectively, consisting of three stacked convolutional layers. and These represent the appearance vector encoders for MRI and PET modal images, respectively, consisting of four stacked convolutional layers, two fully connected layers, and reparameterization operations. MRI and a PET s represents the anatomical structure vectors obtained after decoupling the MRI modality image and the PET modality image, respectively. MRI and s PET These represent the appearance vectors obtained after decoupling the MRI modal image and the PET modal image, respectively.

[0105] Specifically, the synthesis of PET modal images using the appearance vectors of PET modal images and the anatomical structure vectors of MRI modal images involves:

[0106] a MRI and s PET As input to the PET modality generator GP to synthesize PET modal images s MRI and a PET As an MRI modality generator G M Input to synthesize MRI modal images as follows:

[0107]

[0108] Among them, GP and G M These represent generators for MRI and PET modal images, respectively, consisting of three stacked convolutional layers and an adaptive instance normalization layer.

[0109] a MRI and s MRI As an MRI modality generator G M Input to synthesize MRI modal images a PET and s PET As input to the PET modality generator GP to synthesize PET modal images The formula is expressed as follows:

[0110]

[0111] Specifically, the distribution of appearance vectors in PET modal images is learned using Gaussian vectors. During the testing phase, Gaussian vectors are used as appearance vectors in PET modal images to guide the synthesis of MRI to PET medical images.

[0112] The anatomical structure vector a MRI And a randomly sampled Gaussian vector N(z) as the PET mode generator G P Input to predict PET modal images as follows:

[0113]

[0114] Synthesized PET modal images G as a PET modal image appearance vector encoder P The input yields the appearance vector.

[0115] Specifically, by utilizing complementary disease-related information between MRI and PET modal images to predict diagnostic information and optimize the modal properties of MRI and PET images, and then synthesizing PET modal images, the following steps are taken:

[0116] The appearance vector s of the MRI modality image MRI and the appearance vectors of PET modal images PET Enhanced s were obtained by enhancing disease-related features through self-attention mechanisms. MRI and s PET Using mutual attention mechanism to enhance s MRI and s PET The complementary appearance vectors s are obtained by fusion. all The formula is expressed as follows:

[0117] s all = cross_att(self_att(s) MRI ), self_att(s PET ))

[0118] Here, self_att(·) represents the self-attention mechanism, and cross_att(·) represents the mutual attention mechanism, based on complementary appearance vectors s all and MRI anatomical structure vector a MRI The disease diagnostic features F are generated by using adaptive instance normalization, and the disease diagnostic features F are used as input to the disease classifier to achieve accurate AD diagnosis.

[0119] The device also includes a training mechanism for training an MRI-to-PET medical image synthesis network.

[0120] The first stage trains the forward modal attribute decoupling mechanism and the backward modal attribute decoupling mechanism to synthesize PET modal images. The overall loss function for this stage is:

[0121]

[0122] Where α1, α2, α3, and α4 represent the weight coefficients of different loss functions;

[0123] The second stage trains a diagnostic-oriented modal attribute optimization strategy to improve the quality of synthesized PET modal images. The overall loss function for this stage is:

[0124] L second =β1L D

[0125] Where β1 represents the weight coefficient of the loss function;

[0126] The third stage trains the forward modal attribute decoupling mechanism and the backward modal attribute decoupling mechanism. The overall loss function for this stage is L. all =L first +L second .

[0127] The device also includes: an MRI modality image as an anatomical structure vector encoder. The anatomical structure vectors of the MRI modal images are obtained from the input; the randomly sampled Gaussian vectors and the anatomical structure vectors of the MRI modal images are used as the PET modality generator G. P The input is used to synthesize the final PET modal image.

[0128] It should be noted that the device descriptions in the above embodiments correspond to the method descriptions in the embodiments, and the embodiments of the present invention will not be repeated here.

[0129] The execution entities of the aforementioned processor and memory can be devices with computing functions such as computers, microcontrollers, and single-chip microcomputers. In specific implementations, the embodiments of the present invention do not limit the execution entities and can select them according to the needs of actual applications.

[0130] Data signals are transmitted between the memory and the processor via a bus, which will not be elaborated upon in this embodiment of the invention.

[0131] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium, the storage medium including a stored program, which, when the program is running, controls the device where the storage medium is located to execute the method steps in the above embodiments.

[0132] The computer-readable storage medium includes, but is not limited to, flash memory, hard disk, solid-state drive, etc.

[0133] It should be noted that the description of the readable storage medium in the above embodiments corresponds to the description of the method in the embodiments, and the embodiments of the present invention will not be repeated here.

[0134] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated.

[0135] A computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in or transmitted through a computer-readable storage medium. A computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic or semiconductor, etc.

[0136] Unless otherwise specified, the model numbers of the various devices in this embodiment of the invention are not limited, and any device that can perform the above functions is acceptable.

[0137] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0138] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for synthesizing MRI-to-PET medical images, characterized in that, The method includes: Construct a forward modal attribute decoupling mechanism: Decouple the corresponding appearance vector and anatomical structure vector from MRI and PET modal images respectively, and synthesize the PET modal image using the appearance vector of the PET modal image and the anatomical structure vector of the MRI modal image; Constructing a backward modal attribute decoupling mechanism: using Gaussian vectors to learn the distribution of appearance vectors in PET modal images, and using Gaussian vectors as appearance vectors in PET modal images to guide the synthesis of MRI to PET medical images during the testing phase; Design a diagnostic-oriented modal attribute optimization strategy: Utilize complementary disease-related information between MRI and PET modal images to predict diagnostic information to optimize the modal attributes of MRI and PET modal images, and then synthesize PET modal images; Specifically, the decoupling of the corresponding appearance vector and anatomical structure vector from MRI and PET modal images involves: MRI modal images were acquired using an appearance vector encoder and an anatomical structure vector encoder, respectively. MRI and PET modal images I PET The appearance vector and anatomical structure vector are as follows: in, and These represent anatomical structure vector encoders for MRI and PET modal images, respectively, consisting of three stacked convolutional layers. and These represent the appearance vector encoders for MRI and PET modal images, respectively, consisting of four stacked convolutional layers, two fully connected layers, and reparameterization operations. MRI and a PET s represents the anatomical structure vectors obtained after decoupling the MRI modality image and the PET modality image, respectively. MRI and s PET These represent the appearance vectors obtained after decoupling the MRI modal image and the PET modal image, respectively; Specifically, the synthesis of a PET modality image using the appearance vector of the PET modality image and the anatomical structure vector of the MRI modality image involves: a MRI and s PET G as a PET mode generator P Input to synthesize PET modal images s MRI and a PET As an MRI modality generator G M Input to synthesize MRI modal images as follows: Among them, G P and G M These represent generators for MRI and PET modal images, respectively, consisting of three stacked convolutional layers and an adaptive instance normalization layer. a MRI and s MRI As an MRI modality generator G M Input to synthesize MRI modal images a PET and s PRT G as a PET mode generator P Input to synthesize PET modal images The formula is expressed as follows: Specifically, the method of utilizing complementary disease-related information between MRI and PET modal images to predict diagnostic information to optimize the modal properties of MRI and PET modal images, and then synthesizing PET modal images, involves: The appearance vector s of the MRI modality image MRI and the appearance vectors of PET modal images PET Enhanced s were obtained by enhancing disease-related features through self-attention mechanisms. MRI and s PET Using mutual attention mechanism to enhance s MRI and s PET The complementary appearance vectors s are obtained by fusion. all The formula is expressed as follows: s all =cross_att(self_att(s MRI ),self_att(s PET )) Here, self_att(·) represents the self-attention mechanism, and cross_att(·) represents the mutual attention mechanism, based on complementary appearance vectors s all and MRI anatomical structure vector a MRI The disease diagnostic features F are generated by using adaptive instance normalization, and the disease diagnostic features F are used as input to the disease classifier to achieve accurate AD diagnosis.

2. The method for synthesizing MRI-to-PET medical images according to claim 1, characterized in that, The method of learning the distribution of appearance vectors in PET modal images using Gaussian vectors, and using Gaussian vectors as appearance vectors in PET modal images to guide the synthesis of MRI to PET medical images during the testing phase, specifically involves: The anatomical structure vector a MRI And a randomly sampled Gaussian vector N(z) as the PET mode generator G P Input to predict PET modal images as follows: Synthesized PET modal images G as a PET modal image appearance vector encoder P The input yields the appearance vector.

3. The method for synthesizing MRI-to-PET medical images according to claim 1, characterized in that, The method further includes: a training mechanism for training an MRI-to-PET medical image synthesis network; The first stage trains the forward modal attribute decoupling mechanism and the backward modal attribute decoupling mechanism to synthesize PET modal images. The overall loss function for this stage is: Where α1, α2, α3, and α4 represent the weight coefficients of different loss functions; The second stage trains a diagnostic-oriented modal attribute optimization strategy to improve the quality of synthesized PET modal images. The overall loss function for this stage is: L second <β1L D Where β1 represents the weight coefficient of the loss function; The third stage trains the forward modal attribute decoupling mechanism and the backward modal attribute decoupling mechanism. The overall loss function for this stage is L. all =L first +L second .

4. The method for synthesizing MRI-to-PET medical images according to claim 1, characterized in that, The method further includes: using MRI modal images as anatomical structure vector encoders. The anatomical structure vectors of the MRI modal images are obtained from the input; the randomly sampled Gaussian vectors and the anatomical structure vectors of the MRI modal images are used as the PET modality generator G. P The input is used to synthesize the final PET modal image.

5. An MRI-to-PET medical image synthesis device, characterized in that, The device includes a processor and a memory, the memory storing program instructions, the processor invoking the program instructions stored in the memory to cause the device to perform the method according to any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described in any one of claims 1-4.

Images