Generative adversarial network-based MRI-PET mode conversion method and system

Abstract

The invention discloses an MRI-PET mode conversion method and system based on a generative adversarial network, and belongs to the technical field of artificial intelligence medical image generation. And the multi-scale structure representation injection module injects multi-scale anatomical prior information at different stages of the encoder, and overcomes the limitations of insufficient utilization of prior information and single injection scale. And the adaptive semantic residual fusion module adopts semantic attention guidance and double-branch attention weighting, adaptively fuses fine-grained local features and global context information, harmonizes the difference between the fine-grained local features and the global context information in an abstract level and a semantic category, and solves the problems of feature conflict and semantic fuzziness in a bottleneck region. The direction sensing space-frequency discriminator realizes multi-dimensional and fine-grained adversarial supervision through a space, frequency and local image block multi-branch collaborative discrimination mechanism, and improves the structural fidelity and spectrum authenticity of a synthetic image. And the generated image is superior to the existing method in indexes such as structural similarity and peak signal-to-noise ratio, and has higher clinical practical value.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence medical image generation technology, and in particular relates to an MRI-PET modality conversion method and system based on generative adversarial networks. Background Technology

[0002] Magnetic Resonance Imaging (MRI) is a medical imaging technique that uses signals generated by the resonance of atomic nuclei in a strong magnetic field to form images. Different tissues exhibit different signal intensities in the imaging, clearly displaying structures such as soft tissues, bones, and joints. It has advantages such as high resolution and no ionizing radiation, and mainly reflects the structural information of the detected area.

[0003] Positron emission tomography (PET) is a nuclear medicine imaging technique that uses the decay signals of radioactive isotope-labeled tracers injected into the human body to create images. The distribution of radioactive isotopes in the body can indirectly show the functional status of different tissues, mainly reflecting the functional information of the detected site.

[0004] With advancements in medical technology, clinical diagnostic methods for various diseases have become increasingly diverse. Imaging techniques are often a strong indicator of diagnostic results. Furthermore, the combination of multiple imaging methods allows for the complementarity of information from different modalities, which can improve the confidence level of disease diagnoses to a certain extent. Multimodal medical imaging, particularly the combination of magnetic resonance imaging (MRI) and positron emission tomography (PET), provides crucial complementary information on anatomical structure and functional metabolism for the clinical assessment and diagnosis of neurological diseases such as Alzheimer's disease (AD). However, PET imaging is limited in its clinical application due to its high cost, long acquisition time, and biosafety risks associated with radioactive tracers. Therefore, cross-modal medical image synthesis technology has emerged. Its core objective is to synthesize clinically reliable PET images from widely available, non-invasive MRI data to replace or supplement actual PET scans.

[0005] With the development of deep learning technology and its application in the medical field, a series of image generation methods have emerged, such as Generative Adversarial Networks (GANs), Variational Autoencoders, and Diffusion Models, providing effective solutions for the synthesis of positron emission tomography (PET) images. Existing deep learning-based MRI-to-PET generation methods can be mainly classified into the following categories:

[0006] The first category is based on convolutional neural networks (CNNs). These methods leverage the powerful local feature extraction capabilities of CNNs to learn the mapping relationship from MRI to PET. For example, Li et al. proposed a CNN-based volume data mapping framework; Xiang et al. developed a deep context model that iteratively optimizes low-dose positron emission tomography (PET) images using multi-level MRI context information; Zhou et al. proposed the Hi-Net dual-path hybrid fusion architecture, separating modality-specific and shared representations in the encoder-decoder framework; Hu et al. used a 3D Dense-UNet backbone network to improve the anatomical structural realism of synthesized images and assist downstream diagnostic tasks; Zhang et al.'s BPGAN introduced multi-scale convolutional U-Net and gradient contour loss to enhance structural fidelity; and Tu et al.'s CMPGAN improved the structural accuracy of FDG-PET synthesis by learning the constraint generation process through manifold projection and distribution distance. These methods validate the potential of CNNs in capturing fine-grained spatial features for cross-modal synthesis.

[0007] The second category is based on Transformer architecture. To overcome the limitations of CNNs, such as limited receptive fields and difficulty in modeling long-range dependencies, researchers have begun to introduce Transformers with self-attention mechanisms into medical image synthesis. Li et al. integrated Transformer modules into U-Net to enhance multimodal MRI synthesis; Dalmaz et al.'s ResViT combines residual CNN units with visual Transformers to jointly encode detailed textures and high-level semantic features; Yan et al.'s MMTrans utilizes a Swing Transformer generator combined with a deformable registration module to support unpaired cross-modal synthesis.

[0008] The third category involves hybrid models and auxiliary supervision strategies. To maintain both local anatomical and global semantic consistency, Lyu et al. and Jin et al., respectively, proposed DBGAN and HMS-MambaGAN, attempting to integrate sparse / dense attention mechanisms with convolutional backbone networks. Furthermore, some studies focus on leveraging domain prior knowledge and multi-domain supervision to improve synthesis quality. For example, Yu et al.'s edge-aware GAN embeds edge maps into the generator and discriminator to sharpen anatomical boundaries; Zeng et al. employed a hybrid framework combining a 3D generator and a 2D discriminator to enhance continuity between volumetric data slices; Yang et al. designed a fusion network to integrate Fourier-transformed PET and MRI data to maintain spectral consistency; Kang et al. maintained anatomical consistency under domain shifts by decoupling structural and texture components; and Wang et al.'s CycleSGAN introduced semantic feedback to reduce anatomical inconsistencies in unpaired image synthesis.

[0009] Current research largely focuses on cross-modal generation of 2D medical images, but clinical imaging is predominantly 3D. While it's possible to slice a 3D image, process each 2D slice separately, and then overlay them to form a 3D target modal image, this method loses the relationships between adjacent slices. Furthermore, existing 3D cross-modal medical image generation methods often suffer from significant shortcomings in terms of structural similarity and sharpness with the target image. The specific drawbacks and their causes are as follows:

[0010] 1. Insufficient and refined utilization of prior information. Many methods attempt to improve synthesis quality by introducing anatomical priors (such as edges and structures) or multi-domain supervision (such as the frequency domain). However, the main strategies are to inject prior information into the deep layers of the network or at a single scale, or to perform only image-level spectral constraints, failing to achieve multi-scale, hierarchical prior information injection during the coding stage. MRI images contain anatomical structural information that manifests differently at different resolutions. Existing methods fail to fully utilize these multi-scale structural guides, resulting in insufficient alignment of fine structures in diagnostically sensitive areas and a lack of reliability in the details of the synthesized images.

[0011] 2. The feature extraction capabilities of CNN backbone networks have fundamental limitations. Existing methods widely employ CNN encoders (such as U-Net) that rely on local convolutional operations. Their inherent local receptive fields limit the model's ability to capture global semantic interactions and long-range dependencies. However, the mapping between human brain metabolic function patterns and anatomical structures often involves complex contextual relationships across the entire brain. Relying solely on local features is insufficient to fully model this global mapping, leading to semantic inconsistencies or distortions in the overall metabolic patterns of synthesized PET images. To compensate for the shortcomings of CNNs, existing hybrid models attempt to introduce Transformer modules to capture global context. However, traditional Transformer-CNN feature fusion strategies are simple and suffer from semantic gaps. These methods typically fuse local features extracted by CNNs and global representations encoded by Transformers through simple concatenation or addition. Since local convolutional features and global self-attention features differ significantly in abstraction level and semantic scope, simple and direct fusion methods cannot effectively reconcile the differences in representation between the two, resulting in insufficient information fusion or even introducing conflicts, thus producing semantic ambiguity or structural anomalies in bottleneck regions of the synthesized image.

[0012] 3. The discriminator's supervisory signal is not fine-grained or comprehensive enough. Existing GAN frameworks often limit discriminators to real / fake image discrimination in the image spatial domain, or perform only simple multi-scale discrimination. This approach cannot effectively constrain the frequency domain characteristics of the synthesized image (such as directional frequency components), nor can it provide fine-grained adversarial feedback at the local image patch level. This results in generator-generated images that may appear visually clear overall, but lack the necessary texture details, or differ in frequency spectrum from real positron emission tomography (PET) images, reducing the clinical applicability of the synthesized images. Summary of the Invention

[0013] To address the shortcomings of existing technologies, this invention provides an MRI-PET modality conversion method and system based on generative adversarial networks. It aims to solve the problems of insufficient and imprecise utilization of anatomical prior information in existing methods, achieving precise structural alignment of diagnostically sensitive areas; it also addresses the global semantic distortion caused by local receptive field limitations and simple feature fusion strategies in existing generator architectures, enabling efficient interaction between global and local features at a deeper network level; and it overcomes the issues of single, coarse-grained supervisory signals in existing discriminators, comprehensively improving the texture details and spectral realism of synthesized images.

[0014] The technical solution of the present invention is as follows:

[0015] On the one hand, the present invention provides an MRI-PET modality conversion method based on generative adversarial networks, comprising the following specific steps:

[0016] A modality conversion dataset is obtained and divided into a training set, a validation set, and a test set according to a set ratio. The modality conversion dataset includes several samples, each of which contains an MRI image and a corresponding real positron emission tomography (PET) image of a subject.

[0017] A generative adversarial network was constructed to convert input magnetic resonance images (MRI) into positron emission tomography (PET) images.

[0018] The generative adversarial network includes a generator, a discriminator, and a reconstruction network;

[0019] The generator is used to synthesize high-fidelity positron emission tomography (PET) images from input magnetic resonance (MRI) images.

[0020] The reconstruction network encodes the input real positron emission tomography (PET) image using an encoder to obtain a feature map. Based on this feature map, the image is reconstructed to obtain a reconstructed PET image. The encoder parameters are adjusted according to the difference between the reconstructed image and the real PET image.

[0021] The discriminator shares an encoder with the reconstruction network. It determines whether the input positron emission tomography (PET) image is generated or real based on the feature map output by the encoder, and adjusts the generator parameters accordingly.

[0022] The generative adversarial network is trained using the training set and the validation set to obtain the trained generative adversarial network;

[0023] The system acquires MRI images to be converted and uses a generator in a trained generative adversarial network to generate PET images.

[0024] Furthermore, the discriminator simultaneously processes triples. ,in This is a true positron emission tomography (PET) image. For the generated positron emission tomography (PET) images, To reconstruct positron emission tomography (PET) images, Treated as a real sample and All are considered as generated samples.

[0025] Furthermore, the generator includes a generation encoder, a first bottleneck module, a Transformer module, an adaptive semantic residual fusion module, and a generation decoder;

[0026] The generative encoder is used to extract multi-level features from the input magnetic resonance image (MRI) to obtain low-level features, intermediate features, high-level features and ultimate features, wherein the intermediate features, high-level features and ultimate features are used as inputs to the generative decoder.

[0027] The first bottleneck module is used to capture local structural details from the ultimate features to obtain local features;

[0028] The final features extracted by the generator encoder are projected onto a 512-channel space through a 3×3×3 convolutional layer as the input to the Transformer module. The module then undergoes serialization, positional encoding embedding, and multi-layer self-attention transformation to obtain a global representation. The 3×3×3 convolutional layer consists of a 3×3×3 convolution with a stride of 2, batch normalization, and a ReLU activation function.

[0029] The adaptive semantic residual fusion module is used to adaptively fuse the local features obtained by the first bottleneck module and the global representation obtained by the Transformer module to obtain deep global-local fusion features.

[0030] The generator decoder and generator encoder are connected by a skip connection, which is used to gradually restore the spatial resolution of the deep global-local fusion features obtained by the adaptive semantic residual fusion module, the intermediate features, high-level features and ultimate features output by the generator encoder, so as to obtain the generated positron emission tomography (PET) image.

[0031] Furthermore, the generative encoder first extracts features from the input MRI image using a 7×7×7 convolutional layer, and simultaneously projects these features into a higher-dimensional feature space to obtain low-level features. To mitigate the information attenuation common in pooling operations, a convolution-based double downsampling strategy is adopted. Each downsampling pass through a 5×5×5 convolutional layer and a 3×3×3 convolutional layer sequentially. The features obtained after the first downsampling of the low-level features are input together with the low-level features into the first multi-scale structural representation injection module for multi-scale structural feature extraction and... The intermediate features are obtained by fusion; after a second downsampling, the intermediate features are input together with the low-level features into the second multi-scale structural representation injection module for cross-resolution structural prior information injection and feature enhancement to obtain high-level features; finally, the high-level features are passed through a 5×5×5 convolutional layer to obtain the final features; the 5×5×5 convolutional layer includes a 5×5×5 convolution with a stride of 1, batch normalization, and ReLU activation function; the 7×7×7 convolutional layer includes a 7×7×7 convolution, batch normalization, and ReLU activation function.

[0032] The multi-scale structure representation injection module first inputs low-level features into a multi-scale structure convolutional block to obtain the multi-scale structure representation. Multi-scale structural characterization After downsampling to adjust the resolution, the features are concatenated with those obtained after the first or second downsampling by the generative encoder, and then fused using 1×1×1 convolution and GELU activation. To enhance spatial discriminability, the efficient local attention mechanism is extended to a 3D form, and the fused features are used as the input tensor. The input is fed into an efficient local attention mechanism to obtain attention maps in depth, height, and width dimensions, respectively. These maps are then multiplied with the input tensor to enhance spatial encoding for orientation awareness, resulting in intermediate or high-level features. Indicates the number of channels. Represents the number of depth dimensions. Indicates the height dimension. Indicates the width dimension.

[0033] In the multi-scale structural convolution, low-level features are first processed through a 1×1×1 convolution. The convolution result is then input into three parallel branches: a depthwise separable branch, an asymmetric convolution branch, and a dilated convolution branch. The depthwise separable branch uses a 3×3×3 depthwise separable convolution, group normalization, and ReLU activation function to efficiently extract fine-grained local textures. The asymmetric convolution branch applies cascaded asymmetric convolutions (1,3,5) and (5,3,1), group normalization, and ReLU activation function to increase sensitivity to orientation-specific anatomical patterns. The dilated convolution branch uses a 5×5×5 dilated convolution with a dilation rate of 2, group normalization, and ReLU activation function to expand the receptive field to encode mid-range spatial dependencies. The outputs of the three branches are element-wise summed and then processed through a multilayer perceptron. The original low-level features are preserved through residual connections to obtain the multi-scale structural representation. ;

[0034] In efficient local attention mechanisms, first along each dimension Global average pooling is applied to the two orthogonal axes to produce a shape of Compact representation, Representing dimension, Represents depth dimension, Indicates height dimension, Indicates the width dimension. The length of the compact representation is then indicated; subsequently, the three-dimensional compact representations are each passed through a one-dimensional convolution with a kernel size of 5, followed by group normalization and sigmoid activation, resulting in three attention maps; the resulting three attention maps are then reshaped to... , and Dimensions are used to obtain the final attention map with depth, height, and width dimensions.

[0035] Furthermore, the first bottleneck module uses several 3D Res2Net blocks to perform multi-scale feature extraction in a parametrically efficient manner, thereby enhancing the modeling of local anatomical structures.

[0036] Specifically, the feature maps input to each 3D Res2Net block are first processed by a 1×1×1 convolution and then evenly divided into four feature subsets. Then, the four feature subsets are aggregated according to the hierarchical residual method to obtain four sets of output features. Four sets of output features The data is concatenated along the channel dimension and fused through a 1×1×1 convolution to form the final output, i.e., local features;

[0037] The aggregation of the hierarchical residuals is represented as follows:

[0038] (2);

[0039] in, Indicates the kernel size as 3D convolution operation;

[0040] The Transformer module projects the final features through a 3×3×3 convolutional layer onto a 512-channel space, reshaping the result into a voxel sequence. To preserve spatial correspondence, learnable positional encodings are added to the voxel sequence. The voxel sequence with added positional encodings is then passed through four cascaded Transformer blocks to obtain the output sequence. Finally, the output sequence is reshaped back into a 3D structure and upsampled to align with the local features obtained by the first bottleneck module. The upsampling operation includes channel expansion and 3D pixel reconstruction. The Transformer block includes an 8-head self-attention module and a feedforward network, both embedded in residual connections and layer normalization.

[0041] Furthermore, the adaptive semantic residual fusion module adaptively fuses local features extracted by Res2Net. Global representation encoded with the Transformer module :

[0042] First, from the global representation Exporting a global semantic attention graph as global guidance:

[0043] (3);

[0044] in, Represents the global semantic attention graph. This indicates a convolutional layer with a 1×1×1 kernel. This indicates a convolutional layer with a 3×3×3 kernel. Denotes the Gaussian error linear unit activation function;

[0045] The local features are then recalibrated using a residual mechanism:

[0046] (4);

[0047] Where ⊙ denotes successive multiplication. This represents the local features after recalibration;

[0048] Recalibrated local features With global features The data is spliced ​​together to form a unified representation, and then further refined through convolution:

[0049] (5);

[0050] in, Represents a unified global-local representation. This indicates a splicing operation along the channel dimension;

[0051] To generate adaptive fusion weights that control the contributions of each feature branch, a unified global-local representation is required. A two-branch attention mechanism is applied, where the channel attention path is obtained in the following way:

[0052] (6);

[0053] in, This represents the channel attention map. This represents a multilayer perceptron. This indicates a global average pooling operation;

[0054] The spatial attention path first performs max pooling and average pooling along the channel dimension, and then processes the max pooling results... and After splicing and merging:

[0055] (7);

[0056] in, Represents a spatial attention map;

[0057] The outputs of the two attention branches are combined to generate a unified gating signal:

[0058] (8);

[0059] in, Indicates a gating signal;

[0060] Finally, the attention-guided fusion of global representation and local features is expressed as follows:

[0061] (9);

[0062] in, It is a learnable scalar that controls the fusion ratio. To achieve deep global-local fusion features that simultaneously preserve long-range semantic coherence and detailed anatomical details;

[0063] Furthermore, the generator decoder includes three feature fusion decoding blocks equipped with skip connections. An upsampling operation is performed between every two feature fusion decoding blocks. Specifically, the input to the first feature fusion decoding block is the ultimate feature and the deep global-local fusion feature; the input to the second feature fusion decoding block is the result of the upsampling operation on the output of the first feature fusion decoding block and the high-level feature; the input to the third feature fusion decoding block is the result of the upsampling operation on the output of the second feature fusion decoding block and the mid-level feature; finally, the output of the third feature fusion decoding block is passed through a 7×7×7 convolutional layer to obtain the generated positron emission tomography (PET) image.

[0064] The process of a feature fusion decoding block and upsampling operation can be represented as follows:

[0065] (10);

[0066] in, This represents the decoder features that have been fused and enhanced after skip connections. This represents a 5×5×5 convolutional layer. This indicates a squeeze-excitation channel attention module used for adaptive recalibration of the channel response; This represents encoder features, which can be categorized as final features, high-level features, or intermediate features. This represents the decoder features, which can be deep global-local fusion features, the output of the first feature fusion decoding block after upsampling, or the output of the second feature fusion decoding block after upsampling. Indicates splicing.

[0067] Furthermore, the reconstruction network employs an encoder-decoder architecture to reproduce real positron emission tomography (PET) images. The input is a reconstructed positron emission tomography (PET) image. As a semantic reference for modality perception discrimination; the encoder first uses a 7×7×7 convolutional layer and a 5×5×5 convolutional layer to obtain shallow features. Then, through a series of 3×3×3 convolutional layers, 5×5×5 convolutional layers, 3×3×3 convolutional layers, 5×5×5 convolutional layers, and a second bottleneck module, the high-level semantic representation is obtained. The decoder uses high-level semantic representation. As input, the image is passed through three consecutive 5×5×5 convolutional layers. Upsampling is performed between every two 5×5×5 convolutional layers. Finally, the output of the third 5×5×5 convolutional layer is passed through a 7×7×7 convolution to adjust the output to the target resolution, thus obtaining the reconstructed positron emission tomography (PET) image.

[0068] The discriminator includes an encoder shared with the reconstruction network and a direction-aware spatial frequency discriminator. It adopts a reconstruction-guided adversarial framework. The direction-aware spatial frequency discriminator uses three parallel branches to extend the shared encoder, including a spatial branch, a direction frequency branch, and an image patch branch.

[0069] Spatial branch pairs shallow features extracted from the shared encoder Perform the operation to assess the structural authenticity:

[0070] (11);

[0071] in, The discriminative features representing the output of the spatial branch, This indicates a batch normalization operation;

[0072] The directional frequency branch aims to extract anisotropic features by decomposing spatial information along three orthogonal anatomical planes. For each orthogonal anatomical plane direction, the shallow features... First, the permutation is performed to create a series of 2D slices aligned with the selected orthogonal anatomical plane. Then, each 2D slice is separated into four sub-bands using a 2D discrete wavelet transform: low-frequency-low-frequency sub-bands. The corresponding image smoothing information, low-frequency to high-frequency sub-band Corresponding to horizontal edge details, high-frequency to low-frequency sub-bands Corresponding to vertical edge details; high-frequency to high-frequency sub-band Corresponding to texture details; only retaining the low-frequency subband. The three high-frequency component feature maps, excluding those from the original map, are extracted and reorganized through inverse permutation and reshaping to obtain a set of orientation-specific feature maps. This set of orientation-specific feature maps is then refined using a 3×3×3 convolutional layer, batch normalization, and ReLU activation. A SE module adaptively emphasizes the anatomically relevant frequency components to obtain a refined feature map. Finally, the refined feature maps corresponding to the three orthogonal anatomical planes are concatenated along the channel dimension and fused using a 1×1×1 convolutional layer to produce a unified frequency-aware feature representation. The three orthogonal anatomical planes include the axial, sagittal, and coronal planes.

[0073] Image patch branching is based on high-level semantic representations obtained from the deepest layer of the shared encoder. To assess local realism:

[0074] (12);

[0075] in, The discriminative features representing the output of image patch branches, This represents a 4×4×4 convolutional layer. This represents the activation function of a linear rectifier unit with leakage.

[0076] To obtain the final discriminative output, a uniform interpolation operation is first used to align the spatial, frequency, and image patch branch features to the resolution. The resulting feature maps are then concatenated along the channel dimension and fused using a 1×1×1 convolution. Finally, a voxel-level discriminative map is generated using sigmoid activation. This allows for the determination of whether the input positron emission tomography (PET) image is generated or real.

[0077] (13);

[0078] in, This indicates spatial alignment using trilinear interpolation.

[0079] Furthermore, the loss function during training includes the generator loss and the discriminator loss:

[0080] The generator loss is a weighted sum of the adversarial loss and the pixel-level supervision loss:

[0081] (14);

[0082] in, This represents the total loss function of the generator. This represents the mean loss value for all samples in the batch. Represents a generator. This represents the input magnetic resonance imaging (MRI) image. This represents the actual positron emission tomography (PET) images of their pairing. It is a hyperparameter that controls the balance between adversarial learning and pixel-level supervision. For the discriminator to generate the image For probability estimation of the real image, For the generator's adversarial loss term, Generate pixel loss for L1 norm;

[0083] The reconstruction pixel loss of the discriminator is defined as:

[0084] (15);

[0085] in, This represents the loss function of the discriminator. This indicates the need to rebuild the network. This represents the reconstruction output of the reconstruction network from a real positron emission tomography (PET) image. Indicates the L1 norm reconstructed pixel loss;

[0086] The adversarial loss of the discriminator is defined as:

[0087] (16);

[0088] in, This represents the adversarial loss of the discriminator D. This represents the probability estimate by which the discriminator identifies the input image as a real image. This represents two types of generated images. and These represent the mean discrimination loss values ​​for real samples and generated samples in the batch, respectively;

[0089] The overall discriminator loss is defined as:

[0090] (17);

[0091] in, This represents the contribution weight of adjusting the reconstruction constraints.

[0092] During training, the generator and discriminator are minimized separately. and Alternate updates are performed.

[0093] On the other hand, the present invention also provides an MRI-PET modality conversion system based on generative adversarial networks (GANs) for implementing the above-mentioned MRI-PET modality conversion method based on GANs, comprising:

[0094] The dataset construction and partitioning module is used to acquire the modality conversion dataset and divide it into training, validation and test sets according to a set ratio;

[0095] A generative adversarial network (GAN) building module is used to construct a GAN; the GAN is used to convert an input magnetic resonance imaging (MRI) image into a positron emission tomography (PET) image.

[0096] The training module uses the training set and validation set to train the generative adversarial network, resulting in a trained generative adversarial network.

[0097] The image generation module is used to acquire the magnetic resonance imaging (MRI) images to be converted and to generate positron emission tomography (PET) images using a generator in a trained generative adversarial network.

[0098] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0099] 1. This invention achieves hierarchical and refined utilization of anatomical priors by designing a Multi-Scale Structural Characterization Injection (MSRI) module. This module injects multi-scale anatomical prior information at different stages of the encoder, overcoming the limitations of existing methods that inject priors at a single depth or scale. This enables the network to fully perceive and understand the structural information contained in MRI images at multiple resolution levels, thereby significantly improving the alignment accuracy of fine structures in diagnostically sensitive areas such as the hippocampus and cortex, and ensuring the high reliability of synthesized PET images in key anatomical details.

[0100] 2. This invention effectively bridges the semantic gap between local features and global representations by designing an Adaptive Semantic Residual Fusion (ASRF) module. This module employs a semantic attention-guided and dual-branch attention weighting strategy to adaptively fuse fine-grained local features extracted by the bottleneck module with global contextual information encoded by the Transformer, rather than simply concatenating or adding them. This design effectively reconciles the differences between the two in terms of abstraction level and semantic scope, resolving feature conflicts and semantic ambiguity in the bottleneck region. This ensures the semantic consistency of the synthesized image in the overall metabolic pattern and the accuracy of local details, generating high-quality PET images with consistent anatomical structure and semantic coherence.

[0101] 3. This invention achieves multi-domain, fine-grained adversarial supervision by constructing a Direction-Aware Spatial-Frequency Discriminator (DASFD). This discriminator integrates three parallel discriminative branches: spatial, frequency, and image patch. This overcomes the limitation of traditional discriminators, which are confined to the spatial domain. The spatial branch ensures global structural realism, the frequency branch constrains the image's spectral characteristics to preserve necessary texture details, and the image patch branch evaluates local image realism through high-level semantic information. This multi-domain joint supervision mechanism provides the generator with more comprehensive and accurate gradient signals, driving it to synthesize results that closely approximate real PET images in terms of visual texture, spectral properties, and local anatomical realism, significantly enhancing the clinical value of the synthesized images.

[0102] Through the aforementioned collaborative and innovative technical solutions, this invention optimizes the entire process from feature extraction and fusion to adversarial supervision, jointly ensuring that the synthesized positron emission tomography (PET) images possess global semantic consistency, local detail accuracy, and spectral texture authenticity, providing a reliable guarantee for subsequent clinical diagnosis and analysis based on synthesized PET images. Attached Figure Description

[0103] Figure 1 This is a structural diagram of the generated adversarial network in an embodiment of the present invention;

[0104] Figure 2 This is a structural diagram of the multi-scale structural characterization injection module in an embodiment of the present invention;

[0105] Figure 3 This is a structural diagram of the adaptive semantic residual fusion module in an embodiment of the present invention;

[0106] Figure 4 This is a structural diagram of the direction-sensing space-frequency discriminator in an embodiment of the present invention;

[0107] Figure 5 This is a schematic diagram illustrating the performance of different algorithms in generating different samples in embodiments of the present invention;

[0108] Wherein, (a) is the mean absolute error of the images generated by different algorithms for the four disease course samples; (b) is the peak signal-to-noise ratio of the images generated by different algorithms for the four disease course samples; and (c) is the structural similarity of the images generated by different algorithms for the four disease course samples.

[0109] Figure 6 This is a cross-sectional view of the generation effect of different samples in the embodiments of the present invention;

[0110] Figure 7 This is a comparison chart showing the generation effects of different algorithms on the same sample in embodiments of the present invention;

[0111] Figure 8 This is a comparison chart showing the generation effects of different algorithms in different directions in embodiments of the present invention;

[0112] Figure 9 This is a comparison chart of cross-scenario application effects in embodiments of the present invention. Detailed Implementation

[0113] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0114] Example 1:

[0115] The MRI-PET modality transfer method based on generative adversarial networks includes the following specific steps:

[0116] Step 1: Obtain the modality conversion dataset and divide it into training set, validation set and test set according to a set ratio; the modality conversion dataset includes several samples, each sample includes a magnetic resonance imaging (MRI) image and a corresponding real positron emission tomography (PET) image of a subject;

[0117] Step 2: Construct a generative adversarial network; the generative adversarial network is used to convert the input magnetic resonance imaging (MRI) image into a positron emission tomography (PET) image;

[0118] The generative adversarial network, such as Figure 1 As shown, it includes a generator, a discriminator, and a reconstruction network;

[0119] The generator is used to synthesize high-fidelity positron emission tomography (PET) images from input magnetic resonance (MRI) images.

[0120] The reconstruction network encodes the input real positron emission tomography (PET) image using an encoder to obtain a feature map. Based on the feature map obtained by the encoder, the image is reconstructed to obtain the reconstructed PET image. According to the difference between the reconstructed PET image and the real PET image, the encoder parameters are adjusted so that the encoder can better learn the target modal features and enhance the discrimination ability.

[0121] The discriminator shares an encoder with the reconstruction network. Based on the feature map output by the encoder, it determines whether the input positron emission tomography (PET) image is a PET image generated by the generator or a real PET image. In this way, the generator parameters are adjusted to constrain the generator to generate a more realistic PET image.

[0122] During adversarial training, the discriminator simultaneously processes triples. ,in This is a true positron emission tomography (PET) image. For the generated positron emission tomography (PET) images, To reconstruct positron emission tomography (PET) images, Treated as a real sample and All are considered as generated samples, thereby guiding the generator to align its output with the target modality;

[0123] The generator adopts a U-shaped encoder-decoder design, including a generator encoder, a first bottleneck module, a Transformer module, an adaptive semantic residual fusion module, and a generator decoder;

[0124] The generative encoder is used to extract multi-level features from the input magnetic resonance image (MRI) to achieve feature extraction across spatial scales, obtaining low-level features, intermediate features, high-level features, and ultimate features, where the intermediate features, high-level features, and ultimate features serve as inputs to the generative decoder.

[0125] The first bottleneck module is used to capture local structural details from the final features extracted by the generative encoder, thereby achieving robust feature abstraction and obtaining local features;

[0126] The final features extracted by the generator encoder are projected onto a 512-channel space through a 3×3×3 convolutional layer and used as the input to the Transformer module. The module then undergoes serialization, positional encoding embedding, and multi-layer self-attention mechanism transformation to obtain a global representation.

[0127] The adaptive semantic residual fusion module is used to adaptively fuse the local features obtained by the first bottleneck module and the global representation obtained by the Transformer module to obtain deep global-local fusion features.

[0128] The generator decoder and generator encoder are connected by a skip connection, which is used to gradually restore the spatial resolution of the deep global-local fusion features obtained by the adaptive semantic residual fusion module, the intermediate features, high-level features and ultimate features output by the generator encoder, so as to obtain the generated positron emission tomography (PET) image.

[0129] The generative encoder (G-Encoder) first extracts features from the input MRI image using a 7×7×7 convolutional layer, and then projects these features into a higher-dimensional feature space to obtain low-level features. To mitigate the information decay common in pooling operations, a convolution-based double-step downsampling strategy is adopted. The G-Encoder restricts downsampling to two consecutive stages to preserve anatomical fidelity and avoid over-compression, thereby balancing computational efficiency and spatial detail preservation. Specifically, each downsampling step sequentially passes through a 5×5×5 convolutional layer and a 3×3×3 convolutional layer. This operation halves the spatial resolution while doubling the channel dimension. The low-level features, after the first downsampling, are input together with the low-level features into the first multi-scale structural representation injection module for multi-scale structural feature extraction and fusion, resulting in mid-level features. After the second downsampling, the mid-level features, along with the low-level features, are input into the second multi-scale structural representation injection module for cross-resolution structural prior information injection and feature enhancement, resulting in high-level features. Finally, the high-level features pass through a 5×5×5 convolutional layer to obtain the ultimate features. The 3×3×3 convolutional layer sequentially includes a 3×3×3 convolution with a stride of 2, batch normalization, and a ReLU activation function. The 5×5×5 convolutional layer sequentially includes a 5×5×5 convolution with a stride of 1, batch normalization, and a ReLU activation function. The 7×7×7 convolutional layer sequentially includes a 7×7×7 convolution, batch normalization, and a ReLU activation function.

[0130] To address the semantic and spatial misalignment caused by modal differences, two multi-scale structural representation injection (MSRI) modules are integrated into the generative encoder path. These modules introduce structure-aware priors at multiple spatial resolutions, such as... Figure 2As shown, the Multi-Scale Structure Representation Injection (MSRI) module first inputs low-level features into a Multi-Scale Structure Convolution (MSC) block to obtain multi-scale structure representations. Multi-scale structural characterization After downsampling to adjust the resolution, the features are concatenated with those obtained after the first or second downsampling by the generative encoder, and then fused using 1×1×1 convolution and GELU activation. To enhance spatial discriminability, the efficient local attention (ELA) mechanism is extended to a 3D form, and the fused features are used as the input tensor. The input is fed into an efficient local attention (ELA) mechanism to obtain attention maps in depth, height, and width dimensions, respectively. These maps are then multiplied with the input tensor to enhance spatial encoding for orientation awareness, resulting in intermediate or high-level features. Indicates the number of channels. Represents the number of depth dimensions. Indicates the height dimension. Indicates the number of width dimensions;

[0131] In the Multi-Scale Structured Convolution (MSC) block, low-level features are first processed by a 1×1×1 convolution. The convolution result is then fed into three parallel branches: a depthwise separable branch, an asymmetric convolution branch, and a dilated convolution branch. The depthwise separable branch uses a 3×3×3 depthwise separable convolution, group normalization (GN), and ReLU activation function to efficiently extract fine-grained local textures. The asymmetric convolution branch applies cascaded asymmetric convolutions (1,3,5), asymmetric convolutions (5,3,1), group normalization (GN), and ReLU activation function to increase sensitivity to orientation-specific anatomical patterns. The dilated convolution branch uses a 5×5×5 dilated convolution with a dilation rate of 2, group normalization (GN), and ReLU activation function to expand the receptive field to encode mid-range spatial dependencies. The outputs of the three branches are element-wise summed and then passed through a multilayer perceptron (MLP). The original low-level features are preserved through residual connections to obtain the multi-scale structural representation. ;

[0132] In the efficient local attention (ELA) mechanism, first along each dimension Global average pooling (GAP) is applied to the two orthogonal axes to produce a shape of Compact representation, Representing dimension, Represents depth dimension, Indicates height dimension, Indicates the width dimension. The length of the compact representation is then indicated; subsequently, the three-dimensional compact representations are each passed through a one-dimensional convolution with a kernel size of 5, followed by group normalization and sigmoid activation, resulting in three attention maps; the resulting three attention maps are then reshaped to... , and Dimensions, resulting in the final attention map with depth, height, and width dimensions;

[0133] (1);

[0134] in, Representing dimensions Attention map, This represents the sigmoid activation function. Indicates group normalization, This represents a one-dimensional convolution with a kernel size of 5. Indicates along the orthogonal dimension Average pooling is performed on the two axes;

[0135] The first bottleneck module uses several 3D Res2Net blocks to perform multi-scale feature extraction in a parametrically efficient manner, enhancing the modeling of local anatomical structures; in this embodiment, six stacked 3D Res2Net blocks are used to capture detailed anatomical patterns.

[0136] Specifically, the feature maps input to each 3D Res2Net block are first processed by a 1×1×1 convolution and then evenly divided into four feature subsets. Then, the four feature subsets are aggregated according to the hierarchical residual method to obtain four sets of output features. Four sets of output features The data is concatenated along the channel dimension and fused through a 1×1×1 convolution to form the final output, i.e., local features;

[0137] The aggregation of the hierarchical residuals is represented as follows:

[0138] (2);

[0139] in, Indicates the kernel size as 3D convolution operation;

[0140] For global semantic modeling, the Transformer module projects the final features through a 3×3×3 convolutional layer onto a 512-channel space, reshaping the result into a voxel sequence. To preserve spatial correspondence, learnable positional encodings are added to the voxel sequence. The voxel sequence with added positional encodings is then passed through four cascaded Transformer blocks to obtain the output sequence. Finally, the output sequence is reshaped back into a 3D structure and upsampled to align with the local features obtained by the first bottleneck module. The upsampling operation includes channel expansion and 3D pixel reconstruction. The Transformer block includes an 8-head self-attention module and a feedforward network (FFN), both embedded in residual connections and layer normalization.

[0141] To effectively integrate local features and global representations, an Adaptive Semantic Residual Fusion (ASRF) module is designed, which adaptively fuses local features derived from Res2Net. Global representation encoded with the Transformer module ,like Figure 3 As shown, firstly from the global representation Exporting a global semantic attention graph as global guidance:

[0142] (3);

[0143] in, Represents the global semantic attention graph. This indicates a convolutional layer with a 1×1×1 kernel. This indicates a convolutional layer with a 3×3×3 kernel. Denotes the Gaussian error linear unit activation function;

[0144] The local features are then recalibrated using a residual mechanism:

[0145] (4);

[0146] Where ⊙ denotes successive multiplication. This represents the local features after recalibration;

[0147] Recalibrated local features With global features The data is spliced ​​together to form a unified representation, and then further refined through convolution:

[0148] (5);

[0149] in, Represents a unified global-local representation. This indicates a splicing operation along the channel dimension;

[0150] To generate adaptive fusion weights that control the contributions of each feature branch, a unified global-local representation is required. A two-branch attention mechanism is applied, where the channel attention path is obtained in the following way:

[0151] (6);

[0152] in, This represents the channel attention map. This represents a multilayer perceptron. This indicates a global average pooling operation;

[0153] The spatial attention path first performs max pooling and average pooling along the channel dimension, and then processes the max pooling results... and After splicing and merging:

[0154] (7);

[0155] in, Represents a spatial attention map;

[0156] The outputs of the two attention branches are combined to generate a unified gating signal:

[0157] (8);

[0158] in, Indicates a gating signal;

[0159] Finally, the attention-guided fusion of global representation and local features is expressed as follows:

[0160] (9);

[0161] in, It is a learnable scalar that controls the fusion ratio. To simultaneously preserve long-range semantic coherence and detailed deep global-local fusion features;

[0162] The generator decoder includes three feature fusion decoding blocks equipped with skip connections. An upsampling operation is performed between every two feature fusion decoding blocks. Specifically, the input of the first feature fusion decoding block is the ultimate feature and the deep global-local fusion feature; the input of the second feature fusion decoding block is the output of the first feature fusion decoding block after upsampling and the high-level feature; the input of the third feature fusion decoding block is the output of the second feature fusion decoding block after upsampling and the mid-level feature; finally, the output of the third feature fusion decoding block is passed through a 7×7×7 convolutional layer to obtain the generated positron emission tomography (PET) image.

[0163] The process of a feature fusion decoding block and upsampling operation can be represented as follows:

[0164] (10);

[0165] in, This represents the decoder features that have been fused and enhanced after skip connections. This represents a 5×5×5 convolutional layer. This indicates the squeeze excitation (SE) channel attention module, used for adaptive recalibration of the channel response; This represents encoder features, which can be categorized as final features, high-level features, or intermediate features. This represents the decoder features, which can be deep global-local fusion features, the output of the first feature fusion decoding block after upsampling, or the output of the second feature fusion decoding block after upsampling. Indicates splicing;

[0166] The reconstruction network employs an encoder-decoder architecture, using real positron emission tomography (PET) images. The input is a reconstructed positron emission tomography (PET) image. As a semantic reference for modality perception discrimination; the encoder first uses a 7×7×7 convolutional layer and a 5×5×5 convolutional layer to obtain shallow features. Then, through a series of 3×3×3 convolutional layers, 5×5×5 convolutional layers, 3×3×3 convolutional layers, 5×5×5 convolutional layers, and a second bottleneck module, the high-level semantic representation is obtained. The decoder uses high-level semantic representation. As input, the image is passed through three consecutive 5×5×5 convolutional layers. Upsampling is performed between every two 5×5×5 convolutional layers. Finally, the output of the third 5×5×5 convolutional layer is passed through a 7×7×7 convolution to adjust the output to the target resolution, thus obtaining the reconstructed positron emission tomography (PET) image.

[0167] The discriminator includes an encoder shared with the reconstruction network and a Direction-Aware Spatial-Frequency Discriminator (DASFD). Employing a reconstruction-guided adversarial framework, the DSFD extends the shared encoder using three parallel branches: a spatial branch, a direction-frequency branch, and an image patch branch. Figure 4 As shown.

[0168] Spatial branch pairs shallow features extracted from the shared encoder Perform the operation to assess the structural authenticity:

[0169] (11);

[0170] in, The discriminative features representing the output of the spatial branch, This indicates a batch normalization operation;

[0171] The directional frequency branch aims to extract anisotropic features by decomposing spatial information along three orthogonal anatomical planes (axial, sagittal, and coronal). For each orthogonal anatomical plane, shallow features... First, the permutation is performed to create a series of 2D slices aligned with the selected orthogonal anatomical plane. Then, each 2D slice is separated into four sub-bands using 2D discrete wavelet transform (DWT), namely the low-frequency-low-frequency sub-band. The corresponding image smoothing information, low-frequency to high-frequency sub-band Corresponding to horizontal edge details, high-frequency to low-frequency sub-bands Corresponding to vertical edge details; high-frequency to high-frequency sub-band Corresponding to texture details; only retaining the low-frequency subband. The three high-frequency component feature maps, excluding those from the original map, are extracted and reorganized through inverse permutation and reshaping to obtain a set of orientation-specific feature maps. This set of orientation-specific feature maps is then refined using a 3×3×3 convolutional layer, batch normalization, and ReLU activation. A SE module adaptively emphasizes the anatomically relevant frequency components to obtain a refined feature map. Finally, the refined feature maps corresponding to the three orthogonal anatomical planes are concatenated along the channel dimension and fused using a 1×1×1 convolutional layer to produce a unified frequency-aware feature representation. ;

[0172] Image patch branching is based on high-level semantic representations obtained from the deepest layer of the shared encoder. To assess local realism, 3D convolutional layers are applied to extract discriminative cues at the image patch level:

[0173] (12);

[0174] in, The discriminative features representing the output of image patch branches, This represents a 4×4×4 convolutional layer. This represents the activation function of a linear rectifier unit with leakage.

[0175] To obtain the final discriminative output, a uniform interpolation operation is first used to align the spatial, frequency, and image patch branch features to the resolution. The resulting feature maps are then concatenated along the channel dimension and fused using a 1×1×1 convolution. Finally, a voxel-level discriminative map is generated using sigmoid activation. This allows for the determination of whether the input positron emission tomography (PET) image is a PET image generated by the generator or a real PET image.

[0176] (13);

[0177] in, This indicates spatial alignment using trilinear interpolation.

[0178] Step 3: Train the generative adversarial network using the training set and validation set to obtain the trained generative adversarial network;

[0179] The framework described in this invention achieves the synthesis of anatomically consistent positron emission tomography (PET) images from input T1-weighted MRI body data by optimizing the generator G, discriminator D, and auxiliary PET reconstruction network R. The generator and discriminator are jointly trained in an adversarial paradigm, while the reconstruction network provides semantic priors to guide the discrimination process.

[0180] The generator loss consists of two components: an adversarial loss that encourages the generation of realistic synthetic images, and a loss that forces the generated images to be similar to the target PET at the voxel level. Loss, the complete generator loss function is defined as:

[0181] (14);

[0182] in, This represents the total loss function of the generator. This represents the mean loss value for all samples in the batch. Represents a generator. This represents the input magnetic resonance imaging (MRI) image. This represents the true positron emission tomography (PET) images of their paired counterparts. It is a hyperparameter that controls the balance between adversarial learning and pixel-level supervision. For the discriminator to generate the image For probability estimation of the real image, For the generator's adversarial loss term, Generate pixel loss for L1 norm;

[0183] The reconstruction network R, which shares the encoder with the discriminator, provides a reliable semantic reference during adversarial discrimination after optimization. Its loss function is defined as:

[0184] (15);

[0185] in, This represents the loss function of the discriminator. This indicates the need to rebuild the network. This represents the reconstruction output of the reconstruction network from a real positron emission tomography (PET) image. Indicates the L1 norm reconstructed pixel loss;

[0186] Meanwhile, discriminator D was trained to distinguish between real positron emission tomography (PET) images and two types of generated images (generator prediction results). and rebuild network output The corresponding adversarial losses are expressed as follows:

[0187] (16);

[0188] in, This represents the adversarial loss of the discriminator D. This represents the probability estimate by which the discriminator identifies the input image as a real image. This represents two types of generated images. and These represent the mean discrimination loss values ​​for real samples and generated samples in the batch, respectively;

[0189] To incorporate semantic guidance into the adversarial process, the overall discriminator loss is defined as:

[0190] (17);

[0191] in, This represents the contribution weight of adjusting the reconstruction constraints.

[0192] During training, the generator and discriminator are minimized separately. and Alternating updates are performed. This joint optimization mechanism enables the generator to produce PET volume data that is both structurally accurate and perceptually indistinguishable from real images.

[0193] Step 4: Acquire the magnetic resonance image to be converted (MRI) and use the generator in the trained generative adversarial network to generate a positron emission tomography (PET) image;

[0194] The data for this study were derived from a subset of the Alzheimer's Disease Neuroimaging Initiative (ADNI) database, comprising 549 participants. Each participant had registered T1-weighted MRI and PET scans acquired in the same imaging session. To prevent information leakage due to longitudinal variations within the same participant, only one pair of images was retained for analysis for each participant. Based on clinical diagnosis, all participants were divided into four groups: cognitively normal (CN), subjective memory complaints (SMC), mild cognitive impairment (MCI), and Alzheimer's disease (AD). Demographic and neuropsychological statistics—including age, sex, Mini-Mental State Examination (MMSE) score, and Clinical Dementia Rating Scale-Box Sum of Bounds (CDR-SOB) score—are summarized in Table 1.

[0195] Table 1. Statistical Table of Sample Information;

[0196]

[0197] All imaging data underwent a standardized preprocessing procedure to ensure spatial consistency and facilitate model training. The specific procedure included: first, N4 bias field correction was used to address intensity inhomogeneities; then, MRI and PET images were rigidly registered to the MNI152 standard template, and skull dissection was performed to eliminate non-brain tissue and suppress background noise; to unify spatial resolution and reduce computational complexity, all volume data were resampled to a 128×128×128 voxel size; finally, image intensity was normalized to the range [-1, 1] to ensure network training stability.

[0198] All experiments were implemented using the PyTorch 3.8 framework and conducted on a workstation equipped with an NVIDIA GeForce RTX 4090 GPU. The Adam optimizer was used, with parameters set to... =0.9, =0.999. The initial learning rate is set to... The model was trained for 200 epochs. A trade-off was made with the hyperparameters. and Based on experience, all values ​​are set to 100 to balance the contributions of different loss components.

[0199] To quantitatively evaluate the quality of synthesized positron emission tomography (PET) images, three widely used metrics were employed: mean absolute error (MAE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM). These metrics collectively assess voxel-level fidelity, noise robustness, and perceptual consistency.

[0200] The MAE index reflects the average absolute deviation between the generated image and the real image. The lower the MAE value, the higher the accuracy of voxel-level reconstruction.

[0201] PSNR is the ratio of the maximum possible voxel intensity to the logarithm of the mean square reconstruction error. A higher PSNR indicates better signal fidelity and fewer noise artifacts.

[0202] SSIM assesses perceptual similarity by simultaneously comparing brightness, contrast, and structural information. A higher SSIM value indicates better structural alignment and perceptual realism in the synthesized output.

[0203] To quantitatively evaluate the performance of the proposed method, it was compared with several representative image synthesis models on the ADNI dataset. Specifically, pGAN and TransGAN were implemented according to the architectures described in the papers "Image synthesis in multi-contrast MRI with conditional generative adversarial networks," and "3D transformer-gan for high-quality pet reconstruction," respectively; AEGAN was implemented based on the paper "Autoencoder-driven multimodal collaborative learning for medical image synthesis," and adapted to 3D input and unimodal conditions; the model proposed in the paper "Turning brain MRI into diagnostic pet: 15o-water pet CBF synthesis from multi-contrast MRI via attention-based encoder–decoder networks" was modified to support MRI-only input and named TurnNet. All methods were trained and evaluated using a consistent five-fold cross-validation protocol to ensure fairness, and the final metric was the average of the results of each fold.

[0204] Table 2. Performance comparison of T1 generated PET tasks;

[0205]

[0206] As shown in Table 2, the framework proposed in this invention achieves the best performance across all evaluation metrics. Among the baseline methods, pGAN performs the weakest, with an MAE of 0.0238, a PSNR of 24.40 dB, and a SSIM of 86.68%, reflecting its insufficient ability to preserve anatomical structures and maintain strength consistency. TransGAN and TurnNet achieve moderate improvements, with SSIMs reaching 88.90% and 89.17%, respectively, indicating their potential in enhancing structural alignment. However, compared to AEGAN and our proposed method, these two methods have relatively poor MAE and PSNR. These results suggest that attention-based strategies (such as those implemented in the skip connections of TurnNet and the bottleneck of TransformerGAN) can enhance structural alignment at specific feature scales, but may be insufficient in maintaining overall strength level fidelity. AEGAN exhibits competitive performance, benefiting from a reconstruction-guided discriminator that enhances structural awareness in adversarial learning, with an MAE of 0.0208, a PSNR of 25.44 dB, and an SSIM of 89.54%. This method further improves upon AEGAN in MAE, PSNR, and SSIM by 8.6%, 3.6%, and 1.3%, respectively. These improvements demonstrate the effectiveness of our generation strategy and discriminator design in producing PET positron emission tomography images with higher anatomical accuracy and voxel-level precision.

[0207] To assess robustness across different disease severity levels, further stratification was performed based on clinical diagnosis. Figure 5 Performance across diagnostic categories is illustrated using radar charts. While synthetic performance remained relatively stable in the CN and SMC groups, a significant decline was observed in the MCI and AD categories, indicating a decrease in image quality with increasing disease severity. This performance degradation was more pronounced in models with limited structural modeling capabilities (such as pGAN), while models incorporating stronger semantic priors (such as AEGAN and our method) demonstrated better resilience. Decreased accuracy in late-stage subjects can be attributed to increased anatomical variability due to disease-related atrophy (such as in the hippocampus and cortical regions). These structural changes reduce the regularity and information content of MRI features, making the generation of accurate PET positron emission tomography (PET) images more complex. Despite these challenges, our method maintained excellent quantitative performance across all categories, highlighting its robustness and generalization ability under diverse clinical conditions and anatomical variations.

[0208] Figure 6This paper presents multi-view synthetic results from representative test subjects at two different disease stages, and performs qualitative evaluation of the synthesized positron emission tomography (PET) images. This method continuously generates visually coherent PET volumetric data, exhibiting strong structural alignment with real data. It successfully captures global metabolic patterns and local anatomical details, demonstrating powerful characterization capabilities across different spatial scales. This cross-view... Figure 1 Consistency is particularly beneficial for clinical tasks requiring volumetric precision, such as metabolic quantification, surgical planning, and longitudinal monitoring of disease progression.

[0209] Figure 7 The synthetic performance at different disease stages was further investigated by displaying representative outputs and absolute error plots for the four categories: CN, SMC, MCI, and AD. Compared with baseline methods, the PET images generated by this method exhibit clearer anatomical contours and improved structural coherence, particularly in the cortical and subcortical regions. Correlation error plots show a significant reduction in voxel-level residuals, indicating a closer alignment with real data. Notably, in clinically relevant regions (such as the frontal and parietal lobes—regions typically showing early metabolic decline in Alzheimer's disease), this method more accurately reproduces characteristic uptake patterns, highlighting its clinical relevance and modeling capabilities. These improvements are attributed to the integration of modality-aware feature extraction and multi-scale adversarial supervision.

[0210] Figure 8 Further supplementary analysis of anatomical planes provides additional evidence of spatial consistency. In axial, sagittal, and coronal views, this method produces anatomically realistic structures with clear tissue boundaries and evident metabolic contrast, while maintaining high fidelity to reference PET positron emission tomography (PET) images. In contrast, baseline methods typically exhibit varying degrees of spatial blurring and anatomical distortion. These visual results clearly demonstrate the superiority of this framework in synthesizing anatomically realistic and functionally coherent PET positron emission tomography images across a wide range of anatomical planes.

[0211] To comprehensively evaluate the individual contribution of each proposed module, a series of ablation experiments were conducted by progressively integrating the MSRI, ASRF, and DASFD components into the baseline network. The corresponding quantitative results are summarized in Table 3.

[0212] Table 3. Overall ablation;

[0213]

[0214] Starting from the baseline configuration (MAE 0.0211, PSNR 25.34 dB, SSIM 89.26%), all metrics showed consistent improvement after adding the MSRI module. Specifically, MAE decreased by 0.0005, while PSNR and SSIM increased by 0.31 dB and 0.42%, respectively. These gains confirm the effectiveness of embedding multi-scale structural priors in the encoder to enhance anatomical fidelity and spatial representation. Further improvements were observed after adding the ASRF module, which enables semantically guided residual fusion between local and global representations. This configuration achieved an MAE of 0.0199, a PSNR of 25.91 dB, and an SSIM of 89.91%, highlighting the module's ability to enhance semantic consistency and cross-scale feature alignment. While the MAE reduction from adding the DASFD module alone was relatively small, it significantly improved perceptual similarity, increasing SSIM by 0.55%. This underscores the advantages of introducing orientation- and frequency-sensitive adversarial supervision to enhance spatial realism and structural coherence. When MSRI and ASRF are combined, the network benefits from complementary structural and semantic guidance, achieving a MAE of 0.0195, a PSNR of 26.12 dB, and an SSIM of 90.24%. The best overall performance is achieved when all three modules are integrated, with a MAE of 0.0190, a PSNR of 26.35 dB, and an SSIM of 90.66%. These results collectively demonstrate the synergistic benefits of the proposed modules in improving anatomical accuracy, semantic consistency, and cross-modal fidelity.

[0215] To further investigate the contributions of different feature sources within the ASRF module, four model configurations were evaluated under consistent experimental settings: (1) Baseline-C uses a Res2Net-based convolutional path at the bottleneck as a convolution-only baseline; (2) Baseline-T replaces the Res2Net path with a Transformer branch to capture global semantic dependencies; (3) Baseline-CT-cat integrates convolutional and Transformer features by direct concatenation without adaptive fusion; and (4) Baseline-ASRF selectively fuses features from two paths using the proposed ASRF module.

[0216] Table 4 ASRF ablation;

[0217]

[0218] As shown in Table 4, Baseline-T outperforms Baseline-C on all evaluation metrics, demonstrating the advantage of global context modeling at the bottleneck stage for cross-modal synthesis. Baseline-CT-cat further outperforms the two single-branch variants, indicating that local and global features provide complementary information. Among all configurations, Baseline-ASRF achieves the best results with an MAE of 0.0199, a PSNR of 25.91 dB, and an SSIM of 89.91%. These consistent gains confirm the effectiveness of the adaptive fusion strategy, enabling the network to simultaneously preserve fine-grained anatomical structure and align high-level semantic information.

[0219] To investigate the impact of discriminator design on synthesis performance, ablation studies were conducted on three configurations: (1) Patch-Dis uses a conventional image patch-based discriminator; (2) Shallow-Dis incorporates spatial and frequency features extracted from a shallow encoder; and (3) DASFD uses the proposed direction-aware spatial-frequency discriminator.

[0220] Table 5 DASFD ablation;

[0221]

[0222] Quantitative results are summarized in Table 5. Compared to Patch-Dis, the Shallow-Dis variant achieved better performance, validating the effectiveness of incorporating mixed spatial frequency cues into adversarial supervision. Among all variants, the full model employing DASFD achieved the best results, with an MAE of 0.0205, a PSNR of 25.66 dB, and an SSIM of 89.81%. These improvements highlight the advantages of utilizing a multi-branch discriminator, providing richer perceptual guidance to the generator, thereby improving the structural fidelity of synthesized positron emission tomography (PET) images.

[0223] While the proposed framework demonstrates strong performance in MRI-to-PET synthesis tasks, evaluating its generalization ability in other cross-modal transition scenarios remains crucial. Therefore, the task of synthesizing FLAIR images from T1-weighted MRI is further evaluated on the ADNI-3 dataset. Compared to PET synthesis, synthesizing FLAIR from T1 MRI presents additional challenges related to tissue contrast variations and lesion localization, offering complementary evaluations of the model's structural sensitivity and domain adaptability. To ensure consistency, all preprocessing steps, data segmentation, and training protocols are consistent with those used in the PET synthesis experiments.

[0224] Table 6 Performance comparison of T1-generated Flair tasks;

[0225]

[0226] As summarized in Table 6, our method outperforms competing methods on all evaluation metrics. The proposed algorithm achieves the lowest MAE of 0.0138, the highest PSNR of 28.63 dB, and the best SSIM of 93.61%, demonstrating improved voxel-level accuracy and anatomical fidelity. Figure 9 Visual comparisons further highlight the qualitative improvements of this method. Across all diagnostic categories, the synthetic FLAIR images generated by this model exhibit clearer anatomical boundaries and more pronounced lesion contrast.

[0227] Example 2:

[0228] An MRI-PET modality transfer system based on generative adversarial networks (GANs) is used to implement an MRI-PET modality transfer method based on GANs, comprising:

[0229] The dataset construction and partitioning module is used to acquire the modality conversion dataset and divide it into training, validation and test sets according to a set ratio;

[0230] A generative adversarial network (GAN) building module is used to construct a GAN; the GAN is used to convert an input magnetic resonance imaging (MRI) image into a positron emission tomography (PET) image.

[0231] The training module uses the training set and validation set to train the generative adversarial network, resulting in a trained generative adversarial network.

[0232] The image generation module is used to acquire the magnetic resonance imaging (MRI) images to be converted and to generate positron emission tomography (PET) images using a generator in a trained generative adversarial network.

[0233] Example 3:

[0234] This embodiment proposes an electronic device, including: one or more processors, and a memory for storing instructions, which, when executed by the one or more processors, cause the one or more processors to perform the MRI-PET modality conversion method based on generative adversarial networks.

[0235] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the MRI-PET modality conversion method based on generative adversarial networks as described in the embodiments. It is understood that the electronic device may also include input / output (I / O) interfaces and communication components.

[0236] The processor is used to execute all or part of the steps in the generative adversarial network-based MRI-PET modality conversion method as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.

[0237] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the MRI-PET modality conversion method based on generative adversarial networks described in the above embodiments.

[0238] Example 4:

[0239] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0240] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the MRI-PET modality conversion method based on generative adversarial networks described in the various embodiments of this application.

[0241] The aforementioned storage media include: flash memory, hard disks, multimedia cards, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disks, optical discs, servers, APP (Application) application stores, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the aforementioned generative adversarial network-based MRI-PET modality conversion method.

[0242] Example 5:

[0243] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned MRI-PET modality conversion method based on generative adversarial networks.

[0244] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0245] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0246] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of this disclosure and its equivalents, then the intent of this disclosure also includes these modifications and variations.

Claims

1. An MRI-PET modality transfer method based on generative adversarial networks, characterized in that, The specific steps include the following: A modality conversion dataset is obtained and divided into a training set, a validation set, and a test set according to a set ratio. The modality conversion dataset includes several samples, each of which contains a magnetic resonance imaging (MRI) image and a corresponding real positron emission tomography (PET) image of a subject. A generative adversarial network is constructed; the generative adversarial network is used to convert an input magnetic resonance imaging (MRI) image into a positron emission tomography (PET) image. The generative adversarial network includes a generator, a discriminator, and a reconstruction network; The generator is used to synthesize high-fidelity positron emission tomography (PET) images from input magnetic resonance (MRI) images. The reconstruction network encodes the input real positron emission tomography (PET) image using an encoder to obtain a feature map. Based on this feature map, the image is reconstructed to obtain the reconstructed PET image. The encoder parameters are adjusted according to the difference between the reconstructed PET image and the real PET image. The discriminator shares an encoder with the reconstruction network. It determines whether the input positron emission tomography (PET) image is generated or real based on the feature map output by the encoder, and adjusts the generator parameters accordingly. The generative adversarial network is trained using the training set and the validation set to obtain the trained generative adversarial network; The system acquires MRI images to be converted and uses a generator in a trained generative adversarial network to generate PET images.

2. The MRI-PET modality conversion method based on generative adversarial networks according to claim 1, characterized in that, The discriminator simultaneously processes triplets. ,in These are real positron emission tomography (PET) images. PET images generated by the generator. To reconstruct positron emission tomography (PET) images, Treated as a real sample and All are considered as generated samples.

3. The MRI-PET modality transfer method based on generative adversarial networks according to claim 2, characterized in that, The generator includes a generator encoder, a first bottleneck module, a Transformer module, an adaptive semantic residual fusion module, and a generator decoder; The generative encoder is used to extract multi-level features from the input magnetic resonance image (MRI) to obtain low-level features, intermediate features, high-level features and ultimate features, wherein the intermediate features, high-level features and ultimate features are used as inputs to the generative decoder. The first bottleneck module is used to capture local structural details from the final features extracted by the generative encoder to obtain local features; The final features extracted by the generator encoder are projected onto a 512-channel space through a 3×3×3 convolutional layer as the input to the Transformer module. The module then undergoes serialization, positional encoding embedding, and multi-layer self-attention transformation to obtain a global representation. The 3×3×3 convolutional layer consists of a 3×3×3 convolution with a stride of 2, batch normalization, and a ReLU activation function. The adaptive semantic residual fusion module is used to adaptively fuse the local features obtained by the first bottleneck module and the global representation obtained by the Transformer module to obtain deep global-local fusion features. The generator decoder and generator encoder are connected by a skip connection, which is used to gradually restore the spatial resolution of the deep global-local fusion features obtained by the adaptive semantic residual fusion module, the intermediate features, high-level features and ultimate features output by the generator encoder, so as to obtain the generated positron emission tomography (PET) image.

4. The MRI-PET modality transfer method based on generative adversarial networks according to claim 3, characterized in that, The generative encoder first uses a 7×7×7 convolutional layer to extract features from the input magnetic resonance image (MRI) and projects them into a high-dimensional feature space to obtain low-level features. A convolution-based double downsampling strategy is adopted. Each downsampling passes through a 5×5×5 convolutional layer and a 3×3×3 convolutional layer in sequence. The low-level features obtained after the first downsampling are input together with the low-level features into the first multi-scale structural representation injection module for multi-scale structural feature extraction and fusion to obtain the intermediate features. After a second downsampling, the intermediate features are input together with the low-level features into the second multi-scale structural representation injection module to perform cross-resolution structural prior information injection and feature enhancement, resulting in high-level features. The final high-level features are then passed through a 5×5×5 convolutional layer to obtain the ultimate features; The 5×5×5 convolutional layer consists of a 5×5×5 convolution with a stride of 1, batch normalization, and a ReLU activation function; the 7×7×7 convolutional layer consists of a 7×7×7 convolution, batch normalization, and a ReLU activation function. The multi-scale structure representation injection module first inputs low-level features into a multi-scale structure convolutional block to obtain the multi-scale structure representation. Multi-scale structural characterization After downsampling and resolution adjustment, the features are concatenated with those obtained from the first or second downsampling by the generative encoder, and then fused using 1×1×1 convolution and GELU activation. The efficient local attention mechanism is extended to a 3D form, and the fused features are used as the input tensor. The input is fed into an efficient local attention mechanism to obtain attention maps in depth, height, and width dimensions, respectively. These maps are then multiplied with the input tensor to enhance spatial encoding for orientation awareness, resulting in intermediate or high-level features. Indicates the number of channels. Represents the number of depth dimensions. Indicates the height dimension. Indicates the number of width dimensions; In the multi-scale structured convolution, low-level features first undergo a 1×1×1 convolution, and then the convolution result is input into three parallel branches: a depthwise separable branch, an asymmetric convolution branch, and a dilated convolution branch. The depthwise separable branch uses a 3×3×3 depthwise separable convolution, group normalization, and ReLU activation function to efficiently extract fine-grained local textures. The asymmetric convolution branch applies cascaded asymmetric convolutions (1,3,5) and (5,3,1), group normalization, and ReLU activation function to increase sensitivity to orientation-specific anatomical patterns. The dilated convolution branch uses a 5×5×5 dilated convolution with a dilation rate of 2, group normalization, and ReLU activation function to expand the receptive field to encode mid-range spatial dependencies. The outputs of the three branches are summed element-wise and then passed through a multilayer perceptron. The original low-level features are preserved through residual connections to obtain a multi-scale structural representation. ; In efficient local attention mechanisms, first along each dimension Global average pooling is applied to the two orthogonal axes to produce a shape of Compact representation, Representing dimension, Represents depth dimension, Indicates height dimension, Indicates the width dimension. The length of the compact representation is then indicated; subsequently, the three-dimensional compact representations are each passed through a one-dimensional convolution with a kernel size of 5, followed by group normalization and sigmoid activation, resulting in three attention maps; the resulting three attention maps are then reshaped to... , and Dimensions are used to obtain the final attention map with depth, height, and width dimensions.

5. The MRI-PET modality transfer method based on generative adversarial networks according to claim 3, characterized in that, The first bottleneck module uses several 3D Res2Net blocks; Specifically, the feature maps input to each 3D Res2Net block are first processed by a 1×1×1 convolution and then evenly divided into four feature subsets. Then, the four feature subsets are aggregated according to the hierarchical residual method to obtain four sets of output features. The four sets of output features The data is concatenated along the channel dimension and fused through a 1×1×1 convolution to form the final output, i.e., local features; The aggregation of the hierarchical residuals is represented as follows: （2）； in, Indicates the kernel size as 3D convolution operation; The Transformer module reshapes the final features into a voxel sequence by projecting them through a 3×3×3 convolutional layer onto a 512-channel space. Learnable positional encodings are then added to the voxel sequence. The voxel sequence with added positional encodings is then passed through four cascaded Transformer blocks to obtain the output sequence. Finally, the output sequence is reshaped back into a 3D structure and upsampled to align with the local features obtained by the first bottleneck module. The upsampling operation includes channel expansion and 3D pixel reconstruction operations.

6. The MRI-PET modality transfer method based on generative adversarial networks according to claim 3, characterized in that, The adaptive semantic residual fusion module adaptively fuses local features. Global representation encoded with the Transformer module : First, from the global representation Exporting a global semantic attention graph as global guidance: （3）； in, Represents the global semantic attention graph. This indicates a convolutional layer with a 1×1×1 kernel. This indicates a convolutional layer with a 3×3×3 kernel. Denotes the Gaussian error linear unit activation function; The local features are then recalibrated using a residual mechanism: （4）； Where ⊙ denotes successive multiplication. This represents the local features after recalibration; Recalibrated local features With global features The data is spliced ​​together to form a unified representation, and then further refined through convolution: （5）； in, Represents a unified global-local representation. This indicates a splicing operation along the channel dimension; Unified global-local representation A two-branch attention mechanism is applied, where the channel attention path is obtained in the following way: （6）； in, This represents the channel attention map. This represents a multilayer perceptron. This indicates a global average pooling operation; The spatial attention path first performs max pooling and average pooling along the channel dimension, and then processes the max pooling results... and After splicing and merging: （7）； in, Represents a spatial attention map; The outputs of the two attention branches are combined to generate a unified gating signal: （8）； in, Indicates a gating signal; Finally, the attention-guided fusion of global representation and local features is expressed as follows: （9）； in, It is a learnable scalar that controls the fusion ratio. This approach aims to simultaneously preserve long-range semantic coherence and detailed deep global-local fusion features.

7. The MRI-PET modality transfer method based on generative adversarial networks according to claim 3, characterized in that, The generator decoder includes three feature fusion decoding blocks equipped with skip connections. An upsampling operation is performed between every two feature fusion decoding blocks. Specifically, the input to the first feature fusion decoding block is the final feature and the deep global-local fusion feature; the input to the second feature fusion decoding block is the upsampled output of the first feature fusion decoding block and the high-level feature; the input to the third feature fusion decoding block is the upsampled output of the second feature fusion decoding block and the mid-level feature; finally… The output of the third feature fusion decoding block is passed through a 7×7×7 convolutional layer to obtain the generated positron emission tomography (PET) image. The process of a feature fusion decoding block and upsampling operation can be represented as follows: （10）； in, This represents the decoder features that have been fused and enhanced after skip connections. This represents a 5×5×5 convolutional layer. This indicates a squeeze-excitation channel attention module used for adaptive recalibration of the channel response; This represents encoder features, which can be categorized as final features, high-level features, or intermediate features. This represents the decoder features, which can be deep global-local fusion features, the output of the first feature fusion decoding block after upsampling, or the output of the second feature fusion decoding block after upsampling. Indicates splicing.

8. The MRI-PET modality transfer method based on generative adversarial networks according to claim 1, characterized in that, The reconstruction network employs an encoder-decoder architecture, using real positron emission tomography (PET) images. The input is a reconstructed positron emission tomography (PET) image. As a semantic reference for modality perception discrimination; the encoder first uses a 7×7×7 convolutional layer and a 5×5×5 convolutional layer to obtain shallow features. Then, through a series of 3×3×3 convolutional layers, 5×5×5 convolutional layers, 3×3×3 convolutional layers, 5×5×5 convolutional layers, and a second bottleneck module, the high-level semantic representation is obtained. ; Decoder with high-level semantic representation As input, the image is passed through three consecutive 5×5×5 convolutional layers. Upsampling is performed between every two 5×5×5 convolutional layers. Finally, the output of the third 5×5×5 convolutional layer is passed through a 7×7×7 convolution to adjust the output to the target resolution, thus obtaining the reconstructed positron emission tomography (PET) image. The discriminator includes an encoder shared with the reconstruction network and a direction-aware spatial frequency discriminator. The direction-aware spatial frequency discriminator extends the shared encoder using three parallel branches, including a spatial branch, a direction frequency branch, and an image patch branch. Spatial branch pairs shallow features extracted from the shared encoder Perform the operation to assess the structural authenticity: （11）； in, The discriminative features representing the output of the spatial branch, This indicates a batch normalization operation; The directional frequency branch aims to extract anisotropic features by decomposing spatial information along three orthogonal anatomical planes. For each orthogonal anatomical plane direction, the shallow features... First, the permutation is performed to create a series of 2D slices aligned with the selected orthogonal anatomical plane. Then, each 2D slice is separated into four sub-bands using a 2D discrete wavelet transform: low-frequency-low-frequency sub-bands. The corresponding image smoothing information, low-frequency to high-frequency sub-band Corresponding to horizontal edge details, high-frequency to low-frequency sub-bands Corresponding to vertical edge details; high-frequency to high-frequency sub-band Corresponding to texture details; only retaining the low-frequency subband. The three high-frequency component feature maps, excluding those from the original map, are extracted and reorganized through inverse permutation and reshaping to obtain a set of orientation-specific feature maps. This set of orientation-specific feature maps is then refined using a 3×3×3 convolutional layer, batch normalization, and ReLU activation. A SE module adaptively emphasizes the anatomically relevant frequency components to obtain a refined feature map. Finally, the refined feature maps corresponding to the three orthogonal anatomical planes are concatenated along the channel dimension and fused using a 1×1×1 convolutional layer to produce a unified frequency-aware feature representation. The three orthogonal anatomical planes include the axial, sagittal, and coronal planes. Image patch branching is based on high-level semantic representations obtained from the deepest layer of the shared encoder. To assess local realism: （12）； in, The discriminative features representing the output of image patch branches, This represents a 4×4×4 convolutional layer. This represents the activation function of a linear rectifier unit with leakage. To obtain the final discriminative output, a uniform interpolation operation is first used to align the spatial, frequency, and image patch branch features to the resolution. The resulting feature maps are then concatenated along the channel dimension and fused using a 1×1×1 convolution. Finally, a voxel-level discriminative map is generated using sigmoid activation. This allows for the determination of whether the input positron emission tomography (PET) image is a PET image generated by the generator or a real PET image. （13）； in, This indicates spatial alignment using trilinear interpolation.

9. The MRI-PET modality transfer method based on generative adversarial networks according to claim 1, characterized in that, The method for constructing the loss function during the training process is as follows: The generator loss is a weighted sum of the adversarial loss and the pixel-level supervision loss: （14）； in, This represents the total loss function of the generator. This represents the mean loss value for all samples in the batch. Represents a generator. This represents the input magnetic resonance imaging (MRI) image. This represents the actual positron emission tomography (PET) images of their pairing. It is a hyperparameter that controls the balance between adversarial learning and pixel-level supervision. For the discriminator to generate the image For probability estimation of the real image, For the generator's adversarial loss term, Generate pixel loss for L1 norm; The reconstruction pixel loss of the discriminator is defined as: （15）； in, This represents the loss function of the discriminator. This indicates the need to rebuild the network. This represents the reconstruction output of the reconstruction network from a real positron emission tomography (PET) image. Indicates the L1 norm reconstructed pixel loss; The adversarial loss of discriminator D is expressed as: （16）； in, This represents the adversarial loss of the discriminator D. This represents the probability estimate by which the discriminator identifies the input image as a real image. This represents two types of generated images. and These represent the mean discrimination loss values ​​for real samples and generated samples in the batch, respectively; The overall discriminator loss is defined as: （17）； in, This indicates the contribution weight of adjusting the reconstruction constraints; During training, the generator and discriminator are minimized separately. and Alternate updates are performed.

10. An MRI-PET modality conversion system based on generative adversarial networks, used to implement the MRI-PET modality conversion method based on generative adversarial networks as described in any one of claims 1-9, characterized in that, include: The dataset construction and partitioning module is used to acquire the modality conversion dataset and divide it into training, validation and test sets according to a set ratio; A generative adversarial network (GAN) building module is used to construct a GAN; the GAN is used to convert an input magnetic resonance imaging (MRI) image into a positron emission tomography (PET) image. The training module uses the training set and validation set to train the generative adversarial network, resulting in a trained generative adversarial network. The image generation module is used to acquire the magnetic resonance imaging (MRI) images to be converted and to generate positron emission tomography (PET) images using a generator in a trained generative adversarial network.

Images