An r2* image synthesis method based on a generative adversarial network

Abstract

The application provides an R2* image synthesis method based on a generative adversarial network, comprising the following steps: S1, data preprocessing: calculating and correcting R2* values from MEGRE sequences; S2, ROI segmentation: after matching the R2* graph with the AALv3 template, the average R2* values of the regions of interest are extracted from the synthesis graph and the real graph; S3, GAN model: the generator inputs the T1 weighted image and the T2 weighted image, and generates the corresponding R2* image; the discriminator distinguishes the synthesized R2* image generated by the generator from the real R2* image obtained from the real MEGRE sequence; S4, quantitative evaluation of the generated image: the normalized mean square error, the peak signal-to-noise ratio and the structural similarity index are used to evaluate the similarity between the synthesized image and the real R2* graph; S5, statistical analysis: the R2* values of the regions of interest in the synthesized graph and the real R2* graph are analyzed and compared. The application realizes the auxiliary diagnosis of neurodegenerative diseases similar to Parkinson's disease from the perspective of medical image synthesis and processing.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image synthesis, and in particular to an R2* image synthesis method based on a generative adversarial network. Background Art

[0002] Parkinson's disease (PD) is a chronic neurodegenerative disorder characterized by the progressive loss of dopaminergic neurons in the substantia nigra pars compacta (SNpc). Both postmortem histological analysis and in vivo magnetic resonance imaging studies have demonstrated elevated iron levels in the nigrostriatal dopaminergic system, particularly within the substantia nigra, in PD patients.

[0003] Because this iron deposition leads to local magnetic field inhomogeneities, it affects spin-spin interactions and ultimately accelerates the transverse relaxation of magnetic resonance imaging (MRI) signals. Therefore, MRI can indirectly estimate iron content by observing changes in the T2* relaxation time or its reciprocal, the transverse relaxation rate constant (R2* = 1 / T2*).

[0004] Traditional methods for acquiring R2* images, including acquisition and image reconstruction using a multi-gradient echo (MEGRE) sequence, are time-consuming and computationally intensive. Furthermore, MEGRE sequences are sensitive to both susceptibility and motion artifacts. Macroscopic B0 magnetic field inhomogeneities can adversely affect the quantitative measurement of relaxation parameters in MEGRE sequences. During MEGRE signal acquisition, the effects of macroscopic magnetic field inhomogeneities on unconscious body movements and subtle anatomical fluctuations can also introduce motion artifacts into these quantitative images. Consequently, despite their significant clinical value, R2* imaging cannot be incorporated into standard clinical protocols. Summary of the Invention

[0005] The technical problem to be solved by the present invention is how to quickly and accurately generate R2* images from conventional nuclear magnetic resonance sequences.

[0006] In order to solve the above technical problems, the present invention provides an R2* image synthesis method based on a generative adversarial network, the method comprising the following steps:

[0007] S1. Data preprocessing: Calculate and correct the transverse relaxation rate constant R2* from the MEGRE sequence using a single exponential decay model;

[0008] S2. ROI segmentation: registering the first echo image generated by the MEGRE sequence to the AALv3 template; matching the R2* image with the AALv3 template to obtain the region of interest;

[0009] S3, a GAN model: a GAN model is designed, which includes a generator and a discriminator, the generator inputs T1 weighted images and T2 weighted images to generate corresponding R2* images; the discriminator distinguishes between synthetic R2* images generated by the generator and real R2* images obtained by the real MEGRE sequence;

[0010] S4, quantitative evaluation of generated images: normalized mean square error, peak signal-to-noise ratio and structural similarity index are used to evaluate the similarity between synthetic images and real R2* images;

[0011] S5, statistical analysis: the average R2* value of the region of interest is extracted from the synthetic image and the real R2* image; the normality of the evaluation data is evaluated by Kolmogorov-Smirnov test; and the Pearson correlation coefficient and Bland-Altman diagram are calculated to compare the R2* values of the region of interest in the synthetic image and the real R2* image.

[0012] Further, the R2* value calculated from the MEGRE sequence by using the single exponential decay model satisfies the relationship:

[0013]

[0014] wherein, is the frequency shift, and depends on the tissue structure and the macroscopic magnetic field mainly generated by the tissue / air interface; is the voxel spread function; is the echo time; is the phase part of the signal.

[0015] Further, the S1 further comprises using the FLIRT tool to register the T1 weighted image and the T2 weighted image to the mode map corresponding to the first echo of the MEGRE sequence, so that different sequences have spatial consistency.

[0016] Further, the S1 further comprises obtaining data, wherein:

[0017] The database data is listed as external data to prove the generalization of the GAN model;

[0018] The collected data is listed as internal data, and the internal data is divided into a training set and a test set; wherein 80% of the internal data is used for the training set, and 20% of the internal data is used for the test set.

[0019] Further, the region of interest in S2 includes: the reticulatum of the substantia nigra, the compact part of the substantia nigra, the putamen, the globus pallidus and the caudate nucleus.

[0020] Further, the R2* image is synthesized using a pix2pix image-to-image conversion framework in the S3, and the network architecture of the generator and the discriminator is the same as that of the pix2pix image-to-image conversion framework.

[0021] Further, the loss function of the generator is a combination of an adversarial loss, a pixel-level loss, and a perception loss:

[0022] The adversarial loss satisfies the following relationship:

[0023]

[0024] wherein, represents an original image; represents an R2* image generated by the MEGRE sequence; represents the generator; represents the discriminator; represents synthesized data;

[0025] The pixel-level loss is a combination of an L1 weighted reconstruction loss and a multi-scale structural similarity index to represent the difference between the generated image and the expected target image; wherein the multi-scale structural similarity index is defined as:

[0026]

[0027] wherein, is an index for measuring structural similarity at multiple scales, which can incorporate image details at different resolutions;

[0028] The pixel-level loss is defined as:

[0029]

[0030] wherein,

[0031] The perception loss uses a pre-trained VGG-16 model to increase the perception loss, and the perception loss is defined as:

[0032]

[0033] wherein, represents the number of convolution layers of the VGG-16 model used to calculate the perception loss; represents the feature representation of the i-th convolution layer; represents the number of channels of the i-th feature layer.

[0034] Further, the loss function of the generator satisfies the following relationship:​​​

[0035]

[0036] in, represents the generator; represents the discriminator; represents the adversarial loss; represents said perceived loss; Represents pixel-level loss.

[0037] Furthermore, the training details of the GAN model in S3 include:

[0038] Weight updates were performed using minibatch stochastic gradient descent and the Adam optimizer. The batch size of the minibatch stochastic gradient descent was 16. The learning rate of the Adam optimizer was 0.0002, and the momentum parameters b1 = 0.5 and b2 = 0.999.

[0039] Data augmentation techniques were used to address the data imbalance between the PD patient group and the healthy control group; wherein the data augmentation techniques included random cropping and horizontal flipping.

[0040] Furthermore, in said S4:

[0041] The normalized mean square error is used to evaluate the degree of difference between the synthetic image and the real R2* image at the pixel level;

[0042] The peak signal-to-noise ratio is defined as the ratio between the maximum possible power of the signal and the noise level in the image;

[0043] The structural similarity index measures the similarity between two images from three aspects: brightness, contrast and structure;

[0044] in:

[0045] The higher the peak signal-to-noise ratio and the structural similarity index, the better the reconstruction effect; the lower the value of the normalized mean square error, the higher the reconstruction fidelity.

[0046] Furthermore, the S5 further includes:

[0047] The two-sample t-test was used to compare the difference in R2* values ​​between the PD patient group and the healthy control group;

[0048] The R2* values ​​of the SNpc of the synthetic image and the real R2* image were used to perform area under the receiver operating characteristic curve analysis to distinguish the PD patient group from the healthy control group;

[0049] In the external data, AUC analysis was performed using the synthesized R2 plot to distinguish the PD patient group from the healthy control group.

[0050] Furthermore, the generator input further includes an arterial spin labeling image and / or a diffusion weighted image to generate a more accurate R2* image.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] Based on an artificial intelligence model called a generative adversarial network, this invention generates a synthetic transverse relaxation rate constant image, or synthetic R2* image, from T1-weighted and T2-weighted images. The generated image correlates well with the actual R2* image. This approach provides powerful auxiliary diagnosis for neurodegenerative diseases like Parkinson's disease from the perspective of medical image synthesis and processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Schematic diagram of synthesizing an R2* image from a T1-weighted image and a T2-weighted image disclosed in the present invention;

[0054] Figure 2 This is an image of an embodiment disclosed in the present invention. DETAILED DESCRIPTION

[0055] In order to make the technical scheme and technical effects of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0056] Glossary:

[0057] T1-weighted images (T1W) and T2-weighted images (T2W) are two basic sequences in magnetic resonance imaging (MRI).

[0058] T1-weighted images are primarily used to display normal brain tissue structures. On T1-weighted images, long T1 signals are typically darker, while fat or hemorrhage may appear as high signals at certain times. This sequence is particularly effective for observing anatomical structures because short T1 substances such as fat and protein appear as high signals (brighter) on the image, while water and other long T1 substances appear as low signals (darker).

[0059] T2-weighted images are primarily used to demonstrate certain lesions, particularly those that appear as high signal intensity on T2-weighted images, such as cerebral infarction, white matter demyelination, and multiple sclerosis. On T2-weighted images, cerebrospinal fluid in the ventricles and sulci appears as a relatively high signal, or whitish. This sequence is particularly sensitive for detecting tissues or lesions with a high water content.

[0060] The present invention aims to provide an R2* image synthesis method based on a generative adversarial network, which can generate a synthetic transverse relaxation rate constant image, namely a synthetic R2* image, from T1-weighted images and T2-weighted images, providing new ideas for auxiliary diagnosis of neurodegenerative diseases such as Parkinson's disease (PD).

[0061] The method provided by the present invention mainly comprises the following steps:

[0062] 1. Data Collection

[0063] 1.1. Research sample collection

[0064] PD patients and age-matched healthy subjects who underwent MRI examinations in the hospital were screened, and the inclusion criteria of the PD patient group and the healthy control group were defined accordingly.

[0065] 1.2 MRI imaging parameters

[0066] Internal MRI studies were performed on a 3T scanner (Discovery 750w, GE Medical Systems, USA) using a standard 24-channel head coil.

[0067] T1-weighted images (T1W) were acquired using 3D T1 BRAVO with the following parameters: matrix size = 256 × 256; field of view (FOV) = 25.6 cm; repetition time (TR) = 8.5 ms; inversion time (TI) = 450 ms; slice thickness = 1.0 mm; echo time (TE) = 3.5 ms; and flip angle = 12°.

[0068] The parameters of T2-weighted images (T2W) were as follows: matrix size = 512 × 512; field of view = 24 cm; repetition time = 5869 ms; slice thickness = 5 mm; echo time = 116.2 ms; and flip angle = 142°.

[0069] The MEGRE parameters were as follows: matrix size = 256 × 256; field of view = 25.6 cm; repetition time = 31 ms; slice thickness = 1.0 mm; number of echoes = 12; time to first echo (TE_1) = 1.9 ms; echo interval (ΔTE) = 2.3 ms; and flip angle = 12°.

[0070] 1.3 Data Classification

[0071] The database data is listed as external data to prove the generalization of the GAN model; the external data comes from the Parkinson's Progression Markers Initiative (PPMI) database.

[0072] The collected data is listed as internal data, and the internal data is divided into a training set and a test set; 80% of the internal data is used for the training set and 20% of the internal data is used for the test set.

[0073] All MRI images were center-cropped and resized to 256 × 256.

[0074] 2. Data Preprocessing

[0075] 2.1. Calculate the R2* value from the MEGRE sequence using the single exponential decay model, which satisfies the relationship:

[0076]

[0077] in, is a frequency shift and depends on the tissue structure and the macroscopic magnetic field generated primarily by the tissue / air interface; is the voxel expansion function; is the echo time; is the phase portion of the signal.

[0078] 2.2. Use the FLIRT tool in the FMRIB Software Library (FSL) to align the T1-weighted image and the T2-weighted image to the corresponding first echo of the MEGRE sequence to ensure spatial consistency between different sequences.

[0079] 2.3. Use data augmentation techniques to address the data imbalance between the PD patient group and the healthy control group; data augmentation techniques include random cropping and horizontal flipping to increase data diversity when training neural networks.

[0080] Random cropping: This technique increases the diversity of the training dataset by randomly cropping sub-images of different sizes and positions from the original image. This technique can effectively expand the size of the training dataset, prevent model overfitting, and improve the model's generalization ability.

[0081] Horizontal flip: In horizontal flip, the image is flipped along its width, that is, from left to right. This technique can simulate the various situations that images may appear in the real world, thereby helping the model learn more general features.

[0082] 3. ROI Segmentation

[0083] The first echo image generated by the MEGRE sequence was registered to the AALv3 template using the affine registration method with 12 degrees of freedom and the FMRIB linear image registration tool (FLIRT). Then, the linear transformation was optimized by the nonlinear warping implemented by the FMRIB nonlinear image registration tool (FNIRT), which included the bias field correction.

[0084] After matching the R2* image with the AALv3 template, the regions of interest were obtained. The regions of interest included the substantia nigra pars reticulata (SNpr), the substantia nigra pars compacta (SNpc), the putamen (PUT), the globus pallidus (GP), and the caudate nucleus (CAU).

[0085] Four, GAN model

[0086] Figure 1 The process of synthesizing the R2* image from the T1-weighted image and the T2-weighted image is shown. The pix2pix image-to-image conversion framework is used to synthesize the R2* image. The GAN model includes a generator and a discriminator.

[0087] The generator inputs the T1-weighted image and the T2-weighted image to generate the corresponding R2* image. The task of the generator is to create very realistic images so that the discriminator cannot distinguish them from the actual images.

[0088] The discriminator distinguishes between the synthesized R2* image generated by the generator and the real R2* image obtained from the real MEGRE sequence. The task of the discriminator is to evaluate the authenticity of the input image.

[0089] The network architecture of the generator and the discriminator is the same as that of the pix2pix image-to-image conversion framework. The difference is that the loss function of the generator is a combination of the adversarial loss, the pixel-level loss, and the perceptual loss, which is specifically:

[0090] The adversarial loss satisfies the following relationship:

[0091]

[0092] wherein, represents the original image; represents the R2* image generated by the MEGRE sequence; represents the generator; represents the discriminator; represents the synthesized data.

[0093] The pixel-level loss is a combination of the L1 weighted reconstruction loss and the multi-scale structural similarity index to represent the difference between the generated image and the expected target image; wherein the multi-scale structural similarity index is defined as:

[0094]

[0095] in, It is an indicator that measures structural similarity at multiple scales and can incorporate image details at different resolutions;

[0096] Then the pixel-level loss is defined as:

[0097]

[0098] in, .

[0099] The pre-trained VGG-16 model is used to increase the perceptual loss, which is defined as:

[0100]

[0101] in, Represents the number of convolutional layers of the VGG-16 model used to calculate the perceptual loss; Indicates the Feature representation of convolutional layers; Indicates the The number of channels in each feature layer.

[0102] In summary, the generator's loss function satisfies the following relationship:

[0103]

[0104] in, Represents a generator; represents the discriminator; represents adversarial loss; Indicates perceived loss; Represents pixel-level loss.

[0105] 5. Training Details

[0106] The weights are updated using minibatch stochastic gradient descent and the Adam optimizer. The batch size of the minibatch stochastic gradient descent is 16. The learning rate of the Adam optimizer is 0.0002, and the momentum parameters b1 = 0.5 and b2 = 0.999.

[0107] Data augmentation includes random crops and horizontal (left-to-right) flipping.

[0108] Furthermore, a generative adversarial network (GAN) model was trained for 70 epochs using the Ubuntu 18.04 (Canonical) operating system and the PyTorch deep learning framework, with CUDA 11.3 (NVIDIA Corporation) for graphics processing unit acceleration. Model building was performed on a workstation equipped with two NVIDIA Pascal-based GeForce GTX 2080Ti graphics processing units.

[0109] 6. Quantitative Evaluation of Generated Images

[0110] Normalized mean square error (NMSE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM) are used to evaluate the similarity between the synthetic image and the real R2* map. Specifically:

[0111] The normalized mean square error is used to evaluate the degree of difference at the pixel level between the synthesized image and the true R2* image.

[0112] The peak signal-to-noise ratio is defined as the ratio between the maximum possible power of the signal and the noise level in the image.

[0113] The structural similarity index measures the similarity between two images in terms of brightness, contrast and structure.

[0114] Among them: Reference Figure 2 For the example images shown, higher peak signal-to-noise ratio and structural similarity index indicate better reconstruction results; lower normalized mean square error values ​​indicate higher reconstruction fidelity.

[0115] 7. Statistical Analysis

[0116] The average R2* value of the region of interest was extracted from the synthetic images and the real R2* images, and the normality of the data was assessed using the Kolmogorov-Smirnov test.

[0117] Pearson correlation coefficients and Bland-Altman plots were calculated to compare the R2* values ​​of the regions of interest in the synthesized images with those in the true R2* images.

[0118] The two-sample t-test was used to compare the differences in R2* values ​​between the PD patient group and the healthy control group.

[0119] The R2* values ​​of SNpc of the synthetic and real R2* images were used to perform area under the receiver operating characteristic curve analysis to distinguish the PD patient group from the healthy control group.

[0120] In the external data, AUC analysis was performed using the synthesized R2* plot to distinguish the PD patient group from the healthy control group.

[0121] The Spearman correlation test was used to test the correlation between ΔR2* and the percentage change in worsening of the Unified Parkinson's Disease Rating Scale, Part III (UPDRSIII) (ΔUPDRS, %). The 95% confidence interval (CI) was calculated using the bootstrap method (bootstrap sample size = 1000). ΔR2* was defined as the difference between R2*(t1) – R2*(t0); t0 was the baseline MRI scan; t1 was the longitudinal scan three years later. A P < 0.05 was considered statistically significant.

[0122] It should be noted that, in addition to T1-weighted and T2-weighted images, the present invention can also use other medical image modalities (such as proton density-weighted images and diffusion-weighted images) as input to generate more accurate R2* images. This multimodal image fusion technology can expand the applicability of the model and improve the quality and accuracy of the generated images.

[0123] In addition, the method provided by the present invention is also applicable to the R2* image generation network model of other neurodegenerative diseases, such as Alzheimer's disease.

[0124] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An R2* image synthesis method based on generative adversarial networks, characterized in that: The method comprises the following steps: S1. Data preprocessing: Calculate and correct the transverse relaxation rate constant R2* from the MEGRE sequence using a single exponential decay model; S2. ROI segmentation: registering the first echo image generated by the MEGRE sequence to the AALv3 template; matching the R2* image with the AALv3 template to obtain the region of interest; S3. GAN model: Design and generate a GAN model, the GAN model including a generator and a discriminator, the generator inputting a T1-weighted image and a T2-weighted image to generate corresponding R2* images; the discriminator distinguishing between the synthetic R2* images generated by the generator and the real R2* images obtained from the real MEGRE sequence; S4. Quantitative evaluation of generated images: Normalized mean square error, peak signal-to-noise ratio, and structural similarity index are used to evaluate the similarity between the synthesized images and the real R2* images; S5. Statistical analysis: The average R2* value of the region of interest was extracted from the synthetic image and the real R2* image; the normality of the data was assessed using the Kolmogorov-Smirnov test; and the Pearson correlation coefficient and Bland-Altman plot were calculated to compare the R2* values ​​of the region of interest in the synthetic image and the real R2* image.

2. The R2* image synthesis method based on generative adversarial network according to claim 1, characterized in that The R2* value is calculated from the MEGRE sequence using a single exponential decay model, which satisfies the relationship: in, is a frequency shift and depends on the tissue structure and the macroscopic magnetic field generated primarily by the tissue / air interface; is the voxel expansion function; is the echo time; is the phase portion of the signal.

3. The R2* image synthesis method based on generative adversarial network according to claim 1, characterized in that: The S1 further includes registering the T1-weighted image and the T2-weighted image to a model of the first echo corresponding to the MEGRE sequence using a FLIRT tool, so that different sequences have spatial consistency.

4. The R2* image synthesis method based on generative adversarial network according to claim 1, characterized in that The S1 also includes acquiring data, wherein: The database data is listed as external data to prove the generalization of the GAN model; The collected data are listed as internal data, and the internal data are divided into a training set and a test set; wherein, 80% of the internal data are used for the training set, and 20% of the internal data are used for the test set.

5. The R2* image synthesis method based on generative adversarial network according to claim 1, characterized in that: The regions of interest in S2 include the substantia nigra pars reticulata, substantia nigra pars compacta, putamen, globus pallidus, and caudate nucleus.

6. The R2* image synthesis method based on generative adversarial network according to claim 1, characterized in that In S3, the R2* image is synthesized using a pix2pix image-to-image conversion framework, and the network architecture of the generator and the discriminator is the same as the network architecture of the pix2pix image-to-image conversion framework.

7. The R2* image synthesis method based on generative adversarial network according to claim 1 or 6, characterized in that: The loss function of the generator is a combination of adversarial loss, pixel-level loss and perceptual loss: The adversarial loss satisfies the following relationship: in, represents the original image; represents the R2* image generated by the MEGRE sequence; represents the generator; represents the discriminator; represents synthetic data; The pixel-level loss is a combination of an L1-weighted reconstruction loss and a multi-scale structural similarity index to represent the difference between the generated image and the desired target image; wherein the multi-scale structural similarity index is defined as: in, It is an indicator that measures structural similarity at multiple scales and can incorporate image details at different resolutions; Then the pixel-level loss is defined as: in, ; The perceptual loss is added using a pre-trained VGG-16 model, and the perceptual loss is defined as: in, Represents the number of convolutional layers of the VGG-16 model used to calculate the perceptual loss; Indicates the Feature representation of convolutional layers; Indicates the The number of channels in the feature layer.

8. The R2* image synthesis method based on generative adversarial network according to claim 7, characterized in that: The loss function of the generator satisfies the following relationship: in, represents the generator; represents the discriminator; represents the adversarial loss; represents said perceived loss; Represents pixel-level loss.

9. The R2* image synthesis method based on generative adversarial network according to claim 4, characterized in that: The training details of the GAN model in S3 include: Weights were updated using minibatch stochastic gradient descent and the Adam optimizer. The batch size of the minibatch stochastic gradient descent was 16. The learning rate of the Adam optimizer was 0.0002, and the momentum parameters b1 = 0.5 and b2 = 0.

999. Data augmentation techniques were used to address the data imbalance between the PD patient group and the healthy control group; wherein the data augmentation techniques included random cropping and horizontal flipping.

10. The R2* image synthesis method based on generative adversarial network according to claim 1, characterized in that: In said S4: The normalized mean square error is used to evaluate the degree of difference between the synthetic image and the real R2* image at the pixel level; The peak signal-to-noise ratio is defined as the ratio between the maximum possible power of the signal and the noise level in the image; The structural similarity index measures the similarity between two images from three aspects: brightness, contrast and structure; in: The higher the peak signal-to-noise ratio and the structural similarity index, the better the reconstruction effect; the lower the value of the normalized mean square error, the higher the reconstruction fidelity.

11. The R2* image synthesis method based on generative adversarial network according to claim 9, characterized in that: The S5 further includes: The two-sample t-test was used to compare the difference in R2* values ​​between the PD patient group and the healthy control group; The R2* values ​​of the SNpc of the synthetic image and the real R2* image were used to perform area under the receiver operating characteristic curve analysis to distinguish the PD patient group from the healthy control group; In the external data, AUC analysis was performed using the synthesized R2* plot to distinguish the PD patient group from the healthy control group.

12. The R2* image synthesis method based on generative adversarial network according to claim 1, characterized in that: The generator input also includes arterial spin labeling images and / or diffusion weighted images to generate more accurate R2* images.

Images