Image synthesis method from MRI to brain blood volume quantitative image for brain tumor

Abstract

The application discloses an image synthesis method from MRI to brain blood volume quantitative image for brain tumor, comprising 1) a GAN-based image synthesis algorithm, which mainly comprises a GAN network based on feature consistency and a three-dimensional encoder-decoder network with mean absolute error loss; and 2) a 3D incremental Encoder-Decoder network, which mainly comprises an encoder and a decoder facing the CBV mode for each mode. Through the proposed GAN-based image synthesis algorithm, the application can generate a magnetic resonance imaging perfusion map from an easily accessible standard MRI sequence, thereby improving the implementability of the method for distinguishing tumor progression and radiation effects by improving the brain blood volume perfusion. Meanwhile, the 3D incremental Encoder-Decoder network of the application can learn from many subjects with only a few ASL-CBF paired images, thereby generating CBV images under the premise of only a small amount of ASL-CBF paired images.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to an image synthesis method for brain tumors, from MRI to quantitative images of cerebral blood volume. Background Technology

[0002] Perfusion magnetic resonance imaging (MRI), whether employing arterial spin labeling, dynamic contrast enhancement, or dynamic sensitivity contrast (DSC), is a crucial tool for assessing central nervous system pathology. Perfusion can be used in acute stroke settings to assess the volume of high-risk brain tissue and evaluate the likelihood of patient recovery. It can also quantify the physiological effects of vascular disturbances such as moyamoya disease, arterial regulatory failure, or arterial insufficiency. For new brain tumor patients, perfusion can help predict tumor grade and ensure biopsies are taken from the most aggressive portion of the tumor. Because it is difficult to distinguish tumor progression from the effects of therapeutic radiotherapy using conventional MRI sequences, a commonly used technique is to differentiate tumor progression and radiation effects using cerebral blood volume (CBV) from DSC MRI perfusion. This has become a routine method for imaging monitoring of treated high-grade gliomas.

[0003] However, this technique faces two challenges: 1) Perfusion imaging requires high-flow-rate contrast agent injection. High-flow-rate contrast agent injection is prone to errors because it is less adaptable to physiological changes (such as decreased cardiac output) and requires reliable image post-processing by experienced technicians. 2) Even minor patient movements can have a significant negative impact on post-processed images. Although MRI perfusion maps contribute significantly to diagnostic accuracy, they are cumbersome to operate and often fail to provide a definitive diagnosis. Therefore, a method is needed to generate MRI perfusion maps using conventional MRI sequences, thereby circumventing the aforementioned difficulties.

[0004] Arterial spin labeling (ASL) is a magnetic resonance perfusion imaging technique that quantifies cerebral blood flow (CBF) without the use of intravenously injected gadolinium contrast agents. Compared to enhancement techniques such as dynamic susceptibility contrast and dynamic contrast-enhanced perfusion, ASL is easily reproducible, less susceptible to susceptibility artifacts, and suitable for situations where gadolinium contrast agents are contraindicated (e.g., young children, pregnant women, and debilitated patients with liver or kidney failure). Post-processing of ASL images is relatively automated and does not depend on the intravenous contrast agent injection profile. Therefore, understanding the technical basis of ASL and perfusion physiology is crucial for identifying normal variations and artifacts. However, the number of existing CBV maps and paired ASL-CBF images is relatively small, and experiments with a small number of subjects may not yield convincing conclusions. Therefore, a method is needed that can learn from a large number of subjects with only a few paired ASL-CBF images to generate CBV images with limited ASL-CBF paired images. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides an image synthesis method for brain tumors, from MRI to quantitative cerebral blood volume (CBV) images. This method includes: 1) a GAN-based image synthesis algorithm, which mainly comprises a GAN network based on feature consistency and a three-dimensional encoder-decoder network with mean absolute error loss; and 2) a 3D incremental encoder-decoder network, which mainly includes an encoder for each modality and a decoder oriented towards the CBV modality. This invention, through its proposed GAN-based image synthesis algorithm, can generate MRI perfusion maps from readily available standard MRI sequences, thereby improving the feasibility of methods that use cerebral blood volume perfusion to distinguish tumor progression and radiation effects. Simultaneously, the 3D incremental encoder-decoder network of this invention can learn from many subjects with only a few ASL-CBF paired images, thus generating CBV images with only a small number of ASL-CBF paired images.

[0006] The technical solution adopted by this invention to solve its technical problem is as follows:

[0007] Step 1: Combining input sequences for MRI;

[0008] The original data for the five modes are T1WI, T2WI, ADC, T1_C, and T2_C. This means selecting n MRI sequences from all five types of MRI sequences, where n is 1 to 5; the selected n MRI sequences are placed into different channels to obtain an MRI combination, resulting in a total of 31 combinations, which are used as the input to the model.

[0009] Step 2: Build the generator module;

[0010] The generator module consists of an encoding section, a transmission section, and a decoding section. The encoding section uses three convolutional layers to extract information from the input image. The transmission section uses six residual network blocks to transmit information from the encoding section to the decoding section. The decoding section uses two deconvolutional layers and one convolutional layer to construct the target image. The kernel size of all convolutional and deconvolutional layers is set to 3×3×1.

[0011] Step 3: Construct the discriminator module;

[0012] The discriminator module consists of 5 convolutional layers; the input to the discriminator model is a synthetic CBV image or a real CBV image, and the output structure is a binary index that indicates whether the input image is synthetic or real.

[0013] Step 4: Build the MAEED module;

[0014] The backbone of the MAEED module is the generator module in the GAN network. The processed MRI sequence is fed into the MAEED module to obtain the generated CBV image. Then, the average absolute error loss is calculated between the generated CBV image and the real CBV image, thereby constraining the network to generate CBV images that are closer to the real CBV image.

[0015] Step 5: Construct a 3D incremental Encoder-Decoder network;

[0016] For each mode, there exists an encoder. There is a decoder for the CBV mode. The latent spatial feature maps of multiple encoders are averaged into a single mixed feature map; Let A represent a subject with M modalities, where A = {a1, a2, ..., a...} M} indicates whether data for a certain modality is missing, a * =1 indicates that data of this modality exists, a * =0 indicates none; the generated CBV image is represented as:

[0017]

[0018] Step 6: Based on the 3D incremental Encoder-Decoder Network (IEDN), train a model with ASL-CBF using the same subjects, and set a ASL =0 Simulates a model without ASL-CBF;

[0019] Replace CBV images with T1WI images / brain masks and retrain IEDN to obtain a T1WI image / brain mask synthesis model, which can synthesize T1WI images / brain masks from any other mode; co-registration between multiple modes will be transformed into same-mode T1WI registration.

[0020] Preferably, the number of channels in the three convolutional layers of the encoding part in the generator module are 16, 32 and 64, respectively.

[0021] Preferably, the decoding part of the generator module uses two deconvolutional layers with 16 and 32 channels respectively, and one convolutional layer with 1 channel.

[0022] Preferably, the five convolutional layers of the discriminator module have 16, 32, 64, 128 and 1 channels, respectively, and the kernel size of the convolutional layers is 4×4×1.

[0023] The beneficial effects of this invention are as follows:

[0024] The high-flow contrast agent required for perfusion imaging is limited by physiological changes such as decreased cardiac output and the need for reliable image post-processing by experienced technicians, resulting in a high error rate in high-flow contrast agent injection. Furthermore, even minor patient movements can have a significant negative impact on post-processed images. These factors contribute to the difficulty in obtaining MRI perfusion maps from perfusion imaging and the potential for a high error rate. This invention proposes a GAN-based image synthesis algorithm that can generate MRI perfusion maps from readily available standard MRI sequences, thereby improving the feasibility of a method to differentiate tumor progression and radiation effects based on perfused cerebral blood volume. Simultaneously, the 3D incremental Encoder-Decoder network of this invention can learn from many subjects with only a few ASL-CBF paired images, thus generating CBV images even with limited ASL-CBF paired images. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the image synthesis algorithm based on GAN of this invention.

[0026] Figure 2 This is a schematic diagram of the 3D incremental Encoder-Decoder network of the present invention. Detailed Implementation

[0027] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0028] This invention proposes a GAN-based image synthesis algorithm and a 3D incremental encoder-decoder network. The goal is for the GAN-based algorithm to generate quantitative CBV images from standard MRI scans and, with reference to SCALE-PWI, also to generate quantitative CBV images. Simultaneously, the 3D incremental encoder-decoder network aims to synthesize CBV images with only a limited number of ASL-CBF paired images. Specifically, this invention comprises two main parts: 1) a GAN-based image synthesis algorithm, which mainly includes a feature consistency-based GAN network and a 3D encoder-decoder network with mean absolute error loss; 2) a 3D incremental encoder-decoder network, which mainly includes an encoder for each modality and a decoder oriented towards the CBV modality.

[0029] Example:

[0030] 1. Input sequence combination for MRI sequences;

[0031] like Figure 1 As shown on the left, there are initially five modal data: T1WI, T2WI, ADC, T1_C, and T2_C. This indicates the selection of n MRI sequences from all five types of MRI sequences, where n is between 1 and 5. The selected n MRI sequences are then placed into different channels to obtain an MRI combination, resulting in a total of 31 combinations, which are used as input to the model.

[0032] 2. Generator module;

[0033] Generator modules such as Figure 1 As shown in the Generator section at the top right, it consists of encoding, transmission, and decoding parts. The encoding part uses three convolutional layers (16, 32, and 64 channels) to extract information from the input image. The transmission part uses six residual network blocks to transmit information from the encoding part to the decoding part. The decoding part uses two deconvolutional layers (16 and 32 channels) and one convolutional layer (one channel) to construct the target image. All convolutional and deconvolutional layers have a kernel size of 3×3×1, which allows for better utilization of data with large cross-sectional thickness.

[0034] 3. Discriminator module;

[0035] Discriminator module such as Figure 1 As shown in the Discriminator at the top right, this discriminative model consists of five convolutional layers with channels of 16, 32, 64, 128, and 1 respectively, and a kernel size of 4×4×1. The input to the discriminative model is either a synthetic CBV image or a real CBV image, while the output structure is a binary index indicating whether the input image is synthetic or real.

[0036] 4. MAEED module;

[0037] MAEED module, such as Figure 1 As shown in the lower right corner, its backbone is the generator module in the GAN network. The processed MRI sequence is fed into the MAEED module to obtain the generated CBV image. Then, the mean absolute error loss is calculated between the generated CBV image and the real CBV image, thus constraining the network to generate CBV images that are closer to the real CBV image. The mean absolute error loss is a low-level constraint that encourages a pair of synthesized and real CBV images to remain similar in each voxel, while the feature consistency loss is a multi-level constraint from coarse to fine to ensure that the features of a pair of synthesized and real CBV images are similar as they pass through each layer of the discriminative model.

[0038] 5. 3D incremental Encoder-Decoder network;

[0039] The structure diagram of a 3D incremental Encoder-Decoder network is shown below. Figure 2As shown, there is an encoder for each mode. There is a decoder for the CBV mode. The latent spatial feature maps of these multiple encoders are averaged into a single mixed feature map, which allows the structure to remain unaffected by the absence of a particular modality. Let A represent a subject with M modalities, where A = {a1, a2, ..., a...} M} indicates whether data for a certain modality is missing, a * =1 indicates that data of this modality exists, a * =0 indicates none. The generated CBV image can be represented as:

[0040]

[0041] Based on a 3D incremental Encoder-Decoder network, a model with ASL-CBF can be trained using the same subjects as in previous studies, and a... ASL =0 is used to simulate a model without ASL-CBF. Furthermore, previous studies involved some semi-manual image processing steps, making it unsuitable for routine clinical applications, including removing skull and skin tissue from images and performing 3D co-registration while the patient moves between conventional MRI sequence acquisitions. In this invention, a 3D incremental Encoder-Decoder network also overcomes this potential obstacle. Simply replace the CBV image with a T1WI image / brain mask and retrain the IEDN to obtain a T1WI image / brain mask synthesis model, thus enabling the synthesis of T1WI images / brain masks from any other modality. The brain mask removes skull and skin tissue, and co-registration between multiple modalities is transformed into easier intramodal (T1WI) registration.

Claims

1. An image synthesis method for brain tumors, from MRI to quantitative images of cerebral blood volume, characterized in that, Includes the following steps: Step 1: Combining input sequences for MRI; The original data for the five modes are T1WI, T2WI, ADC, T1_C, and T2_C. This means selecting n MRI sequences from all five types of MRI sequences, where n is between 1 and 5; the selected n MRI sequences are placed into different channels to obtain an MRI combination, resulting in a total of 31 combinations, which are used as input to the model. Step 2: Build the generator module; The generator module consists of an encoding section, a transmission section, and a decoding section. The encoding section uses three convolutional layers to extract information from the input image. The transmission section uses six residual network blocks to transmit information from the encoding section to the decoding section. The decoding section uses two deconvolutional layers and one convolutional layer to construct the target image. The kernel size of all convolutional and deconvolutional layers is set to 3×3×1. Step 3: Construct the discriminator module; The discriminator module consists of 5 convolutional layers; the input to the discriminator model is a synthetic CBV image or a real CBV image, and the output structure is a binary index that indicates whether the input image is synthetic or real. Step 4: Build the MAEED module; The backbone of the MAEED module is the generator module in the GAN network. The processed MRI sequence is fed into the MAEED module to obtain the generated CBV image. Then, the average absolute error loss is calculated between the generated CBV image and the real CBV image, thereby constraining the network to generate CBV images that are closer to the real CBV image. Step 5: Construct a 3D incremental Encoder-Decoder network; For each mode, there exists an encoder. There is a decoder for the CBV mode. The latent spatial feature maps of multiple encoders are averaged into a single mixed feature map; Let A represent a subject with M modalities, where A = {a1, a2, ..., a...} M } indicates whether data for a certain modality is missing, a * =1 indicates that data of this modality exists, a * =0 indicates none; the generated CBV image is represented as: Step 6: Based on the 3D incremental Encoder-Decoder Network (IEDN), train a model with ASL-CBF using the same subjects, and set a ASL =0 Simulates a model without ASL-CBF; Replace CBV images with T1WI images / brain masks and retrain IEDN to obtain a T1WI image / brain mask synthesis model, which can synthesize T1WI images / brain masks from any other mode; co-registration between multiple modes will be transformed into same-mode T1WI registration.

2. The image synthesis method for brain tumors from MRI to quantitative images of cerebral blood volume according to claim 1, characterized in that, The number of channels in the three convolutional layers of the encoding part in the generator module are 16, 32 and 64, respectively.

3. The image synthesis method for brain tumors from MRI to quantitative images of cerebral blood volume according to claim 1, characterized in that, The decoding part of the generator module uses two deconvolutional layers with 16 and 32 channels respectively, and one convolutional layer with 1 channel.

4. The image synthesis method for brain tumors from MRI to quantitative images of cerebral blood volume according to claim 1, characterized in that, The discriminator module has five convolutional layers with 16, 32, 64, 128, and 1 channels, respectively, and the kernel size is 4×4×1.

Images