A deep learning-based brain MR image normalization system

Through the brain MR image standardization system based on deep learning, the problem of inconsistency in medical MR images under multi-center, different devices and different imaging parameters is solved, and the image stability and resolution are improved, and the accuracy of the imaging microscopy model is significantly improved.

CN114387220BActive Publication Date: 2025-03-28FUDAN UNIVERSITY
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202111562916.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-20
Publication Date
2025-03-28
Estimated Expiration
2041-12-20

AI Technical Summary

Technical Problem

The prior art is difficult to effectively standardize medical MR images under multi-center, different equipment and different imaging parameters, resulting in inconsistent image quality and affecting the stability and accuracy of imaging omics models.

Method used

A brain MR image standardization system based on deep learning is adopted, including a modal standardization module, a grayscale standardization module and a layer spacing standardization module. The modal normalization is achieved through a fully connected neural network with multiple inputs and multiple outputs, the grayscale normalization is achieved using the CycleGAN network, and the layer spacing is normalized through SPM, and the normalized MR image is finally synthesized.

Benefits of technology

Effective normalization of uneven MR images was achieved, the stability and resolution of the image were improved, and the stability and accuracy of the imaging omics model were enhanced. Specifically, the Dice coefficient of glioma segmentation was increased by 9%, the AUC of pathological grading was increased by 32%, and the IDH1 prediction was increased by 25%.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114387220B_ABST
    Figure CN114387220B_ABST
Patent Text Reader

Abstract

The present invention relates to a brain MR image normalization system based on deep learning. First, an encoder-decoder network is used to realize modality normalization, so that the feature extractor has more useful features; secondly, grayscale normalization is realized by using a cyclic generative adversarial network, and the basic eigenvalues ​​are limited to a specific interval, which reduces the sensitivity of the model to data; finally, statistical parameter mapping (SPM) is used to achieve uniform inter-layer spacing, improve the isotropy and resolution of the input image, and import more stable and complete three-dimensional features into the model to obtain better results. Compared with the prior art, the present invention can maximize the use of existing multi-center data and improve the effectiveness and robustness of imaging omics diagnosis and treatment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer information processing, and in particular to a brain MR image normalization system based on deep learning. Background Art

[0002] In the era of big data, radiomics methods rely on a large number of medical images to build intelligent diagnostic models. Image quality is crucial to the stability and clinical usability of the model. High-quality medical images are complete, consistent, and high-resolution, enabling the model to extract more stable and useful information. Since radiomics models require a large amount of data, data needs to be collected in multiple centers, with different equipment, and with different imaging parameters. Taking MRI as an example, there are three main differences between images: missing modality (M), uneven grayscale distribution (I), and inconsistent inter-layer spacing (L).

[0003] Modality loss is the most common difference in medical images, and the lack of modality will directly lead to modeling failure. Current research mainly focuses on modality loss reconstruction based on deep learning, and the networks used can be basically divided into convolutional neural networks (CNNs), U-Net, and generative adversarial networks (GANs). These networks achieve modality-to-modality mapping through an end-to-end structure and introduce a loss function to reduce the difference between the real image and the generated result. Regarding the impact of modality normalization on radiomics, Bakas et al. explored the impact of modality normalization on tumor segmentation. He believed that modality normalization can combine information from multiple modalities to improve segmentation accuracy. Julip et al. aimed to predict the aggressiveness of prostate cancer and studied the effectiveness of multimodal radiomics features in classifying low Gleason score (GS) groups and high GS score groups. They concluded that the classification task after modality normalization had higher accuracy (ACC), sensitivity (SEN), and negative predictive value (NPV).

[0004] Uneven grayscale distribution is the most obvious type of difference. The difference in image features caused by uneven grayscale may exceed the difference caused by the pathological characteristics themselves. Therefore, in multi-center validation, differences in grayscale distribution can lead to instability in the imaging omics model. Some researchers have achieved grayscale normalization through histogram mapping between sub-regions, and some researchers have completed the migration of grayscale distribution using deep learning methods such as cycle consistent adversarial networks (CycleGANs) and U-Nets. Among the above normalization methods, the grayscale normalization method based on the CycleGAN network does not require strict pairing of the target and reference images, so it is more widely used in multi-center grayscale normalization. Based on the existing methods, some researchers have explored the impact of grayscale normalization on imaging omics analysis. Kociolek et al. aimed to analyze the impact of different image normalization methods on texture classification and found that large differences in grayscale distribution would affect the calculation of some classic imaging omics features, such as grayscale co-occurrence matrix (GLCM) and grayscale run length matrix (GRLM). Um et al. revealed that grayscale normalization reduces the correlation and instability between features, thereby improving the stability of radiomics, such as overall survival estimation. Isaksson et al. studied the role of grayscale normalization in prostate cancer radiomics and found that more than 87% of radiomics features changed dramatically after normalization.

[0005] Due to different machine scanning speeds, the inter-layer spacing of the collected data is inconsistent. The difference in inter-layer spacing will lead to different inter-layer features and spatial resolution of the image, which will also directly lead to differences in three-dimensional features. The most effective method is to use software (SPM, FSL, FreeSurfer) to unify the inter-layer spacing, and some neural networks (DeepVolume, 3D-Y-Net) are also used to complete the end-to-end mapping of three-dimensional data. Although deep network-based methods have also achieved excellent performance in inter-layer spacing reconstruction, their results are largely dependent on SPM-based preprocessing. Therefore, researchers have found that inconsistent inter-layer spacing is also an important factor affecting imaging omics. Zhao Tao et al. completed the conversion of thick-layer data to thin-layer data by interpolation and evaluated the impact of this normalization on the segmentation task. They concluded that inter-layer spacing normalization can improve the accuracy of 3D medical image segmentation. Saha et al. studied the effect of inter-layer spacing normalization on feature extraction and found that normalization improved the stability of the extracted texture features. Nandigam et al. explored the effect of inter-layer spacing on cerebral hemorrhage recognition and concluded that the recognition accuracy of thin-layer spacing data is 3 times that of thick-layer spacing data.

[0006] The above studies only performed one of the three types of normalization (e.g., modality, grayscale, or interslice normalization) and only evaluated the effect of radiomics under one normalization condition, without attempting to normalize multicenter data and verify its performance on various radiomics tasks. Therefore, how to better optimize non-standardized MR images and apply them to the normalization of brain MR images to more effectively achieve image segmentation, grading, and gene prediction has become an important area of ​​current research. Summary of the invention

[0007] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a brain MR image normalization system based on deep learning.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] A brain MR image normalization system based on deep learning, the system comprising:

[0010] The modality normalization module sets up a multi-input, multi-output fully connected neural network to synthesize the existing modalities of the MR images into the missing modalities and perform modality normalization;

[0011] Grayscale normalization module, builds a CycleGAN network, performs grayscale normalization on MR images, and maps the generated output image to the reference image space;

[0012] The inter-slice normalization module uses SPM to normalize the inter-slice spacing of MR images and reconstructs each MR image to the same inter-slice spacing;

[0013] The synthesis module synthesizes the results processed by the modality normalization module, the grayscale normalization module and the interslice spacing normalization module to obtain a normalized MR image.

[0014] Furthermore, before synthesizing the missing modality with the existing modality of the MR image, the mask region of the MR image is extracted, and the features of the extracted mask region include intensity, shape, texture and wavelet features.

[0015] Furthermore, the mask region of the MR image is extracted by multiplying the expert-annotated segmentation label with the original MR image.

[0016] Furthermore, the existing modalities of the MR images are synthesized into the missing modalities to achieve modality normalization. In this process, each modality image of the MR images is sent to an independent encoder, and the encoding channel of each modality is embedded into the multi-channel space using the improved U-Net. The step size of the multi-input and multi-output fully connected neural network is set to 1.

[0017] The fusion layer of the multi-input, multi-output fully connected neural network combines the single-modal features generated by each encoder, and then combines the features of multiple modalities. In each channel, if the eigenvalue of a certain modality is the maximum among all modal eigenvalues, it is selected as the fusion representation.

[0018] Furthermore, in the modality normalization module, the corresponding decoder adopts a fully convolutional network, which includes two residual modules, and each output modality is provided with an independent decoder to map the potential fusion representation to the output modality.

[0019] In the grayscale normalization module, the CycleGAN network includes a generator A2B and a discriminator B, the generator A2B includes a convolution downsampling path, a residual convolution path and a convolution upsampling path, and the discriminator B adopts PatchGAN. Preferably, the PatchGAN adopts LeakyReLU as the activation function.

[0020] Furthermore, the specific content of the inter-slice normalization module using SPM to normalize the inter-slice distance of the MR image is as follows:

[0021] Based on SPM, the original space of MR images is aligned with the MNI space and reconstructed based on the same voxel size; each MR image is reconstructed to the same slice spacing by adjusting the voxel size, and the corresponding mask is interpolated based on quartic interpolation to match the input size.

[0022] Furthermore, each MR image was reconstructed into the same inter-slice spacing by adjusting the voxel size, setting the deviation regularization to 0.0001 and the deviation FWHM to a cutoff of 60 mm; all inter-slice spacings were normalized to 1 mm in the z direction by quartic interpolation.

[0023] The brain MR image normalization system based on deep learning provided by the present invention has at least the following beneficial effects compared with the prior art:

[0024] The lack of normalization of existing medical image data has become a bottleneck in data-driven medical image research. To this end, the present invention proposes a brain MR image normalization system based on deep learning that converts inhomogeneous MR images into normalized images. First, an encoder-decoder network is used to implement modality normalization, so that the feature extractor has more useful features; secondly, a CycleGAN network is used to implement grayscale normalization, and the basic eigenvalues ​​are limited to a specific interval, reducing the sensitivity of the model to the data; finally, statistical parameter mapping is used to achieve uniform inter-layer spacing, improve the isotropy and resolution of the input image, and import more stable and complete three-dimensional features into the model to obtain better results; in the three tasks of glioma segmentation, pathological grading and gene prediction based on three databases provided in the implementation method, the normalized system of the present invention improves the Dice of segmentation by 9%, the AUC of pathological grading by 32%, and the IDH1 prediction by 25% compared with the non-normalized system. The system of the present invention can maximize the use of existing multi-center data and improve the effectiveness and robustness of imaging genomics judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a structural block diagram of a brain MR image normalization system based on deep learning in an embodiment;

[0026] Figure 2 Schematic diagram of a modality normalization network structure in a brain MR image normalization system based on deep learning in an embodiment;

[0027] Figure 3 Schematic diagram of the grayscale normalization network structure in the brain MR image normalization system based on deep learning in the embodiment;

[0028] Figure 4 This is a flow chart of normalization of interlayer spacing based on SPM in the embodiment;

[0029] Figure 5 This is a diagram of the normalization results of the brain MR image normalization system based on deep learning in the embodiment. DETAILED DESCRIPTION

[0030] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should belong to the scope of protection of the present invention.

[0031] Example

[0032] The present invention proposes a brain MR image normalization system based on deep learning (hereinafter referred to as Brain-Norm Net normalization system), which can reconstruct various brain MR images into a high-quality database with complete modality, uniform grayscale distribution, and uniform inter-layer spacing. The normalization process of the present invention follows the order of M→I→L (modality, grayscale, and inter-layer spacing are normalized in sequence), and the Brain-Norm Net normalization system consists of three modules: (1) multimodal synthesis based on encoder-decoder network; (2) grayscale normalization based on CycleGAN; (3) inter-layer spacing unification based on statistical parameter mapping (SPM). This embodiment uses three data sets containing 800 brain glioma images to verify the role of this system in three imaging omics: glioma segmentation, pathological grading, and IDH1 status prediction.

[0033] In this embodiment, three public data sets (BraTs17, TCGA and Huashan Hospital Affiliated to Fudan University, hereinafter referred to as Huashan Hospital) and MR images of a patient with glioma were collected to test the effectiveness of the Brain-Norm Net normalization system in the order of M→I→L in the segmentation, pathological grading and IDH1 status prediction of the present invention. The image data set contains four modalities: T1, T1c, T2 and FLAIR, and the image size is 256×256×155. The training set and the validation set are divided in a ratio of 2:1.

[0034] Glioma segmentation is the first step in imaging genomics analysis. In this example, 3D U-Net is selected to verify the impact of Brain-Norm Net on the segmentation task. The segmentation label is encoded using one-hot, where zero in the matrix represents the background area, and the other numbers represent the entire tumor, the tumor core, and the enhanced part of the tumor core. The cross entropy between the softmax layer output and the one-hot encoding is used as the loss function for network training. During the testing phase, morphological post-processing is performed on the upsampled segmentation results. The largest connected area of ​​each layer is selected, the connected areas less than 300 pixels are removed, the glioma is smoothed using a convolution kernel, and finally the small holes in the segmentation results are filled.

[0035] The grading of malignant tumors is crucial for treatment and prognosis. According to the WHO standard, grades I and II are low-grade gliomas (LGG), and grades III and IV are high-grade gliomas (HGG). The present invention first extracts the mask area using the result of multiplying the expert-annotated segmentation label with the original image. In addition, 555 features of each modality tumor area are extracted, including intensity, shape, texture, and wavelet features. The iterative sparse representation algorithm is used to select features with p-value <0.01. In order to adapt to the multimodal input form, the present invention introduces a regularization coefficient term and uses sparse representation for feature screening. Finally, the present invention uses a support vector machine to classify malignant tumors and uses the leave-one-out cross validation (LOOCV) method for verification.

[0036] As a predictive biomarker, isocitrate dehydrogenase (IDH) plays an important role in the prognosis of patients. Similar to the classification of malignant tumors, this embodiment extracts high-throughput features from the tumor region of interest (ROI), i.e., 555 features per modality. Sparse representation and support vector machine are used for feature screening and classification.

[0037] Figure 1 The system block diagram of the present invention is shown as follows: Figure 1 The specific contents of the three modules are as follows:

[0038] Module 1: Modal Normalization

[0039] like Figure 2 As shown, the present invention designs a multi-input, multi-output fully connected neural network to achieve modal normalization, that is, to use existing modalities to synthesize missing modalities. Each modality image is sent to an independent encoder. The improved 2DU-Net can initialize different numbers of encoders according to the number of modalities, can accept inputs of multiple modalities at the network input end, and embed the encoding channels of each modality into the multi-channel space respectively. The implementation here adopts the improved 2D U-Net that can realize the above operations using the existing technology, which will not be elaborated here.

[0040] To keep the output size consistent with the input size, the stride of the entire network is 1. The fusion layer effectively combines the features of multiple modalities by combining the unimodal features produced by each encoder. Specifically, in each channel, if the eigenvalue of a modality is the largest among all modal eigenvalues, it will be selected as the fused representation. The decoder of the present invention uses a fully convolutional network consisting of two residual modules, and each output modality has its own independent decoder, which maps the potential fused representation to the output modality.

[0041] The loss function consists of the sum of three loss functions. The first is the mean absolute error (MAE) loss between each pixel and the reference pixel output after direct encoding and decoding, without the need for a fusion layer, which is used to reduce the error between the generated image and the reference image. The second is the average loss of pixel variance between the encoded modalities, which is used to ensure the similarity of all input modalities after encoding. The third is the MAE loss between each pixel of the final output of the network and the reference pixel, which is used to supervise the entire network to update the weights. In this embodiment, the model can be trained preferentially using the Adam optimizer with a learning rate of 0.001. This embodiment uses a batch size of 16 images and sets the spatial channels to 16. All images are resized to 256×256.

[0042] Module 2: Grayscale Normalization

[0043] like Figure 3 As shown, the present invention constructs a CycleGAN network to achieve grayscale normalization. Through this dual learning process, the generated output image is mapped to the reference image space. More importantly, after introducing cycle consistency damage, the network can complete the mapping of the two domains without the need for strongly paired data for network training.

[0044] like Figure 3 As shown, CycleGAN consists of generator A2B and discriminator B, which can achieve the conversion from A to B. However, any input will be converted to the same B, and the generated B will not keep the original texture. After adding generator B2A and discriminator A, generator B2A retains the generated B in its original texture features. The generator is suitable for the improved U-Net. Each generation network consists of a convolution downsampling path, a residual convolution path, and a convolution upsampling path. The network discriminator uses PatchGAN to discriminate the network. PatchGAN contains 3 consecutive 4×4 downsampling convolutions with a step size of 2 and uses LeakyReLU as the activation function. Generally, GAN only needs to output a vector to judge whether it is true or not, indicating the evaluation of the entire image. However, PatchGAN outputs an N×N matrix, each element in which actually represents a relatively large receptive field in the original image. The number of convolution kernel channels is 64, 128, 256, 512, and 1 respectively. Finally, a 30×30 discriminant feature map is generated and synthesized to obtain the final discriminant result.

[0045] The loss of CycleGAN can be divided into three parts: the loss function of generator A to B, the loss function of generator B to A, and the cycle consistency loss. The three losses are added with a weight of 10:10:1 as the final optimized loss function. In this embodiment, the model is preferentially trained using the Adam optimizer with a learning rate of 0.0002 and a momentum term of 0.5. This embodiment uses a batch size of 1 image. All images are resized to 256×256.

[0046] Module 3: Normalization of inter-layer spacing

[0047] The present invention uses SPM (Statistical Parametric Mapping, www.fil.ion.ucl.ac.uk / spm / ) to normalize the inter-slice spacing. The original space of the image is aligned with the MNI space (Montreal Neurological Institute, a coordinate system established based on a series of magnetic resonance images of the normal human brain, established by the Montreal Neurological Institute in Canada) and reconstructed based on the same voxel size. Each input image is reconstructed to the same inter-slice spacing by adjusting the voxel size, and the corresponding mask is interpolated based on quartic interpolation to match the size of the input. Specifically, the deviation regularization is set to 0.0001 and the deviation FWHM is set to a cutoff of 60mm. Finally, all inter-slice spacings are normalized to 1mm in the z direction through quartic interpolation. The flow chart is as follows Figure 4 shown.

[0048] After normalization, the modes are normalized into four types (the BraTs dataset has full modes, and two of the modes are synthetic when evaluating the impact of mode normalization). Figure 5 As shown, the grayscale is normalized to the same intensity as the Siemens Magnetom Verio 3.0T magnetic resonance scanner. The inter-slice spacing of the three data sets is normalized to 1 mm. The present invention uses M±, I±, and L± to represent the normalization of the modality, grayscale, and inter-slice spacing, respectively, + represents normalization, and - represents non-normalization.

[0049] The results of the three data sets before and after M, I, and L normalization are shown in the figure below. Figure 5As shown. The original number of modalities of the three databases ranges from 1 to 3 (the ones with * are original modalities). After modality normalization, the images are reconstructed into four modalities, T2-flair, T1, T1-contract, and T2, with the same grayscale distribution and inter-slice spacing. The enlarged image view is shown in the box. The dotted box represents the image before inter-slice spacing normalization, and the solid box represents the image after inter-slice spacing normalization. There are obvious grayscale differences in the original T2-Flair images in the three databases. After grayscale normalization, the differences in grayscale distribution among the four modes are eliminated. Inter-slice spacing normalization means that the inter-slice spacing of all images is normalized to 1mm, so that the reconstructed image has better resolution. It can be seen from the enlarged view that the image resolution and texture details after inter-slice spacing normalization (solid box) are clearer than those before normalization (dashed box), and the coronal and sagittal planes are more obvious.

[0050] This embodiment also analyzes the effects of different normalized models on image segmentation, and specifically analyzes the segmentation task results of tumor images. Table 1 shows the segmentation results of different normalized models.

[0051] Table 1 Results of the effect of the normalization system on segmentation

[0052]

[0053] For the whole tumor (WT) segmentation task, the Dice of M+I+L+ (Dice is the most frequently used metric in medical image comparison. It is a set similarity metric, usually used to calculate the similarity of two samples, with a value threshold of [0, 1]) is 0.89, with a standard deviation of 0.08, and the Dice of MIL- is 0.81, with a standard deviation of 0.14. For the tumor core (TC) segmentation task, the Dice of M+I+L+ is 0.81, and the Dice of MIL- is 0.69. For the tumor core enhancement (ET) segmentation task, the Dice of M+I+L+ is 0.74, and the Dice of MIL- is 0.63. In tumor segmentation, for a single modality, the FLAIR modality is most beneficial for WT segmentation, and the T1C modality is most beneficial for TC and ET segmentation. After modality normalization, multimodal input is theoretically more accurate than single-modality segmentation, but the results show that multimodal input reduces the Dice coefficient of single-modality segmentation without further normalization of grayscale and inter-slice spacing. This is because the features of the synthetic mode are not well integrated and utilized before grayscale and inter-slice spacing normalization. Modality normalization combines the advantages of Flair (favorable for WT segmentation) and T1c (favorable for TC and ET segmentation); grayscale normalization and inter-slice spacing normalization improve image stability and spatial resolution, thereby improving the recognition of image details.

[0054] Table 2 shows the results of pathological classification on Huashan Hospital and TCGA datasets before and after MIL normalization for the impact of MR image normalization on pathological analysis. For pathological grading tasks, on Huashan Hospital and TCGA datasets, the AUC (Area Under Curve) of M+I+L+ is 0.89, and the AUC of MIL- is 0.69. The three normalization systems of M+IL-, M-I+L- and MI-L+ all have different degrees of improvement in case grading tasks. Compared with grayscale and inter-slice normalization, modality normalization has a more obvious improvement effect.

[0055] Table 2 Performance comparison of three normalization methods on pathology grading tasks

[0056]

[0057] Furthermore, the results of IDH1 status prediction before and after normalization on the Huashan Hospital and TCGA datasets are shown in Table 3. The AUC of the M+I+L+ model is 0.91, and the AUC of the MIL- model is 0.70. Modality normalization improves the results more significantly than grayscale normalization and inter-layer normalization.

[0058] Table 3 Performance comparison of three normalization methods on IDH1 status prediction task

[0059]

[0060]

[0061] The lack of normalization of medical image data has become a bottleneck in data-driven medical image research. This paper proposes a Brain-Norm Net normalization system that converts inhomogeneous MR images into normalized images. First, an encoder-decoder network is used to implement modality normalization, so that the feature extractor has more useful features. Secondly, grayscale normalization is implemented based on the CycleGAN network, which limits the basic eigenvalues ​​to a specific interval and reduces the sensitivity of the model to the data. Finally, statistical parameter mapping (SPM) is used to unify the inter-layer spacing, improve the isotropy and resolution of the input image, and import more stable and complete three-dimensional features into the model to obtain better results. In the three tasks of glioma segmentation, pathological grading and gene prediction based on three databases, the MIL normalization system improves the Dice of segmentation by 9%, the AUC of pathological grading by 32%, and the IDH1 prediction by 25% compared with the non-normalized system. The MIL normalization system can maximize the use of existing multi-center data and improve the effectiveness and robustness of radiomics diagnosis.

[0062] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed by the present invention, and these modifications or substitutions should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A brain MR image normalization system based on deep learning, characterized in that: include: The modality normalization module sets up a multi-input, multi-output fully connected neural network to synthesize the missing modality based on the existing modalities of the MR image and perform modality normalization; Grayscale normalization module, builds a CycleGAN network, performs grayscale normalization on MR images, and maps the generated output image to the reference image space; The inter-slice normalization module uses SPM to normalize the inter-slice spacing of MR images and reconstructs each MR image to the same inter-slice spacing; A synthesis module synthesizes the results processed by the modality normalization module, the grayscale normalization module and the interslice spacing normalization module to obtain a normalized MR image; In the process of modality normalization, the existing modalities of MR images are synthesized into the missing modalities. Each modality image of the MR images is sent to an independent encoder, and the encoding channel of each modality is embedded into the multi-channel space using the improved U-Net. The step size of the multi-input and multi-output fully connected neural network is set to 1. The CycleGAN network includes a generator A2B and a discriminator B, wherein the generator A2B includes a convolution downsampling path, a residual convolution path and a convolution upsampling path, and the discriminator B adopts PatchGAN; The specific content of the inter-slice normalization module for normalizing the inter-slice distance of MR images using SPM is as follows: Based on SPM, the original space of MR images is aligned with the MNI space and reconstructed based on the same voxel size; each MR image is reconstructed to the same slice spacing by adjusting the voxel size, and the corresponding mask is interpolated based on quartic interpolation to match the input size.

2. The deep learning-based brain MR image normalization system according to claim 1, characterized in that: Before synthesizing the missing modality with the existing modality of the MR image, the mask region of the MR image is extracted, and the features of the extracted mask region include intensity, shape, texture and wavelet features.

3. The deep learning-based brain MR image normalization system according to claim 2, characterized in that: The mask region of the MR image is extracted by multiplying the expert-annotated segmentation label with the original MR image.

4. The deep learning-based brain MR image normalization system according to claim 1, characterized in that: The fusion layer of the multi-input, multi-output fully connected neural network combines the single-modal features generated by each encoder, and then combines the features of multiple modalities. In each channel, if the eigenvalue of a certain modality is the maximum among all modal eigenvalues, it is selected as the fusion representation.

5. The deep learning-based brain MR image normalization system according to claim 4, characterized in that: In the modality normalization module, the corresponding decoder adopts a fully convolutional network, which includes two residual modules, and each output modality is provided with an independent decoder to map the potential fusion representation to the output modality.

6. The deep learning-based brain MR image normalization system according to claim 1, characterized in that: The PatchGAN uses LeakyReLU as the activation function.

7. The deep learning-based brain MR image normalization system according to claim 1, characterized in that: Each MR image was reconstructed into the same interslice spacing by adjusting the voxel size, setting the bias regularization to 0.0001, and the bias FWHM to a cutoff of 60 mm; all interslice spacings were normalized to 1 mm in the z direction by quartic interpolation.

Citation Information

Patent Citations

  • A method for intelligent assisted identification for diseases like pancreatic cancer and pancreatitis

    CN108537773A

  • Multi-modal medical image classification method under improved GoogLeNet neural network

    CN112465058A