A deep learning based 3t mra to 7t mra prediction method

Abstract

The present application belongs to the technical field of medical image processing, and specifically relates to a 3T MRA to 7T MRA prediction method based on deep learning. The present application uses the advantages of 7T MRA, such as high resolution, high signal-to-noise ratio and high contrast, to enhance the blood vessels of 3T MRA and improve the image quality; the present application comprises the following steps: acquiring 3T and 7T MRA paired original data; image preprocessing, including registration, field inhomogeneity correction, de-scalping, image interpolation and pixel value normalization; training a deep learning model; inputting the 3T MRA used for testing into the trained model to predict a '7T-like' MRA. In the model training process, the present application uses a knowledge distillation network to introduce MIP information of MRA to improve the vascular integrity, continuity and authenticity of the prediction result; the present application uses heteroscedastic aleatoric uncertainty to adaptively weight the error between the real 7T MRA and the predicted 7T-like MRA, prevents the spatial mismatch between 3T and 7T MRA from bringing adverse effects to the model learning and causing unreasonable deviation of the prediction result.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to a prediction method from 3T MRA to 7T MRA. Background Art

[0002] Magnetic Resonance Angiography (MRA) is a non-invasive vascular imaging method that does not require the use of contrast agents. Compared with other vascular imaging examinations, MRA has the advantages of no ionizing radiation, no radioactive damage, and no reliance on contrast agents. It is helpful for the diagnosis and treatment of cardiovascular and cerebrovascular diseases such as coronary artery stenosis, carotid bifurcation lesions, and intracranial aneurysms, and is currently widely used in clinical practice.

[0003] Although MRA can produce three-dimensional images, it is usually observed and analyzed using Maximum Intensity Projection (MIP). The main disadvantage of MIP is that non-vascular structures overlap and small blood vessels with low contrast are almost invisible. Enhancing blood vessels in MRA can significantly improve this problem and enhance imaging quality. Existing methods for blood vessel enhancement mainly include Frangi filtering, Gabor wavelet transform, and Jerman filtering. These methods use filters for blood vessel enhancement, which requires manual parameter adjustment and is easily affected by blurred blood vessel edges and image noise. In addition, the enhancement effect in distal small blood vessel areas with weak vascular signal intensity and low contrast needs to be improved.

[0004] In summary, the effectiveness of these filter-based vascular enhancement methods is limited to some extent by the quality of vascular images, such as spatial resolution, signal-to-noise ratio, and contrast. In recent years, significant efforts have been made to improve the quality of magnetic resonance images, including increasing the field strength of magnetic resonance imaging equipment. Compared with conventional clinical MRI (3T and 1.5T), ultra-high-field MRI (7T) offers advantages such as higher resolution, higher signal-to-noise ratio, and stronger tissue contrast, which facilitate early screening and precise diagnosis and treatment of small lesions. Regarding MRA, 7T MRA contains richer vascular information, clearer vascular edges, and stronger signals of distal small vessels than 3T MRA, making it more accurate in vascular segmentation tasks. This has significant diagnostic value and clinical significance for cerebrovascular diseases, especially lesions occurring in distal vessels or side branches.

[0005] However, 7T MRI scanners are currently expensive and scarce. If 3T MRA images could be used to predict images that could, to a certain extent, replace 7T MRA, i.e., high-quality 7T-like MRA, and applied to practical medical diagnosis or scientific research, this would potentially enhance the diagnostic capabilities of the current mainstream 3T MRA and provide a cost-effective solution. Therefore, the present invention aims to predict 7T MRA from 3T MRA. Essentially, this approach leverages the advantages of 7T MRA, such as high resolution, high signal-to-noise ratio, and high contrast, to enhance vascular quality in 3T MRA, thereby obtaining higher-quality vascular images. Summary of the Invention

[0006] To overcome the shortcomings and limitations of the above-mentioned existing vascular enhancement methods, the present invention provides a deep learning-based prediction method for 3T magnetic resonance vascular images to 7T magnetic resonance vascular images. The purpose is to utilize the advantages of 7T MRA such as high resolution, high signal-to-noise ratio, and high contrast to perform vascular enhancement on 3T MRA and improve the quality of vascular images.

[0007] The deep learning-based prediction method for 3T MRI angiography to 7T MRI angiography provided by this paper uses a generative adversarial network (GAN) as the foundational network of the overall model framework to learn the nonlinear mapping relationship from 3T MRA to 7T MRA. To further improve the quality of the prediction results, this paper mainly includes the following two parts:

[0008] 1. Based on the characteristics of MRA vascular imaging, the MIP image of a slice covers the vascular morphology information of several consecutive slices centered on that slice. Therefore, to improve the vascular integrity, continuity, and authenticity of the prediction results, this paper uses a knowledge distillation (KD) network model to introduce MRA MIP information to guide model learning.

[0009] 2. Learning the nonlinear mapping relationship from 3T MRA to 7T MRA requires registered 3T and 7T paired data. However, there is still a certain degree of spatial mismatch between the 3T and 7T paired data after registration. To prevent this mismatch from adversely affecting model learning and causing unreasonable deviations in prediction results, the present invention adaptively weights the error between the true 7T MRA and the predicted 7T-like MRA by incorporating heteroscedastic aleatoric uncertainty (AU).

[0010] The present invention provides a deep learning-based prediction method for 3T MRA to 7T MRA, specifically including:

[0011] (1) Using the knowledge distillation network to introduce MIP information of MRA

[0012] The knowledge distillation network framework of the present invention consists of multiple teacher networks, a single student network and several feature fusion modules. The teacher network and the student network are both 3D GANs with the same network structure. The GAN generator adopts 3D U-net; the feature fusion module is an autoencoder structure, and encoding and decoding are respectively implemented by independent 3D convolution. Each teacher network learns the mapping from 3T MRA MIP to 7T MRA MIP under one MIP direction, with three MIP directions: axial, coronal and sagittal; the student network learns the mapping from 3T MRA original image to 7T MRA original image; the MIP information learned by all teacher networks is fused by the feature fusion module and extracted to the student network through knowledge distillation.

[0013] The present invention is based on image patches for training and prediction. For ease of explanation, the following symbols are defined:

[0014] remember is a three-dimensional 3T MRA image patch, the size of each dimension is N; is the 3T MRA MIP image patch obtained after X is MIPed in the mth direction, m=1,…,M, where M is the total number of MIP directions, i.e. the number of teacher networks. Similarly, is the 7T MRA image patch corresponding to X; is the 7T MRA MIP image patch obtained after MIP in the mth direction of Y, m=1,…,M. The teacher network is denoted as T={G (m) ,D (m)},m=1,…,M,G (m) ,D (m) are the generator and discriminator of the mth teacher network respectively; the student network is S = {G, D}, G, D are the generator and discriminator of the student network respectively; the feature fusion module is A (l) ,l=1,…,L, L is the number of network layers of the generator of the teacher network, which is equal to the number of layers of the generator of the student network, and each layer corresponds to a feature fusion module.

[0015] Next, we will introduce the training process and loss function of knowledge distillation by introducing MIP information in detail. The main steps are as follows:

[0016] Step 1: Train the teacher network. First, let the 7T-like MIP image patch predicted by the mth teacher network be Right now The loss function of the teacher network consists of the following two parts:

[0017] ① Generate adversarial loss

[0018]

[0019] ② Mean absolute error (MAE)

[0020]

[0021] Combining ① and ②, the loss function of the teacher network is as follows:

[0022]

[0023] Where α is the weight of MAE loss, m=1,…,M.

[0024] It should be noted that after all teacher networks are trained, the network parameters will no longer change.

[0025] Step 2: Train the student network. First, the 7T-like image patch predicted by the student network is recorded as Right now The loss function of the student network consists of the following four parts:

[0026] ① Generate adversarial loss

[0027]

[0028] ② Mean absolute error

[0029]

[0030] ③ Feature reconstruction loss

[0031] The generators of M teacher networks (G (m) ,m=1,…,M) Each layer at the same position corresponds to a feature fusion module, which is used to fuse the M feature maps output by the M teacher networks at the network layer at that position. Here, the lth feature fusion module A (l) (corresponding to the lth layer of each teacher network generator) as an example.

[0032] remember G (m) The feature map output at layer l, m = 1,…,M. C (l) ×H (l) ×H (l) ×H (l) is the size of the feature map of the first layer, C (l)is the channel dimension, and since all 3D convolution kernels in the network have the same size in each dimension, the sizes of the other dimensions of the feature map are the same. First, (1)(l) ,F (2)(l) ,…,F (M)(l) Splicing is performed in the channel dimension as A (l) Input

[0033]

[0034] Feature fusion module A (l) Essentially, it is an automatic encoder, consisting of the encoding part Enc (l) and decoding part Dec (l) Composition, encoding and decoding are each implemented by a 1×1×1 3D convolution layer:

[0035]

[0036] in, It's A (l) Output; It is the output of the encoding part, that is, the feature fusion module A (l) The learned fusion features contain the feature information of M teacher networks.

[0037] In order to make A (l) To effectively fuse and extract features, and The error between them should be as small as possible.

[0038]

[0039] in, It's A (l) The reconstructed feature map of the mth MIP direction, then the problem is transformed into making F (m)(l) and The error between them is as small as possible, m = 1, ..., M, l = 1, ..., L. Therefore, the feature reconstruction loss of all L layers of all M teacher networks is summed up to get the total feature reconstruction loss as follows:

[0040]

[0041] ④Knowledge distillation loss

[0042] remember The feature map output by the generator G of the student network at layer l is obtained by minimizing and The error between them is used to realize the guiding role of MIP information for 7T-like MRA prediction, l = 1,…,L. Therefore, the knowledge distillation loss of all L layers is summed up to get the total knowledge distillation loss as follows:

[0043]

[0044] Combining ①②③④, the loss function of the student network is as follows:

[0045]

[0046] Among them, α is the weight of MAE loss, β is the weight of feature reconstruction loss, and γ is the weight of knowledge distillation loss.

[0047] 2. Error Adaptive Weighting Based on Heteroscedasticity Accidental Uncertainty

[0048] For deep learning models, learning the mapping relationship from 3T MRA to 7T MRA requires paired 3T and 7T data after registration as a training set, namely X and Y above. Rigid registration is usually used for 3T MRA and 7T MRA of the same subject. However, there is still a certain degree of spatial mismatch between the registered 3T and 7T. According to Bayesian deep learning, this is essentially the inherent noise of the observation data and can be modeled by aleatoric uncertainty. Aleatoric uncertainty can be further divided into heteroscedastic aleatoric uncertainty and homoscedastic aleatoric uncertainty. Heteroscedastic aleatoric uncertainty depends on the input data and is predicted as an output of the model; homoscedastic aleatoric uncertainty is independent of the input data and is not an output of the model, but a quantity that remains constant for all input data and varies between different tasks. Because the spatial mismatch between X and Y mainly occurs at the edges of blood vessels rather than in relatively uniform grayscale areas such as the image background or brain tissue, it is clear that this noise is related to the input data. Therefore, the present invention uses heteroscedastic aleatoric uncertainty to model it, that is, the noise (spatial mismatch) is measured by heteroscedastic aleatoric uncertainty. In this case, the student network not only predicts the 7T-like patch Also predicted The corresponding heteroskedasticity aleatoric uncertainty is denoted as Right now The mean absolute error shown in formula (5) is replaced by:

[0049]

[0050] The essence of this formula is to use heteroscedastic aleatoric uncertainty to adaptively weight the error between the true 7T MRA and the predicted 7T-like MRA, thereby preventing spatial mismatch from adversely affecting model learning and causing unreasonable deviations in prediction results. i,j,k , They are Y, Elements in The second term is a regular term that prevents the average weight of the error from being too small or too large.

[0051] (3) Adaptive loss weight

[0052] In order to explore and compare the contribution of different MIP directions and different network layers to the total feature reconstruction loss, the present invention applies the homoscedasticity accidental uncertainty described in content (II) to different values ​​of m and l. The feature reconstruction loss is adaptively weighted, and the feature reconstruction loss shown in formula (9) is Rewrite as follows:

[0053]

[0054] in, It's F (m)(l) and The reconstruction loss weight between The weight matrix consists of m=1,…,M,l=1,…,L.

[0055] Similarly, in order to explore and compare the contribution of different network layers to the total knowledge distillation loss, the present invention applies homoscedasticity accidental uncertainty to different l values. The knowledge distillation loss is adaptively weighted to convert the knowledge distillation loss shown in (10) Rewrite as follows:

[0056]

[0057] Among them, ω l yes and The knowledge distillation loss weight between ω is ω l ,l=1,…,L is the weight vector.

[0058] (4) Total loss function of the student network

[0059] Combining (1), (2), and (3), the total loss function of the student network in the present invention is:

[0060]

[0061] Among them, α is the weight of the accidental uncertainty weighted MAE loss, β is the weight of the feature reconstruction loss, and γ is the weight of the knowledge distillation loss.

[0062] In summary, the present invention's deep learning-based prediction method for 3T MRA to 7T MRA has the following specific steps:

[0063] (1) Constructing 3T and 7T MRA paired datasets;

[0064] (2) Image preprocessing;

[0065] (2-1) Each subject's 3T MRA was registered to its corresponding 7T MRA;

[0066] (2-2) Field inhomogeneity correction was performed on all 7T MRAs;

[0067] (2-3) All 3T and 7T MRAs were scalped and brain images were extracted;

[0068] (2-4) Interpolate all 3T and 7T MRA image matrices and normalize the pixel values;

[0069] After image preprocessing, the dataset was divided into a training set and a test set, which were used to train the deep learning model and predict 7T-like MRA, respectively.

[0070] (3) Training

[0071] (3-1) Perform MIP on the 3T and 7T MRA of the training set in the axial, coronal, and sagittal planes to obtain MIP images;

[0072] (3-2) Divide the 3T and 7T MRA and 3T and 7T MRA MIP in the training set into image blocks, and perform training based on the image blocks;

[0073] (4) Forecast

[0074] (4-1) The 3T MRA images of the test set are divided into several image blocks of size N×N×N with a certain step size, and prediction is performed based on the image blocks;

[0075] (4-2) Input the 3T image block into the generator G of the trained student network, and the network predicts a 7T-like image block of the same size of N×N×N;

[0076] (4-3) All 7T-like image blocks predicted from all image blocks of the original 3T MRA are spliced ​​into a complete 7T-like MRA image, and the average of the overlapping parts between different 7T-like patches is taken.

[0077] The beneficial effects of the technical solution provided by the present invention are:

[0078] (1) This paper uses deep learning to predict 7T MRA from 3T MRA, effectively utilizing the advantages of 7T MRA, such as high resolution, high signal-to-noise ratio, high contrast, and rich small vessel signals. It breaks through the limitations of traditional vascular enhancement methods and can obtain higher-quality vascular images from 3T MRA. It has certain reference significance for the clinical diagnosis of small vessel diseases.

[0079] (2) The present invention uses a knowledge distillation algorithm to introduce MIP information during the model training process. This allows the model to be guided by the 3T MRA MIP to 7T MRA MIP mapping while learning the 3T MRA to 7T MRA mapping, thereby improving the vascular integrity, continuity, and authenticity of the prediction results.

[0080] (3) The present invention uses heteroscedastic aleatoric uncertainty to adaptively weight the error between the true 7T MRA and the predicted 7T-like MRA, thereby preventing the spatial mismatch between 3T and 7T data from adversely affecting model learning and causing unreasonable deviations in the prediction results, thereby enhancing the robustness of the algorithm to noisy data. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 Flowchart of the prediction method from 3T MRA to 7T MRA based on deep learning of the present invention.

[0082] Figure 2 The prediction results of the proposed prediction method on a test case and its corresponding 3T TOF and 7T TOF are shown in the figure. From left to right, they are 3T, 7T-like, and 7T, and from top to bottom, they are the axial, coronal, and sagittal MIP projections of the test case. DETAILED DESCRIPTION

[0083] The present invention proposes a deep learning-based prediction method for 3T MRA to 7T MRA. Below, we use time-of-flight (TOF) MRA, one of the MRA imaging techniques, as an example, and describe the specific embodiments of the present invention in detail with reference to the accompanying drawings. Note that the embodiments described below are only used to illustrate and explain the present invention and are not intended to limit the applicability or use of the present invention. For example, the present invention can also be used for other MRA imaging techniques such as phase contrast (PC) MRA and contrast-enhanced (CE) MRA. Any omissions, substitutions, changes, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

[0084] like Figure 1 As shown in the figure, the prediction method from 3T MRA to 7T MRA based on deep learning, taking TOF as an example, the specific steps are:

[0085] (1) Constructing 3T and 7T TOF paired datasets

[0086] We recruited multiple subjects and collected 3T TOF and 7T TOF images for each subject on a 3T MRI scanner and a 7T MRI scanner, respectively, to obtain multiple pairs of 3T and 7T TOF data. The matrix size of the 3T TOF image is 696×768×168, and the spatial resolution is 0.2604×0.2604×0.7mm. 3 The matrix size of the 7T TOF image is 944×1152×232, and the spatial resolution is 0.1736×0.1736×0.5mm. 3 The three dimensions of the matrix correspond to the sagittal, coronal, and axial planes in order.

[0087] (2) Image preprocessing

[0088] (2-1) Use SPM12 to align each subject's 3T TOF to its corresponding 7T TOF;

[0089] (2-2) Perform N4 correction on all 7T TOFs to eliminate field inhomogeneity;

[0090] (2-3) Use BET to remove the scalp of all 3T and 7T TOF and extract brain images;

[0091] (2-4) Interpolate the matrix size of all 3T and 7T TOFs to 512×512×320, and normalize the pixel values ​​to between 0 and 255.

[0092] After image preprocessing, the dataset is divided into a training set and a test set, which are used to train the deep learning model and predict 7T-like TOF, respectively.

[0093] (3) Training

[0094] (3-1) Obtaining MIP images

[0095] MIP images were performed on the 3T and 7T TOF images in the training set in the axial, coronal, and sagittal planes, with an MIP thickness of W. The specific method for performing MIP in the axial plane is as follows: (W-1) / 2 slices of all-zero slices are padded at both ends of the image dimension corresponding to the axial plane (W must be an odd number), resulting in an image matrix size of 512×512×(320+W-1). The MIP window was then slid in the axial direction with a window width of W and a step size of 1, resulting in an axial MIP image with a matrix size of 512×512×320. The MIP method for the coronal and sagittal planes is the same as that for the axial plane and is not repeated here.

[0096] (3-2) Dividing image blocks

[0097] The 3T and 7T TOF and 3T and 7T TOF MIP in three directions of the training set are divided into image blocks of size N×N×N. The present invention performs training based on image blocks.

[0098] To facilitate the description of subsequent steps, the following symbols are defined:

[0099] remember is a three-dimensional 3T TOF image block, the size of each dimension is N; is the 3T TOF MIP image block obtained after X is MIPed in the mth direction, m=1,…,M, M is the total number of MIP directions, that is, the number of teacher networks. Similarly, is the 7T TOF image block corresponding to X; is the 7T TOF MIP image block obtained after Y is MIPed in the mth direction, m=1,…,M. The teacher network is denoted as T={G (m) ,D (m)},m=1,…,M; the student network is S={G,D}; the feature fusion module is A (l) ,l=1,…,L, L is the number of network layers of the generator of the teacher network (equal to the number of generator layers of the student network), and each layer corresponds to a feature fusion module.

[0100] (3-3) Training Teacher Network

[0101] First, the 7T-like MIP image block predicted by the m-th teacher network is recorded as Right now The loss function of the teacher network consists of the following two parts:

[0102] ① Generate adversarial loss

[0103]

[0104] ② Mean absolute error (MAE)

[0105]

[0106] Combining ①②, the loss function of the teacher network is as follows:

[0107]

[0108] Where α is the weight of MAE loss, m=1,…,M.

[0109] It should be noted that after all teacher networks are trained, the network parameters will no longer change.

[0110] (3-4) Training student network

[0111] First, the 7T-like patch predicted by the student network is The corresponding heteroskedastic aleatoric uncertainty is Right now The space mismatch between X and Y is caused by Metrics. The loss function of the student network consists of the following four parts:

[0112] ① Generate adversarial loss

[0113]

[0114] ②Accidental uncertainty weighted MAE loss

[0115]

[0116] Among them, y i,j,k , They are Y, Elements in The second term is a regular term that prevents the average weight of the error from being too small or too large.

[0117] ③ Feature reconstruction loss

[0118] The generators of M teacher networks (G (m) ,m=1,…,M) Each layer at the same position corresponds to a feature fusion module, which is used to fuse the M feature maps output by the M teacher networks at the network layer at that position. Here, the lth feature fusion module A (l) (corresponding to the lth layer of each teacher network generator) as an example.

[0119] remember G (m) The feature map output at layer l, m = 1,…,M. C(l) ×H (l) ×H (l) ×H (l) is the size of the feature map of the first layer, C (l) is the channel dimension, and since all 3D convolution kernels in the network have the same size in each dimension, the sizes of the other dimensions of the feature map are the same. First, (1)(l) ,F (2)(l) ,…,F (M)(l) Splicing is performed in the channel dimension as A (l) Input

[0120]

[0121] Feature fusion module A (l) Essentially, it is an automatic encoder, consisting of the encoding part Enc (l) and decoding part Dec (l) Composition, encoding and decoding are each implemented by a 1×1×1 3D convolution layer:

[0122]

[0123] in It's A (l) Output; It is the output of the encoding part, that is, the feature fusion module A (l) The learned fusion features contain the feature information of M teacher networks.

[0124] In order to make A (l) To effectively fuse and extract features, and The error between them should be as small as possible. in It's A (l) The reconstructed feature map of the mth MIP direction, then the problem is transformed into making F (m)(l) and The error between them is as small as possible, m=1,…,M,l=1,…,L. Therefore, the feature reconstruction losses of all L layers of all M teacher networks are summed, and the homoscedastic accidental uncertainty is used for different values ​​of m and l. The feature reconstruction loss is adaptively weighted to obtain the total feature reconstruction loss as follows:

[0125]

[0126] in It's F (m)(l) and The reconstruction loss weight between The weight matrix consists of m=1,…,M,l=1,…,L.

[0127] ④Knowledge distillation loss

[0128] remember The feature map output by the generator G of the student network at layer l is obtained by minimizing and The error between them realizes the guiding role of MIP information for 7T-like TOF prediction, l = 1, ..., L. Therefore, the knowledge distillation loss of all L layers is summed, and the homoscedastic accidental uncertainty is used to calculate the loss of different l values. The knowledge distillation loss is adaptively weighted to obtain the total knowledge distillation loss as follows:

[0129]

[0130] Among them, ω l yes and The knowledge distillation loss weight between ω is ω l ,l=1,…,L is the weight vector.

[0131] Combining ①②③④, the loss function of the student network is as follows:

[0132]

[0133] Where α is the weight of the accidental uncertainty weighted MAE loss, β is the feature reconstruction loss weight, and γ is the knowledge distillation loss weight.

[0134] (4) Forecast

[0135] (4-1) The 3T TOF image of the test set is divided into several image blocks of size N×N×N with a certain step size, and the present invention performs prediction based on the image blocks;

[0136] (4-2) Input the 3T image block into the generator G of the trained student network, and the network predicts a 7T-like image block of the same size of N×N×N;

[0137] (4-3) All 7T-like image blocks predicted from all image blocks of the original 3T TOF are spliced ​​into a complete 7T-like TOF image, and the average of the overlapping parts between different 7T-like image blocks is taken.

[0138] Figure 2The following figure shows the prediction results of the proposed prediction method on a test case, along with the corresponding 3T TOF and 7T TOF images. From left to right, the 3T, 7T-like, and 7T images are shown, and from top to bottom, the axial, coronal, and sagittal MIP projections of the test case are shown. As can be seen from the figure, the proposed prediction method can effectively improve the contrast and richness of blood vessels in 3T TOF images, and this improvement is more pronounced in small cerebral vessels.

Claims

1. A prediction method for 3T MRA to 7T MRA based on deep learning, characterized in that: A generative adversarial network (GAN) is used to learn the nonlinear mapping relationship from 3T MRA to 7T MRA. At the same time: Using the knowledge distillation network, the MIP information of MRA is introduced to guide the learning of the network model; The error between the true 7T MRA and the predicted 7T-like MRA is adaptively weighted by incorporating heteroskedastic aleatoric uncertainty; The knowledge distillation network is used to introduce the MIP information of MRA, where: The knowledge distillation network consists of multiple teacher networks, a single student network, and several feature fusion modules. Both the teacher and student networks are 3D GANs with the same network structure, and the GAN generator uses a 3D U-net. The feature fusion module is an autoencoder structure, with encoding and decoding implemented by independent 3D convolutions. Each teacher network learns the mapping from 3T MRA MIP to 7T MRA MIP in one MIP direction: axial, coronal, and sagittal. The student network learns the mapping from 3T MRA original images to 7T MRA original images. The MIP information learned by all teacher networks is fused by the feature fusion module and then extracted into the student network through knowledge distillation.

2. The method for predicting 3T MRA to 7T MRA according to claim 1, characterized in that: Use image patches to train the knowledge distillation network; remember is a three-dimensional 3T MRA image block, the size of each dimension is N; is the 3T MRA MIP image block obtained after X is MIPed in the mth direction, m=1,…,M, M is the total number of MIP directions, that is, the number of teacher networks; similarly, is the 7T MRA image block corresponding to X; is the 7T MRA MIP image block obtained after Y is MIPed in the mth direction, m=1,…,M; the teacher network is denoted as T={G (m) ,D (m) },m=1,…,M,G (m) ,D (m) are the generator and discriminator of the mth teacher network respectively; the student network is S = {G, D}, G, D are the generator and discriminator of the student network respectively; the feature fusion module is A (l) ,l=1,…,L, L is the number of network layers of the teacher network generator, which is equal to the number of layers of the student network generator. Each layer corresponds to a feature fusion module. The training process and loss function of the MIP information introduced by knowledge distillation are as follows: Step 1: Train the teacher network; First, the 7T-like MIP image block predicted by the m-th teacher network is recorded as Right now m=1,…,M; the loss function of the teacher network consists of the following two parts composition: ① Generate adversarial loss: ②Mean absolute error MAE: Combining ① and ②, the loss function of the teacher network is as follows: Where α is the weight of MAE loss, m=1,…,M; After all teacher networks are trained, the network parameters are no longer changed; Step 2: Train the student network; First, the 7T-like image block predicted by the student network is Right now The loss function of the student network consists of the following four parts: ① Generate adversarial loss: ②Mean absolute error: ③ Feature reconstruction loss: Generator G of M teacher networks (m) ,m=1,…,M, each layer at the same position corresponds to a feature fusion module, which is used to fuse the M feature maps output by the M teacher networks at the network layer at that position; for the lth feature fusion module A (l) , corresponding to the lth layer of each teacher network generator; remember G (m) The feature map output at layer l, m = 1, ..., M; C (l) ×H (l) ×H (l) ×H (l) is the size of the feature map of the first layer, C (l) is the channel dimension, and since all 3D convolution kernels in the network have the same size in each dimension, the sizes of the other dimensions of the feature map are the same; first, (1)(l) ,F (2)(l) ,…,F (M)(l) Splicing is performed in the channel dimension as A (l) Input Feature fusion module A (l) It is an automatic encoder, consisting of the encoding part Enc (l) and decoding part Dec (l) Composition, encoding and decoding are each implemented by a 1×1×1 3D convolution layer: in, It's A (l) Output; It is the output of the encoding part, that is, the feature fusion module A (l) The learned fusion features contain M teacher network feature information; In order to make A (l) To effectively fuse and extract features, and The error between them should be as small as possible; in, It's A (l) The reconstructed feature map of the mth MIP direction, then the problem is transformed into making F (m)(l) and The error between them is as small as possible, m = 1,…,M, l = 1,…,L; then, the feature reconstruction loss of all L layers of all M teacher networks is summed up to get the total feature reconstruction loss as follows: ④Knowledge distillation loss: remember The feature map output by the generator G of the student network at layer l is obtained by minimizing and The error between them is used to realize the guiding role of MIP information for 7T-like MRA prediction, l = 1,…,L, so the knowledge distillation loss of all L layers is summed up to get the total knowledge distillation loss as follows: Combining ①, ②, ③, and ④, the loss function of the student network is as follows: Among them, α is the weight of MAE loss, β is the weight of feature reconstruction loss, and γ is the weight of knowledge distillation loss.

3. The prediction method from 3T MRA to 7T MRA according to claim 2, characterized in that: The error adaptive weighting based on the heteroskedastic aleatoric uncertainty is as follows: Learning the mapping relationship from 3T MRA to 7T MRA requires the registered 3T and 7T paired data as the training set, namely X and Y in the above text. Since the spatial mismatch between X and Y mainly occurs at the edge of the blood vessels rather than in relatively uniform grayscale areas such as the image background or brain tissue, this noise is related to the input data. Therefore, heteroscedastic random uncertainty is used to model it, that is, the noise is measured by heteroscedastic random uncertainty. In this case, the student network not only predicts 7T-like image patches, but also predicts the image quality of the 7T-like patches. Also predicted The corresponding heteroskedasticity aleatoric uncertainty is denoted as Right now The mean absolute error shown in (5) is replaced by: The essence of this formula is to use heteroscedastic accidental uncertainty to adaptively weight the error between the true 7T MRA and the predicted 7T-like MRA, so as to prevent the spatial mismatch from causing adverse effects on model learning and causing unreasonable deviations in the prediction results; where y i,j,k , They are Y, Elements in The second term is a regular term that prevents the average weight of the error from being too small or too large.

4. The method for predicting 3T MRA to 7T MRA according to claim 2, characterized in that: In order to explore and compare the contribution of different MIP directions and different network layers to the total feature reconstruction loss, the homoscedasticity accidental uncertainty is applied to different values ​​of m and l. The feature reconstruction loss of is adaptively weighted, and the feature reconstruction loss shown in formula (9) is Rewrite as follows: in, It's F (m)(l) and The reconstruction loss weight between The weight matrix composed of m=1,…,M,l=1,…,L; Similarly, to explore and compare the contribution of different network layers to the total knowledge distillation loss, we apply homoscedastic accidental uncertainty to different l values. The knowledge distillation loss is adaptively weighted, and the total knowledge distillation loss shown in formula (10) is Rewrite as follows: Among them, ω l yes and The knowledge distillation loss weight between ω is ω l ,l=1,…,L composed of weight vector; The total loss function of the student network is: Among them, α is the weight of the accidental uncertainty weighted MAE loss, β is the weight of the feature reconstruction loss, and γ is the weight of the knowledge distillation loss.

5. The deep learning-based prediction method for 3T MRA to 7T MRA according to any one of claims 1 to 4, characterized in that: The specific steps are as follows: (1) Constructing 3T and 7T MRA paired datasets; (2) Image preprocessing; (2-1) Each subject's 3T MRA was registered to its corresponding 7T MRA; (2-2) Field inhomogeneity correction was performed on all 7T MRAs; (2-3) All 3T and 7T MRAs were scalped and brain images were extracted; (2-4) Interpolate all 3T and 7T MRA image matrices and normalize the pixel values; After image preprocessing, the dataset was divided into a training set and a test set, which were used to train the deep learning model and predict 7T-like MRA, respectively. (3) Training (3-1) Perform MIP on the 3T and 7T MRA of the training set in the axial, coronal, and sagittal planes to obtain MIP images; (3-2) Divide the 3T and 7T MRA and 3T and 7T MRA MIP in the training set into image blocks, and perform training based on the image blocks; (4) Forecast (4-1) The 3T MRA images of the test set are divided into several image blocks of size N×N×N with a certain step size, and prediction is performed based on the image blocks; (4-2) Input the 3T image block into the generator G of the trained student network, and the network predicts a 7T-like image block of the same size of N×N×N; (4-3) All 7T-like image blocks predicted from all image blocks of the original 3T MRA are spliced ​​into a complete 7T-like MRA image, and the average of the overlapping parts between different 7T-like patches is taken.

Images