Multi-mode dMRI image distortion correction system based on unsupervised deep learning

Abstract

The invention belongs to the technical field of medical image processing, and particularly relates to a multi-mode dMRI image distortion correction system based on unsupervised deep learning. The correction system comprises a coarse-to-fine deep learning network-multi-modal distortion correction network, wherein the network is composed of a rigid transformation network and a dense transformation network which are cascaded; in addition, the system also comprises a cascade training strategy module. According to the invention, rich information of two modes is fused and input into the network; the method comprises the following steps: learning linear dislocation between two images subjected to opposite phase coding by using a rigid transformation network; learning a distortion field from the images of the two modes by using a double-flow encoder and a multi-mode attention module through a dense transformation network; in addition, a dynamic weighted average (DWA) method is adopted to realize simultaneous optimization of multi-modal images. Compared with an existing magnetic susceptibility distortion correction method, the method has the advantage that the magnetic susceptibility distortion of the dMRI image can be more efficiently and accurately corrected.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to an image correction system for unsupervised deep learning of dMRI images. Background Art

[0002] Diffusion magnetic resonance imaging (dMRI) is a non-invasive imaging technique that can capture the diffusion pattern of water molecules in biological tissues and is widely used to track white matter fiber tracts in the human brain. However, the echo planar imaging (EPI) sequence commonly used to acquire dMRI is susceptible to geometric and intensity distortion caused by magnetic susceptibility in the phase encoding direction. This distortion often occurs at the boundaries of different tissues and cavities, such as the brainstem region. These distortion phenomena significantly affect the accuracy and reliability of subsequent analysis and interpretation of brain data.

[0003] At present, the most commonly used magnetic susceptibility distortion correction method is "blip-up blip-down", which is to scan two B0 images in opposite phase encoding directions during acquisition and then estimate their distortion fields from them. Among them, the topup method in the FSL toolbox uses discrete cosine basis functions to derive a smooth displacement field from the two acquired B0 images, and combines the two images by the least squares method to obtain the distortion field. In recent years, with the development of deep learning technology, some methods use unsupervised deep learning to estimate the distortion field from two B0 images with opposite phases, and achieve correction effects similar to topup with shorter estimation time. However, due to the insufficient contrast of B0 images in some areas, the above distortion correction algorithms based on B0 images have poor correction effects in some brain areas, such as the brainstem and corpus callosum.

[0004] To address this problem, some methods began to use other feature images to correct the magnetic susceptibility distortion of dMRI, such as fiber orientation distribution (FOD) images. Qiao et al. proposed FODReg, which aims to correct the residual distortion in dMRI data preprocessed by topup by estimating a fine displacement field. Thereafter, they further developed DrC-Net to further improve the performance of FOD image-based distortion correction through a deep learning-based approach. However, FODReg and DrC-Net simply concatenate FOD image-based methods after topup correction, failing to fully integrate the useful information in the two modal images. In addition, magnetic susceptibility distortion is ubiquitous in different dMRI-derived modality images, and using only a single modality image to learn the distortion field may lead to a decrease in generalization ability in other modality images. Summary of the invention

[0005] The purpose of the present invention is to provide a multimodal diffusion magnetic resonance imaging image distortion correction system to automatically correct the image distortion of dMRI images caused by magnetic susceptibility and improve the quality and accuracy of dMRI images.

[0006] The multimodal dMRI image distortion correction system provided by the present invention is based on unsupervised deep learning, and specifically includes: a coarse-to-fine deep learning network constructed - a multimodal distortion correction network (denoted as MMDC-Net), which is composed of cascaded rigid transformation and dense transformation networks; in addition, a cascade training strategy module is designed; wherein:

[0007] (1) The rigid transformation network is used to correct the linear misalignment between two phase encoding direction images. Specifically, it is a convolutional neural network, including an encoder, a multi-layer perceptron and a spatial transformation network; the encoder is responsible for extracting feature information from the spliced ​​B0 and FOD images, where the B0 and FOD images respectively include two images in opposite encoding directions. The encoder contains a total of four convolutional layers, each of which also includes a batch normalization layer. The feature maps output by each convolutional layer have 16, 16, 32, and 32 channels, respectively, and the sizes of the feature maps are 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the original image size, respectively. The subsequent multi-layer perceptron includes two consecutive fully connected layers, which map the feature maps learned by the encoder into a set of rigid transformation parameters μ:

[0008] μ=(θ x ,θ y ,θ z ,t x ,t y ,t z ), (1)

[0009] Among them, x, y, z are three coordinate directions, θ x ,θ y ,θ z Respectively represent the rotation angle of the corresponding coordinate axis, t x ,t y ,t z They represent the translation distance of the corresponding coordinate axis. According to the topup settings, the translation parameter t is set to 0 in phase encoding because it is impossible to distinguish the subject's translation from the deformation offset caused by magnetic susceptibility distortion in this direction.

[0010] According to the learned rigid transformation parameters μ, the geometric transformation matrix R can be calculated x , R y , R z :

[0011]

[0012] Finally, calculate the affine transformation matrix: Through the spatial transformer network, the learned affine transformation matrix is ​​applied to the B0 and FOD matrices of the negative phase to correct the linear misalignment between the two phase encoding direction images.

[0013] The spatial transformer network is an existing network module (Jaderberg M, Simonyan K, Zisserman A. Spatial transformer networks [J]. Advances in neural information processing systems, 2015, 28.), which is used to apply a rigid or non-rigid deformation field to an image for deformation processing;

[0014] (2) The dense transformation network is used to correct the non-rigid distortion of the image in two opposite coding directions. Specifically, a U-shaped structure neural network is constructed, which includes a dual-stream encoder, a decoder, a skip connection layer connecting the encoder and the decoder, and a multimodal attention module; the dual-stream encoder processes B0 and FOD information respectively: the B0 branch splices the B0 images of positive and negative phases into a 2-channel image as input; the FOD branch splices the FOD images of positive and negative phases into a 12-channel image as input. Each branch of the dual-stream encoder includes four convolution blocks, each of which consists of a 3×3×3 convolution layer and a LeakyReLU activation layer. By using a small convolution kernel and a convolution step size of 2, the encoder can gradually extract the deep information of the image while maintaining the spatial resolution.

[0015] After being processed by the encoder, the decoder uses several multimodal attention modules of different scales to fuse feature information from different modalities, thereby achieving more accurate distortion correction. Each multimodal attention module receives FOD and B0 feature information of different scales from the encoder and the output feature map of the decoder of the previous scale, aggregates and outputs the fused information of the corresponding scale and performs upsampling. In the final stage of the decoder, a 3D convolutional layer is used to estimate a three-dimensional non-rigid deformation field from the decoder features. This deformation field only retains the deformation in the phase encoding direction, thereby achieving magnetic susceptibility distortion correction.

[0016] Furthermore, the multimodal attention module, including a region attention module and a channel attention module, is used to aggregate feature information from B0 and FOD images in the dual-stream encoder during the decoding stage.

[0017] The regional attention module is the core component of the decoder, responsible for dynamically estimating and assigning attention weights to various regions of different modalities during the distortion correction process. This module focuses on the feature contribution of the same region in different modalities, and generates an attention map representing the contribution of different modal regions by capturing the latent features from the encoder and decoder. The module first concatenates the encoder and decoder feature maps of the same scale and feeds them into a convolutional layer to extract regional features. Then, the output dual-channel feature map is passed through a softmax layer to obtain an attention map, in which the two channels correspond to the contributions of B0 and FOD in each region. The attention map can be expressed as:

[0018]

[0019] in, is the attention map learned by the regional attention module, i represents the i-th layer of the network, m represents different modalities, including B0 and FOD images, [·;·] represents the concatenation operation, and e i and d i Represent the encoder and decoder feature maps respectively, f 3 ×3×3 represents a convolutional layer with a kernel size of 3×3×3, and M is a set of different modalities, namely {B0,FOD}.

[0020] Then, the learned feature map is multiplied by the input encoder feature per voxel to obtain the attention-weighted feature output

[0021]

[0022] The channel attention module is responsible for estimating the overall contribution of different modal inputs, and it dynamically adjusts the weight of each channel. Unlike the regional attention module, the channel attention module pays more attention to the contribution of the overall features rather than the feature weights of a single region. The module first concatenates the decoder features and the output of the regional attention module, and then generates two different feature vectors through average pooling and maximum pooling operations. These two vectors are then input into a multi-layer perceptron with shared weights. The output of the multi-layer perceptron passes through a sigmoid activation function, mapping the generated weight matrix to between 0 and 1, representing the contribution of different modal inputs to the output features in the current layer:

[0023]

[0024] Among them, σ represents the sigmoid activation function, f MLP represents a multilayer perceptron with shared weights, fr i and d iRepresents the output features of the regional attention module and the encoder in different scales, AvgPool(·) represents the average pooling layer, and MaxPool(·) represents the maximum pooling layer;

[0025] Finally, the channel attention module applies these weights to the feature maps output by the regional attention module, thereby adjusting the influence of different modal features on the correction effect:

[0026]

[0027] (3) The cascade training strategy module is used to improve the accuracy of estimating non-rigid deformation fields by a deep learning network, MMDC-Net, including using a dynamic weighted averaging (DWA) method to achieve simultaneous optimization of multimodal images.

[0028] For the dense transformation network, its input is the positive and negative phase FOD and B0 image pairs to be corrected, and the output is a three-dimensional deformation field. Using the spatial transformation module, the learned deformation fields are applied to the image pairs to be corrected respectively: the positive deformation field is applied to the positive phase B0 and FOD images to obtain the corrected positive phase image; the size of the deformation field remains unchanged, the direction is reversed to obtain the reverse deformation field applied to the negative phase B0 and FOD images to obtain the corrected negative phase image. Subsequently, the roughly corrected positive and negative phase B0 and FOD image pairs can be input into the dense transformation network again to learn the residual error between them to obtain a more refined deformation field. After a series of cascade operations, the difference between the images of the two phases tends to converge. Adding the deformation fields output by this series of networks can obtain the final deformation field of the original input image to be corrected:

[0029]

[0030] is the deformation field output by each sub-network, and n is the number of cascaded sub-networks.

[0031] The training method of MMDC-Net of the present invention is as follows:

[0032] Data preparation: Acquire positive and negative phase dMRI data, and preprocess the dMRI data using an eddy current correction algorithm; extract the B0 image from the dMRI data and normalize it, and calculate the FOD image from the dMRI data.

[0033] During training, the positive and negative phase B0 and FOD images are spliced ​​and input into the neural network respectively; the stochastic gradient descent strategy with momentum is used, and the learning rate is set to 0.0001. In order to ensure the convergence of the neural network, a total of 500 rounds are trained, one round includes 100 steps, and the batch size is set to 1. Unsupervised learning is adopted, and all sub-networks are cascaded and trained in an end-to-end manner. The optimization goal of training is the similarity of positive and negative phase B0 and FOD deformed images, so two similarity loss functions are required:

[0034] in, is the rigid body deformation field, is a dense deformation field, and is the deformed FOD image, and is the deformed B0 image, λ sim is a weight hyperparameter used to balance the similarity loss function between FOD and B0. Here, the similarity loss function of FOD images uses the mean square error; the similarity loss function of B0 images uses the cross-correlation function. For multimodal optimization tasks, the weight hyperparameter λ sim It is difficult to achieve a balance between the two due to random iterations during network training. Therefore, the dynamic weight average (DWA) algorithm is introduced in the optimization process to dynamically adjust the weights according to the iteration speed of different loss functions during training. Therefore, for k tasks, the weight λ k It can be defined as:

[0035]

[0036] Among them, t is the iteration coefficient, and T is the temperature parameter used to control the task weight. At the same time, in order to ensure the smoothness of the generated deformation field, a regularization term needs to be added:

[0037]

[0038] Therefore, the overall loss function is:

[0039]

[0040] Among them, λ 1 and λ 2 They are magnitude hyperparameters, which are used to balance the magnitude difference of the two similarity loss functions and are selected based on debugging / multiple experiments.

[0041] In the present invention, for data that need to be corrected for magnetic susceptibility distortion, the distortion field of dMRI data can be obtained mainly through one forward propagation, which greatly improves the efficiency of dMRI preprocessing.

[0042] The novel magnetic susceptibility-induced distortion correction system proposed in the present invention based on unsupervised learning designs a coarse-to-fine deep learning network, a multimodal distortion correction network, denoted as MMDC-Net, which consists of a rigid transformation and a dense transformation network. Unlike the existing methods that only use B0 or ​​fiber direction distribution (FOD) images, the present invention integrates the rich information of the two modalities and inputs it into the network. Specifically, the linear misalignment between two images encoded with opposite phases is first learned using a rigid transformation network. Then, a cascaded dense transformation network is designed to learn the distortion field from images of the two modalities through a dual-stream encoder and a multimodal attention module. In addition, the dynamic weighted average (DWA) method is used to achieve simultaneous optimization of multimodal images. During training, the input images are B0 and FOD images of positive and negative phases, and the output is the corresponding deformation field for correcting distortion. During reasoning, the B0 and FOD images of positive and negative phases in the dMRI image to be corrected are input, and the output is the deformation field. Experimental results show that the present invention can effectively correct the magnetic susceptibility distortion of dMRI, is significantly better than other contrast methods, effectively improves the efficiency of dMRI preprocessing, and improves the accuracy of subsequent analysis such as fiber bundle tracking and quantitative analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Susceptibility distortion correction process.

[0044] Figure 2 Architectural details of the distortion correction network.

[0045] Figure 3 Correction effects of different distortion correction methods in the brainstem region. DETAILED DESCRIPTION

[0046] The embodiments of the present invention are described in detail below, but the protection scope of the present invention is not limited to the embodiments.

[0047] The present invention adopts Figure 1 The magnetic susceptibility distortion correction process is shown in the figure. The specific process is:

[0048] During acquisition, the dMRI images of the same subject in both positive and negative phases are scanned, denoted as I + and I - Subsequently, the original dMRI images were eddy current corrected, and the FOD images of the positive and negative phases were estimated, so that the eddy current corrected B0 and FOD images were obtained as the input of the subsequent correction network.

[0049] During training, the B0 image and FOD image are first normalized to the maximum and minimum. Figure 2As shown in the figure, the B0 and FOD images are concatenated into [N, 2, H, W, T] and [N, 12, H, W, T] inputs in the channel dimension, where N is the batch size and H, W, and T are the three dimensions of the input image. For the FOD image, we use the first 6 channels to increase the feature information. The correction algorithm is trained using the ADAM optimizer with a learning rate of 2e-4.

[0050] like Figure 2 As shown, there are three loss functions during training, namely B0 similarity loss function L simB0 , FOD similarity loss function L simFOD and the deformation field regularization L reg . In addition, in order to balance the convergence speed of the two similarity losses, a dynamic weighted average algorithm DWAM was added to balance the convergence speed of the two loss functions through the temperature T. The weight parameter of the similarity loss was set to 1e-4, and the regularization parameter was set to 5e-3. The temperature T of the DWAM method was set to 2. During the training process, 400 epochs were used, each containing 100 steps. Finally, a trained neural network was obtained to correct the magnetic susceptibility distortion of dMRI.

[0051] During testing, the B0 and FOD images of positive and negative phases are obtained by the same preprocessing method as that of training. They are input into the neural network with frozen parameters to obtain a 3D deformation field. Subsequently, the deformation field is used to deform the positive phase dMRI image to obtain a positive phase correction image; the deformation field is reversed to deform the negative phase deformation field to obtain a negative phase correction image. Finally, the average of the positive and negative phase correction dMRI images is calculated to obtain a merged distortion-free image.

[0052] Table 1. Correction effects of different correction methods on HCP data

[0053]

[0054] Table 1 shows the correction effects of different correction methods on the HCP dataset. In order to quantify the distortion correction performance of the method of the present invention, two metrics are used to evaluate the differences between the dMRI images of the two phases after correction by different methods. The mean square deviation (MSD) of the fractional anisotropy (FA) generated by the dMRI data tensor model is measured. The minimum angle difference (MAD) of the main fiber direction of the FOD between two opposite PEs is also measured. The angle difference is obtained by calculating the minimum angle between the main fiber of the positive PE FOD (with the largest amplitude) and the first three fibers of the negative PE FOD (sorted by amplitude). In order to better meet the needs of practical applications, special attention is paid to local brain regions where significant residual distortion still exists after processing by other correction methods, including the brainstem region and the corpus callosum region. The brainstem region is crucial for studying the pathological changes of brain diseases such as Alzheimer's disease, and this region is usually severely distorted in dMRI images. Therefore, it is of great clinical significance to study the correction effect of this region, and the pons region of the brainstem is selected as the region of interest (ROI) in the experiment. The corpus callosum (cc) is the largest white matter pathway and nerve bundle in the brain and is usually significantly distorted in dMRI images. The correction effect of the corpus callosum is crucial for white matter tract fiber tracking. Therefore, the corpus callosum region was designated as the ROI. The brainstem and corpus callosum regions were segmented from the T1 images using Freesurfer 6.0, and then the pons region and corpus callosum region in the brainstem were linearly mapped to the dMRI image space through linear registration between T1 and dMRI images.

[0055] like Figure 3 As shown, the correction effects of different distortion correction methods topup, DrC-Net and the distortion correction method of the present invention in the brainstem area are shown. It can be seen from the figure that even if topup or DrC-Net correction is applied, the FOD fiber bundle trajectory generated from two opposite phase encoding directions is still irregular and there is a lot of noise, which may lead to subsequent erroneous analysis. In contrast, the method of the present invention can completely correct the distortion of the two phase images, and the fiber bundle trajectory in the brainstem area is clear and regular.

[0056] In addition to better correction effects, as a learning-based distortion correction method, the method of the present invention only takes about 3 seconds to correct a pair of dMRI data on a GPU, while topup and FODReg based on traditional iterative methods take more than an hour. In addition, DrC-Net first needs to perform a rough correction through topup, which takes extra time.

Claims

1. A multimodal dMRI image distortion correction system based on unsupervised deep learning, characterized in that: Specifically include: The coarse-to-fine deep learning network constructed is a multimodal distortion correction network, denoted as MMDC-Net, which consists of a cascaded rigid transformation network and a dense transformation network. In addition, it also includes a designed cascade training strategy module; among which: (1) The rigid transformation network is used to correct the linear misalignment between the two phase encoding direction images; the rigid transformation network is a convolutional neural network, including an encoder, a multi-layer perceptron and a spatial transformation network; the encoder is responsible for extracting feature information from the spliced ​​B0 and FOD images, where the B0 and FOD images respectively include two images in opposite encoding directions; the subsequent multi-layer perceptron maps the feature map learned by the encoder into a set of rigid transformation parameters μ: μ=(θ x ,i y ,i z ,t x ,t y ,t z ), (1) Among them, x, y, and z are three coordinate directions, θ x ,θ y ,θ z Respectively represent the rotation angle of the corresponding coordinate axis, t x ,t y ,t z They represent the translation distance of the corresponding coordinate axis respectively; according to the topup setting, the translation parameter t is set to 0 in phase encoding, because it is impossible to distinguish the translation of the subject from the deformation offset caused by magnetic susceptibility distortion in this direction; According to the learned rigid transformation parameters μ, the geometric transformation matrix R is calculated x , R y , R z : Finally, calculate the affine transformation matrix: A = R x °R y °R z °T; Apply the learned affine transformation matrix to the negative phase B0 and FOD matrices through the spatial transformer network to correct the linear misalignment between the two phase encoding direction images; (2) The dense transformation network is used to correct the non-rigid distortion of the image in two opposite coding directions; specifically, a U-shaped structure neural network is constructed, which includes a dual-stream encoder, a decoder, a jump connection layer connecting the encoder and the decoder, and a multimodal attention module; the dual-stream encoder processes B0 and FOD information respectively: the B0 branch splices the B0 images of positive and negative phases into a 2-channel image as input; the FOD branch splices the FOD images of positive and negative phases into a 12-channel image as input; the dual-stream encoder gradually extracts deep information of the image while maintaining spatial resolution; After being processed by the encoder, the decoder fuses feature information from different modalities through several multimodal attention modules of different scales to achieve more accurate distortion correction; each multimodal attention module receives FOD and B0 feature information of different scales from the encoder and the output feature map of the decoder of the previous scale, aggregates and outputs the fused information of the corresponding scale and performs upsampling; in the final stage of the decoder, a 3D convolutional layer is used to estimate a three-dimensional non-rigid deformation field from the decoder features; this deformation field only retains the deformation in the phase encoding direction, thereby achieving magnetic susceptibility distortion correction; (3) The cascade training strategy module is used to improve the accuracy of the deep learning network-MMDC-Net in estimating the non-rigid deformation field. For the dense transformation network, its input is the positive and negative phase FOD and B0 image pairs to be corrected, and the output is a three-dimensional deformation field; the spatial transformation module is used to apply the learned deformation field to the image pairs to be corrected respectively: wherein the positive deformation field is applied to the positive phase B0 and FOD images to obtain the corrected positive phase image; the size of the deformation field remains unchanged, the direction is reversed to obtain the reverse deformation field, which is applied to the negative phase B0 and FOD images to obtain the corrected negative phase image; then, the roughly corrected positive and negative phase B0 and FOD image pairs can be input into the dense transformation network again to learn the residual error between them to obtain a more refined deformation field; after a series of cascade operations, the difference between the images of the two phases tends to converge, and the deformation fields output by this series of networks are added to obtain the final deformation field of the original input image to be corrected: is the deformation field output by each sub-network, and n is the number of cascaded sub-networks.

2. The multimodal dMRI image distortion correction system according to claim 1, characterized in that: In the rigid transformation network: The encoder includes four convolutional layers, each of which also includes a batch normalization layer. The feature maps output by each convolutional layer have 16, 16, 32, and 32 channels respectively, and the sizes of the feature maps are 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the original image size respectively; The multilayer perceptron includes two consecutive fully connected layers; The spatial transformation network is used to apply a rigid or non-rigid deformation field to the image to perform deformation processing.

3. The multimodal dMRI image distortion correction system according to claim 1, characterized in that: In the dense transformation network, each branch of the dual-stream encoder includes four convolution blocks, each convolution block consists of a 3×3×3 convolution layer and a LeakyReLU activation layer; by using a small convolution kernel and a convolution step of 2.

4. The multimodal dMRI image distortion correction system according to claim 1, characterized in that: The multimodal attention module in the dense transformation network includes a regional attention module and a channel attention module, which is used to aggregate feature information from B0 and FOD images in the dual-stream encoder in the decoding stage; wherein: The regional attention module is responsible for dynamically estimating and allocating the attention weights of each region of different modalities during the distortion correction process; this module focuses on the feature contribution of the same region in different modalities, and generates an attention map representing the contribution of different modal regions by capturing the potential features from the encoder and decoder; in this module, the encoder and decoder feature maps of the same scale are first concatenated and sent to a convolutional layer to extract regional features; then, the output dual-channel feature map is passed through a softmax layer to obtain an attention map, in which the two channels correspond to the contributions of B0 and FOD in each region respectively; the attention map is expressed as: in, is the attention map learned by the regional attention module, i represents the i-th layer of the network, m represents different modalities, including B0 and FOD images, [·;·] represents the concatenation operation, and e i and d i Represent the encoder and decoder feature maps respectively, f 3×3×3 represents a convolutional layer with a convolution kernel size of 3×3×3; M is a set of different modalities, i.e., {B0,FOD}; Then, the learned feature map is multiplied by the input encoder feature per voxel to obtain the attention-weighted feature output The channel attention module is responsible for estimating the overall contribution of different modal inputs, and dynamically adjusts the weight of each channel; the channel attention module pays more attention to the contribution of the overall features; in this module, the decoder features and the output of the regional attention module are first concatenated, and then two different feature vectors are generated through average pooling and maximum pooling operations; then these two vectors are input into a multi-layer perceptron with shared weights; the output of the multi-layer perceptron is passed through a sigmoid activation function, and the generated weight matrix is ​​mapped to between 0 and 1, representing the contribution of different modal inputs to the output features in the current layer: Among them, σ represents the sigmoid activation function, f MLP represents a multilayer perceptron with shared weights, fr i and d i represents the output features of the regional attention module and the encoder in different scales, and AvgPool(·) represents the average pooling layer; Finally, the channel attention module applies these weights to the feature maps output by the regional attention module, thereby adjusting the influence of different modal features on the correction effect:

5. The multimodal dMRI image distortion correction system according to claim 1, characterized in that: In the cascade training strategy module, the training method of MMDC-Net is as follows: Data preparation: Acquire positive and negative phase dMRI data, preprocess the dMRI data using eddy current correction algorithm; extract B0 image from dMRI data and normalize it, and calculate FOD image from dMRI data; During training, the B0 and FOD images of positive and negative phases are spliced ​​and input into the neural network respectively; Using the stochastic gradient descent strategy with momentum and unsupervised learning, all sub-networks are cascaded and trained in an end-to-end manner; the optimization goal of the training is the similarity of positive and negative phase B0 and FOD deformed images. For this purpose, two similarity loss functions are used: in, is the rigid body deformation field, is a dense deformation field, and is the deformed FOD image, and is the deformed B0 image, λ sim is a weight hyperparameter used to balance the similarity loss functions of FOD and B0; here, the similarity loss function of FOD images uses the mean square error; the similarity loss function of B0 images uses the cross-correlation function.

6. The multimodal dMRI image distortion correction system according to claim 5, characterized in that: For multimodal optimization tasks, the weight hyperparameter λ sim It is difficult to achieve a balance between the two due to random iterations during network training. The dynamic weighted average (DWA) algorithm is introduced in the optimization process to dynamically adjust the weights according to the iteration speed of different loss functions during training. For k tasks, the weight λ k Defined as: Among them, t is the iteration coefficient, T is the temperature parameter used to control the task weight; at the same time, in order to ensure the smoothness of the generated deformation field, a regularization term needs to be added: Therefore, the overall loss function is: Among them, λ1 and λ2 are magnitude hyperparameters, which are used to balance the magnitude difference of the two similarity loss functions.

Images