Diffusion magnetic resonance fiber bundle imaging fiber reconstruction method based on deep learning

Abstract

The invention belongs to the technical field of medical image processing, and particularly relates to a diffusion magnetic resonance fiber bundle imaging fiber reconstruction method based on deep learning. The diffusion model is applied to fiber bundle imaging, distribution of fiber directions is directly predicted through the diffusion model, and the limitation that a traditional probabilistic tracking method depends on specific sampling of a unit spherical surface is overcome. In addition, the model is combined with the recurrent neural network, the front-back dependency relationship in the fiber bundle sequence can be effectively captured, and therefore the modeling capacity of the complex fiber path is remarkably improved. In order to further enhance the accuracy and robustness of fiber direction prediction, the model additionally introduces dispersion information of adjacent voxels based on the characteristics of fiber bundle imaging, the extraction of local features is enriched, and the prediction is more accurate and reliable.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and specifically is a diffusion magnetic resonance fiber bundle imaging fiber reconstruction method based on deep learning. Background Art

[0002] Fiber tract imaging can reveal the paths of brain white matter fibers and their potential structural connections. As the only method that can reconstruct brain white matter connections in vivo on a macroscopic scale, it has important application value.

[0003] Traditional fiber bundle imaging relies on complex physical models and tracking algorithms, such as diffusion tensor imaging (DTI), spherical deconvolution, and Q-ball imaging. However, with the advancement of machine learning technology, fiber bundle imaging methods based on machine learning have gradually attracted attention. The more well-known models include the Learnto Track model published at the MICCAI conference in 2017 and the DeepTract model in 2019, which predict the direction of fiber bundles point by point through recurrent neural networks (RNN). In addition, the Track-to-Learn model published in the journal Medical Image Analysis in 2021 uses a reinforcement learning method to enable the agent to learn and optimize tracking strategies through trial and error.

[0004] Although these methods have made some progress, they still have some shortcomings: (1) The physical model is sensitive to data noise and cannot fully represent the actual fiber arrangement in complex structures; (2) When introducing a probabilistic model for tracking, it is usually necessary to perform specific sampling on the unit sphere, and changes in the number of sampling points will directly affect the tracking effect; (3) The model is difficult to train, and its performance is highly dependent on the design of the reward function. Summary of the invention

[0005] In response to the above problems, the present invention proposes a diffusion magnetic resonance fiber bundle imaging fiber reconstruction method based on deep learning.

[0006] The technical solution of the present invention is:

[0007] A diffusion magnetic resonance fiber bundle imaging fiber reconstruction method based on deep learning, comprising the following steps:

[0008] S1. Construct training data, specifically to obtain diffusion magnetic resonance images, brain masks and fiber bundles, use brain masks to preprocess the diffusion magnetic resonance image data, filter out irrelevant areas outside the brain, resample the diffusion magnetic resonance image data to the specified magnetic field direction, and finally convert the fiber bundle data to the voxel space of the diffusion magnetic resonance image; obtain the direction of each point on the streamline by calculating the discrete difference method, and combine the diffusion information of the voxel where the point is located and the seven neighboring voxels above, below, left, right, front and back to generate the feature data required for training. The dimension of the feature data x is Sequence×7×gradients; obtain the direction of each point on each streamline of the fiber bundle by calculating the discrete difference method, and normalize the direction to obtain the dimension of the benchmark data y is Sequence×3;

[0009] S2. Construct a fiber reconstruction network model, including a convolution module, an LSTM module, a linear layer and a diffusion module connected in sequence. The convolution module includes three one-dimensional convolution layers. After each convolution operation, a ReLU activation function is connected to extract the correlation features of seven neighboring voxels. The LSTM module is a three-layer LSTM network. The hidden state of each LSTM layer is transmitted in sequence and finally transmitted to the linear layer. The output of the linear layer is used as the context feature information of the diffusion module. During the model training process, the noise data generated by gradually adding noise to the baseline data is used as the input feature of the diffusion module; the diffusion module is composed of six MFL modules in series. The MFL module combines the input features and context features, adjusts the input features through weighting and bias, and finally generates output. The forward diffusion process of the diffusion module is as follows: First, given the input feature (i.e., the direction of the fiber bundle), random noise is added to the input feature according to a series of diffusion time steps T to perturb it, and noise data is generated by gradually adding noise. Subsequently, the diffusion module receives the noise data of each diffusion time step, the corresponding diffusion time step T and the context feature, and generates an estimated value represents the denoising direction; at the same time, the difference between the estimated value of each diffusion time step and the noise data is used to calculate the added noise estimate; the diffusion module calculates two errors based on the denoised prediction result and the actual noise disturbance: one is the error between the predicted direction and the true direction, and the other is the error between the predicted noise and the actual noise; the cosine distance is used to calculate the loss of the two parts of the error, and the time step information is used as the weight to adjust the ratio of the noise error to the direction error;

[0010] S3, using the training data constructed in step S1, the network model designed in step S2 is trained, the number of cycles K is set, the batch size is B, and the quantization parameters of the network are updated through the back propagation algorithm. When the loss of the test set fails to continue to decrease in 20 consecutive iterations, the training stops, and the final trained reconstruction network model is obtained;

[0011] S4, obtaining target data, including diffusion magnetic resonance images, brain masks and white matter masks, is different from the fiber bundles used in the S1 stage. The fiber bundles in the S1 stage are mainly used to calibrate the benchmark, that is, the actual structural distribution of the fiber bundles. In this stage, the white matter mask is chosen mainly to generate the initial seed points. At the same time, the generated fiber bundles are regionally constrained to ensure the rationality and accuracy of the generated path. The diffusion MRI data are preprocessed using a brain mask to filter out irrelevant areas outside the brain, and then the diffusion MRI data are resampled to the specified magnetic field direction; an initial seed point is randomly generated in each voxel in the white matter mask area as the starting point for fiber tracking, and the diffusion information of each seed point and its seven surrounding voxels is input into the trained reconstruction network model. After feature extraction by the convolutional neural network and the recurrent neural network, the direction of the fiber bundle is predicted through the reverse sampling process of the diffusion module. The position of the seed point in space is updated based on the direction predicted by the model and the preset step size; after the forward tracking is completed, the model will also perform reverse tracking, starting from the second seed point and continuing to predict the fiber direction in the opposite direction of the forward tracking; forward and reverse tracking are combined to generate a complete fiber bundle trajectory.

[0012] The beneficial effect of the present invention is that the present invention applies the diffusion model to fiber bundle imaging, and directly predicts the distribution of fiber directions through the diffusion model, thus overcoming the limitation of relying on specific sampling of the unit sphere in the traditional probabilistic tracking method. In addition, the model combines a recurrent neural network, which can effectively capture the front-to-back dependencies in the fiber bundle sequence, thereby significantly improving the modeling ability of complex fiber paths. In order to further enhance the accuracy and robustness of fiber direction prediction, the model also introduces additional diffusion information of neighboring voxels based on the characteristics of fiber bundle imaging, enriches the extraction of local features, and makes the prediction more accurate and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram of the process of the present invention.

[0014] Figure 2 It is a schematic diagram of the network model structure of the present invention.

[0015] Figure 3 Schematic diagram of the diffusion model noise network structure. DETAILED DESCRIPTION

[0016] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0017] Step 1: Dataset construction: First, obtain the original data of each subject, including diffusion magnetic resonance images (DWI), brain masks, and fiber bundle files. Use the brain mask to preprocess the DWI data to filter out irrelevant areas outside the brain, ensuring that only valid information inside the brain is retained in the data set. Next, resample the DWI data to a specific magnetic field direction (for example, using the directional sampling scheme of repulsion100) to ensure consistent directional information. After that, the fiber bundle data is converted to the voxel space of DWI so that each point on the fiber bundle can accurately correspond to a specific voxel position of DWI.

[0018] On this basis, the direction of each point on the streamline is obtained as the benchmark data by calculating the discrete difference method. Then, the feature data required for training is generated by combining the diffusion information of the voxel where the point is located and its seven neighboring voxels above, below, left, right, front and back. Since the lengths of different streamlines may be different, in order to ensure the uniformity of the data format, the padding method is used to process all streamlines to make their lengths consistent. The training data set finally generated includes feature data and benchmark data: the dimension of the feature data x is Sequence×7×gradients, that is, each point of each streamline has a feature vector related to its gradient information; the benchmark data y is Sequence×3, that is, the direction vector of each point. The data set constructed in this way can be used for the training of the fiber bundle imaging model. Finally, the data set is divided into a training set and a test set in a ratio of 8:2.

[0019] Step 2: Build the network model: Figure 2 As shown in the figure, first, the data passes through three one-dimensional convolutional layers (Conv1D) in sequence, each with a kernel size of 3 and padding of 0. After each convolution operation, a ReLU activation function is applied to extract local features between spatially adjacent voxels. The convolutional features are input into a three-layer LSTM network, and the hidden state of each LSTM layer is passed in turn. The final output is passed to the linear layer, as shown in the figure. Figure 1 As shown, the LSTM module uses the hidden state h of the previous time step t-1 And the current information, calculate the hidden state h of the next time step t The output of the linear layer is used as the context feature information of the diffusion module. During the model training process, the benchmark data is used as the input feature of the diffusion module. The noise prediction network of the diffusion module is shown in the attached figure. Figure 3 As shown on the left side (a), it is composed of six MFL modules connected in series. The specific structure of the MFL module is shown in the attached figure. Figure 3 As shown in the right side (b), by combining the input features and context features, and adjusting the input features through weighting and bias, the output is finally generated. The forward diffusion process of the diffusion module is as follows:

[0020] As attached Figure 1 As shown in the forward diffusion process, given the input features (i.e., fiber bundle direction), random noise is added to the baseline data according to a series of diffusion time steps T to perturb the baseline data, and noise data is generated step by step. In the "data zeroing" stage, the analytical To gradually attenuate the original fiber distribution direction, gradually attenuating it to zero distribution in the time step. In this invention, we use a constant function Ensure that the input features transition smoothly to the zero distribution; next, in the "zero to noise" stage, normal noise is gradually added to the zero data The zero distribution gradually evolves into pure noise data Subsequently, the diffusion module receives the noise data of each diffusion time step, the corresponding diffusion time step T and the context features, and generates an estimate represents the denoising direction; at the same time, the difference between the estimated value of each diffusion time step and the noise data is used to calculate the estimated value of the added noise; the model calculates two errors based on the denoised prediction results and the actual noise disturbance: one is the error between the predicted direction and the true direction, and the other is the error between the predicted noise and the actual noise; the cosine distance is used to calculate the loss of the two parts of the error, and the time step information is used as the weight to adjust the ratio of the noise error to the direction error.

[0021] Step 3: Model training: Use the training data constructed in step 1 to train the network model designed in step 2. Set the number of cycles K, the batch size B, and update the network's quantization parameters through the back-propagation algorithm. When the loss of the test set fails to continue to decrease in 20 consecutive iterations, the training stops and the final trained network model is obtained.

[0022] Step 4: Fiber bundle imaging: First, load the model weights trained in step 3, initialize and prepare the neural network model for fiber bundle direction tracking. Similar to step 1, obtain the relevant data of the subject, including diffusion magnetic resonance imaging (DWI), brain mask and white matter mask, and preprocess the DWI data through the brain mask to filter out the extra-brain area, and resample the DWI data to a specific magnetic field direction to ensure the consistency of the data direction. Different from the fiber bundles used in step 1, the fiber bundles in step 1 are mainly used to calibrate the benchmark, that is, the actual structural distribution of the fiber bundles, while the white matter mask is chosen at this stage mainly to generate the initial seed points, and at the same time, the generated fiber bundles are regionally constrained to ensure the rationality and accuracy of the generated path.

[0023] Next, an initial seed point is randomly generated in each voxel in the white matter mask area as the starting point for fiber tracking. The model uses the diffusion information of each seed point and its seven surrounding voxels as input. After these input data are extracted by convolutional neural networks and recurrent neural networks, the direction of the fiber bundle is predicted through the reverse sampling process of the diffusion module. Figure 1 As shown, at a certain time step, a sub-point p t , according to the direction predicted by the model Combined with the preset step size (usually equal to the voxel size), according to the formula Calculate and update the seed point position p for the next time step t+1 .

[0024] This process is repeated, and the model is updated with each step, using the new position and information from neighboring voxels to predict the next fiber direction. Tracking continues until any of the following conditions is met: (1) the angle between the fiber directions changes by more than 60 degrees; (2) the fiber bundle crosses the white matter mask area; (3) the model's direction prediction stops.

[0025] After the forward tracking is completed, the model also performs reverse tracking. Reverse tracking starts from the second seed point and continues to predict the fiber direction in the opposite direction of the forward tracking. Combining forward and reverse tracking, the model can generate a complete fiber bundle trajectory. During the fiber bundle generation process, the model filters the trajectory based on the streamline length and image resolution, filtering out fiber bundles with a length of less than 20 mm or greater than 260 mm to ensure that the generated trajectory is physically and physiologically reasonable and meets the expected length requirements.

[0026] The reverse sampling process of the diffusion module is as follows:

[0027] As attached Figure 1 As shown in the reverse sampling process, the initial noise sample is first randomly sampled from the standard normal distribution Diffusion starts from the initial diffusion time step T and gradually iterates towards the diffusion time step t = 0. At each diffusion time step, the diffusion module takes the sample x of the current diffusion time step as t As input features, the output of the linear layer is used as context features to calculate the intermediate data at the current moment and noise component Then, based on the predicted intermediate data and the noise component, the sample x of the next diffusion time step t-Δt is calculated t-Δt As the diffusion time step progresses, the module gradually reduces the noise in the sample and gradually approaches the original data distribution. When the iteration reaches the diffusion time step t = 0, the generated sample x 0 The original target data distribution is approximated and the fiber bundle directions are sampled.

Claims

1. A diffusion magnetic resonance fiber bundle imaging fiber reconstruction method based on deep learning, characterized in that: The following steps are involved: S1. Construct training data, specifically, obtain diffusion magnetic resonance images, brain masks, and fiber bundles, use brain masks to preprocess the diffusion magnetic resonance image data, filter out irrelevant areas outside the brain, resample the diffusion magnetic resonance image data to the specified magnetic field direction, and finally convert the fiber bundle data to the voxel space of the diffusion magnetic resonance image; The direction of each point on the streamline is obtained by calculating the discrete difference method, and the feature data required for training is generated by combining the diffusion information of the voxel where the point is located and the seven neighboring voxels above, below, left, right, front and back. The dimension of the feature data x is Sequence×7×gradients. The direction of each point on each streamline of the fiber bundle is obtained by calculating the discrete difference method, and the direction is normalized to obtain the dimension of the benchmark data y is Sequence×3. S2. Construct a fiber reconstruction network model, including a convolution module, an LSTM module, a linear layer and a diffusion module connected in sequence, wherein the convolution module includes three one-dimensional convolution layers, and each convolution operation is followed by a ReLU activation function to extract the correlation features of seven neighboring voxels; the LSTM module is a three-layer LSTM network, and the hidden state of each LSTM layer is transmitted in sequence and finally transmitted to the linear layer; the output of the linear layer is used as the context feature information of the diffusion module; during the model training process, the noise data generated by gradually adding noise to the benchmark data is used as the input feature of the diffusion module; The diffusion module consists of six MFL modules connected in series. The MFL module combines the input features and context features, adjusts the input features through weighting and bias, and finally generates output. The forward diffusion process of the diffusion module is as follows: first, given the input features, random noise is added to the input features according to the diffusion time step T to perturb them, and noise data is generated step by step; then, the diffusion module receives the noise data of each diffusion time step, the corresponding diffusion time step T and context features, and generates an estimated value Represents the denoising direction; at the same time, the added noise estimate is calculated using the difference between the estimated value and the noise data at each diffusion time step; The diffusion module calculates two errors based on the denoised prediction results and the actual noise disturbance: one is the error between the predicted direction and the true direction, and the other is the error between the predicted noise and the actual noise. The cosine distance is used to calculate the loss of the two parts of the error, and the time step information is used as the weight to adjust the ratio of the noise error to the direction error. S3, using the training data constructed in step S1, the network model designed in step S2 is trained, the number of cycles K is set, the batch size is B, and the quantization parameters of the network are updated through the back propagation algorithm. When the loss of the test set fails to continue to decrease in 20 consecutive iterations, the training stops, and the final trained reconstruction network model is obtained; S4, obtaining target data, including diffusion magnetic resonance images, brain masks and white matter masks, which are different from the fiber bundles used in the S1 stage. The fiber bundles in the S1 stage are used to calibrate the benchmark, that is, the actual structural distribution of the fiber bundles. The white matter mask is used to generate initial seed points and impose regional constraints on the generated fiber bundles. The diffusion MRI data are preprocessed using a brain mask to filter out irrelevant areas outside the brain, and then the diffusion MRI data are resampled to the specified magnetic field direction; an initial seed point is randomly generated in each voxel in the white matter mask area as the starting point for fiber tracking, and the diffusion information of each seed point and its seven surrounding voxels is input into the trained reconstruction network model. After feature extraction by the convolutional neural network and the recurrent neural network, the direction of the fiber bundle is predicted through the reverse sampling process of the diffusion module. The position of the seed point in space is updated based on the direction predicted by the model and the preset step size; after the forward tracking is completed, the model will also perform reverse tracking, starting from the second seed point and continuing to predict the fiber direction in the opposite direction of the forward tracking; forward and reverse tracking are combined to generate a complete fiber bundle trajectory.

Images