Deep learning registration method for diffusion magnetic resonance fiber bundle imaging based on point cloud
Based on the deep learning standard method based on the point cloud, the encoder-decoder network and thin plate sample technology are used to solve the problem of the existing fiber beam imaging method. Fibrous beam imaging data.
Patent Information
- Application Number
- CN202411458888.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-18
- Publication Date
- 2025-01-21
AI Technical Summary
The existing streamline-based dMRI fiber bundle imaging registration method adopts traditional image registration formulas, resulting in high computing resources consumption, slow speed, and lack of deep learning methods to improve accuracy and speed.
Put forward a method of deep learning based on Dianyun, using the encoder-decoder network and thin plate sample technology to build a neural network fiber beam standard model, through feature extraction, multi-resolution feature stitching, displacement generation and other steps, etc. Realize fast and accurate fiber beam imaging data.
It has significantly improved the calculation efficiency and shortened the processing time, especially in the interpolation processing on the rules grid, realizing the fast and accurate fiber beam imaging data standard.
Smart Images
Figure CN119338874A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of medical image data processing, and in particular to a deep learning registration method for diffusion magnetic resonance fiber bundle imaging based on point cloud. Background Art
[0002] Diffusion magnetic resonance imaging (dMRI) tractography is an advanced medical imaging technique used to map brain white matter fiber tracts in volume. It can estimate the three-dimensional anatomical trajectory of nerve fiber paths, called streamlines. The registration of fiber tractography data is a key step in applications such as inter-group fiber tract quantification and fiber tract atlas construction.
[0003] Currently, such registration is usually achieved by applying a transformation after voxelization registration using volumetric images (such as scalar-valued fractional anisotropy FA). Meanwhile, many registration methods are designed to directly align fiber tract imaging streamlines. Compared with traditional volumetric registration methods, streamline-based registration methods are able to utilize the 3D geometric information of fiber paths, which has advantages in applications where the ultimate goal is to model and analyze white matter fiber tracts.
[0004] Currently, all existing streamline-based registration methods adopt the traditional image registration formulation, using an iterative optimization problem to align fiber tractography datasets through an energy function. Solving this optimization problem can consume a lot of computational resources and is therefore slow in practical applications.
[0005] In recent years, deep learning has shown significant improvement in both computational speed and registration accuracy in medical image registration. However, there is currently no deep learning method for streamline-based dMRI fiber bundle imaging registration.
[0006] Existing point cloud-based registration methods mainly focus on rigid registration, which is suitable for aligning objects in natural images, such as planes and scenes, but they are not sufficient for handling fiber bundles with highly variable geometric structures. Although recent research has made progress in non-rigid registration, point cloud-based methods often have non-smooth sampling results in areas with sparse points due to the lack of a regular grid. This is a limitation for bundle imaging because the neighbor relationship of points inside streamlines is predefined and needs to be consistently moved to avoid unexpected smooth deformations. Summary of the invention
[0007] In view of the above problems, the present invention proposes a point cloud-based diffusion magnetic resonance fiber bundle imaging deep learning registration method for quickly and accurately aligning dMRI fiber bundle imaging data.
[0008] The technical solution of the present invention is:
[0009] The deep learning registration method for diffusion magnetic resonance fiber bundle imaging based on point cloud includes the following steps:
[0010] S1. Extracting whole-brain fiber bundle imaging data based on diffusion magnetic resonance imaging, obtaining fiber bundle imaging streamlines to produce training data, each training data includes a pair of fiber bundles, each pair of fiber bundles includes a moving fiber bundle and a fixed fiber bundle;
[0011] S2. Construct a neural network fiber bundle registration model based on a deep learning network. The model consists of four modules in series: feature extraction unit, encoder, decoder, and displacement field generation unit. The model processes the training data by processing the moving fiber bundle and the fixed fiber bundle through the same feature extraction unit and encoder module to obtain their respective high-dimensional features at multiple resolutions. Then, the high-dimensional features are spliced at different resolutions, and a dense displacement field on the grid is generated through the decoder and the displacement field generation unit.
[0012] S3, using the training data to train the constructed model;
[0013] S4. Use the trained model to align the input fiber bundle imaging data.
[0014] The beneficial effect of the present invention is that the present invention introduces an encoder-decoder network based on deep learning and combines it with thin plate spline (TPS) technology for alignment, which greatly improves the computational efficiency, especially in the interpolation processing on a regular grid, and significantly shortens the processing time. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 The figure is a schematic diagram of the logical structure of the method of the present invention. DETAILED DESCRIPTION
[0016] The scheme of the present invention is further described below in conjunction with the accompanying drawings.
[0017] like Figure 1 As shown, the scheme of the present invention generally includes the following steps:
[0018] Step 1: Extract whole-brain fiber bundle imaging data based on diffusion magnetic resonance imaging, and extract 1,000 fiber bundle imaging streamlines in each brain as training data.
[0019] Step 2: Construct a neural network fiber bundle registration model based on a deep learning network: The network consists of coarse-to-fine encoder-decoder modules. For a pair of input fiber bundles (moving, fixed), the same network is used for processing and feature extraction, and the displacement field on the standard grid is predicted by the TPS algorithm.
[0020] Step 3: Train the neural network model; use the training sample set prepared in step 1 to train the network, calculate the error with the semantic segmentation annotation, perform error back propagation, and update the model parameters.
[0021] Step 4: Input the fiber bundle pair to be registered into the model for forward reasoning and automatically output the registration displacement field.
[0022] The specific process of constructing the dataset is as follows:
[0023] 100 unrelated individuals were randomly selected from the Human Connectome (HCP) dataset, 80 of which were randomly selected as training data, 10 as validation data, and the other 10 as test data. Fiber bundle streamlines were extracted from the original diffusion magnetic resonance images in each segmented dataset.
[0024] The following is a detailed explanation of the specific construction process of the model with the help of illustrations:
[0025] The main body of the model consists of four modules connected in series: feature extraction unit, encoder, decoder, and displacement field generation unit. The functions of each module and its input and output are described in detail below.
[0026] 1) Feature Extraction Unit: Its purpose is to extract high-dimensional features of the initial point cloud. This module reads the properties of the moving and fixed fiber bundle point clouds and extracts high-dimensional features from the input point cloud 3D coordinates through a series architecture of linear layers and linear rectification functions.
[0027] 2) Encoder module: Its purpose is to perform multi-resolution feature extraction on the high-dimensional features of the point cloud extracted by the feature extraction module. For this purpose, 4 groups of U-Net networks are designed to extract features from point clouds at different resolutions. For one of the resolutions, the high-dimensional features of the point cloud are first downsampled to a low resolution by sampling the farthest point. This method can obtain fiber bundle geometric features of different resolutions while maintaining the basic geometric shape of the brain. At this resolution, a higher-dimensional feature representation is learned based on a multi-layer perceptron containing two linear layers. In order to prevent information loss during feature extraction, the jump connection-splicing mechanism in U-Net is borrowed. Specifically, the higher-dimensional features learned at low resolution are raised to high resolution through upsampling operations. The features at high resolution are reduced in dimension through another linear layer, and the features are spliced with the upsampled features, in order to combine the global information captured at low resolution with the local details captured at high resolution, thereby obtaining a richer and more comprehensive feature representation.
[0028] 3) Decoder module: Its purpose is to decode the sparse velocity field. The high-dimensional features of the point clouds with different resolutions of the moving fiber bundles and the fixed fiber bundles are spliced to generate velocity fields of different resolutions from coarse to fine. Specifically, based on the PointPWC framework. The decoding module adds the upsampled low-resolution displacement field to the moving point cloud coordinates to obtain the preliminary aligned moving point cloud coordinates. Based on the coordinate distance of the point cloud pair, each point in the moving point cloud is matched with several adjacent points in the fixed point cloud, and the features and relative position vectors of the two are spliced as the input of the cost volume estimation module. The cost volume estimation module predicts the cost volume based on the series architecture of linear layers and linear rectification functions. Finally, the calculated cost volume is spliced with the upsampled low-resolution displacement field, and the sparse velocity field is calculated based on another multi-layer perceptron.
[0029] 4) Displacement field generation module: Generates a dense displacement field on the grid based on the sparse velocity field decoded upstream. This displacement field can be applied to data of different properties such as point clouds, streamlines, and images. Generating a displacement field on the grid can be further decomposed into the following two steps:
[0030] 4.1 Generate a dense velocity field on a standard grid. In order to solve the sparsity problem of the deformation field, map the deformation field to a regular grid, and improve the interpolation efficiency, we designed a thin plate spline densification module. In the actual implementation, we use the thin plate spline (TPS) algorithm to estimate the non-rigid and smooth transformation. By applying this transformation to the coordinates of a regular grid, we obtain the velocity field on the regular grid. By implementing the TPS interpolation module, a smooth velocity field on a regular grid is achieved.
[0031] 4.2 Generate a displacement field with diffeomorphic properties. In order to maintain the diffeomorphic properties of the transformation, we will predict the velocity field and obtain the complete displacement field by integration. To achieve this, the relevant modules can be implemented through spatial transformation functions.
[0032] The process of training the fiber bundle registration model in the solution of the present invention is as follows:
[0033] 1. Tracing fiber bundle imaging streamlines from diffusion magnetic resonance imaging, randomly sampling 1,000 whole-brain fiber bundles to generate training set data.
[0034] 2. Extract the point cloud data from the fiber bundle imaging streamline and input it into the network.
[0035] 3. Add the predicted displacement field to the coordinates of the moving fiber bundle, calculate the Chamfer distance loss and line alignment distance loss with the fixed fiber bundle, and calculate the displacement field regularization loss. Back-propagate the sum of the loss values, update the network parameters, and obtain a trained registration network.
[0036] The Chamfer distance loss is: for each p(q) point in the point cloud P(Q), find its nearest point q(p) in Q(P), calculate their distances and add them up.
[0037]
[0038] The line alignment distance is, for each fiber streamline in the fiber bundle Sa (Sb), the shortest distance between it and the fiber streamline in another fiber bundle is calculated, which is the smaller of the direct distance and the reverse distance.
[0039]
[0040] in,
[0041]
[0042] The regularization loss is the 2-norm of the gradient field of the displacement field, thereby ensuring the smoothness of the displacement field.
Claims
1. A deep learning registration method for diffusion magnetic resonance fiber bundle imaging based on point cloud, characterized in that: The following steps are involved: S1. Extract the fiber bundles of the whole brain based on diffusion magnetic resonance imaging, obtain the fiber bundle streamlines to produce training data, each training data includes a pair of fiber bundles, each pair of fiber bundles includes a moving fiber bundle and a fixed fiber bundle; S2. Construct a neural network fiber bundle registration model based on a deep learning network. The model is composed of four modules in series: feature extraction unit, encoder, decoder, and displacement field generation unit. The model processes the training data as follows: After the training data is input into the feature extraction unit, the feature extraction unit obtains the point cloud attributes of the moving fiber bundle and the fixed fiber bundle, and extracts high-dimensional features from the input point cloud three-dimensional coordinates; The encoder uses 4 groups of U-Net networks to extract features at different resolutions from the acquired high-dimensional features; The decoder concatenates the high-dimensional features of point clouds with different resolutions and generates velocity fields with different resolutions from coarse to fine. The displacement field generation unit generates a dense displacement field on the grid based on the sparse velocity field decoded upstream; S3. Use the training data to train the constructed model, specifically, add the predicted displacement field to the coordinates of the moving fiber bundle, calculate the Chamfer distance loss and the line alignment distance loss with the fixed fiber bundle, and calculate the displacement field regularization loss, back-propagate the sum of the obtained loss values, update the network parameters, and finally obtain the trained registration network; S4. Use the trained model to align the input fiber bundle imaging data.