A brain magnetic resonance image registration method based on self-supervised learning

Abstract

The application discloses a brain magnetic resonance image registration method based on self-supervised learning, which fully excavates the features of the image itself, including image semantic features, image high-level features and image low-level features, and effectively solves the problems of slow speed and complicated process of traditional brain magnetic resonance registration methods. The application can better comprehensively and quantitatively analyze the brain structure, or be used for subsequent other related tasks. Compared with the existing brain magnetic resonance image registration method using fsl, ants and other tools, the application can greatly improve the registration speed and efficiently complete the registration task when facing large quantities of registration requirements. Meanwhile, the application combines the self-supervised learning method, can effectively enable the model to learn the characteristics of the brain magnetic resonance image itself, and complete the iteration and training process of the model in combination with the semantic information, so that the model can fully retain the features of the original image, and researchers can more conveniently quantitatively analyze the brain structure.

Description

Technical Field

[0001] This invention relates to a method for registering brain magnetic resonance images based on self-supervised learning, belonging to the field of brain magnetic resonance image processing. Background Technology

[0002] Brain MRI images utilize an external high-frequency magnetic field to generate signals from the energy emitted by substances within the brain towards the surrounding environment. Compared to CT images, they are safer, harmless, and provide clearer structural images. Brain MRI images are frequently used to observe whether there are lesions in a patient's brain and whether there are structural changes in intracranial tissues, with a very high diagnostic accuracy for certain specific diseases. However, during the brain MRI imaging process, variations in patient head movement, comfort positions, and the effects of different times and imaging devices can lead to varying levels of noise and distortion in the brain MRI images. These differences make comprehensive quantitative analysis of diseases difficult. Therefore, registering brain MRI images and matching them to a standard space is of great significance in eliminating the influence of noise and distortion.

[0003] Current methods for registering brain MRI images mostly rely on publicly available registration tools, which are time-consuming. Furthermore, most research on image registration is limited to traditional methods, with limited research on deep learning approaches, making it difficult to meet the demands of large-scale data registration. For example, Chinese patent CN107133946A utilizes unsupervised learning to register medical images. While it doesn't employ deep learning methods for processing large datasets more quickly, it still improves registration accuracy. Chinese patent CN109993709A provides a deep learning-based image registration error correction method. Although it doesn't apply deep learning to brain MRI registration, it significantly improves the accuracy of multi-image segmentation results. Summary of the Invention

[0004] The purpose of this invention is to register brain MRI images and match them to a standard space.

[0005] To achieve the above objectives, the technical solution of the present invention provides a brain magnetic resonance imaging registration method based on self-supervised learning, characterized by the following steps:

[0006] Step 1, Pre-task process: Design the pre-tasks for the brain registration self-supervised learning method and construct the training dataset, specifically including the following steps:

[0007] Step 101: Prepare a publicly available standard dataset of brain MRI images. Select any image from the standard dataset of brain MRI images as the current standard data image.

[0008] Step 102: Randomly generate the perturbation affine matrix A gt Then we have:

[0009]

[0010] Perturbation affine matrix A gt The element in the i-th row and j-th column is represented as a ij a ij For the perturbations randomly generated on different channels, represent the transformations in four degrees of freedom (translation, rotation, scaling, and tilt) on each channel;

[0011] Step 103: Randomly generate the nonlinear perturbation matrix B gt Then we have:

[0012]

[0013] Nonlinear perturbation matrix B gt The element in the i-th row is represented as b i b i For nonlinear perturbations randomly generated on different channels;

[0014] Step 104: Calculate the brain MRI image after adding random perturbation to the current image, based on the following formula (1) for superimposing perturbation:

[0015]

[0016] In equation (1), x, y, and z are the three channels of the current standard data image, and x', y', and z' are the three channels of the brain MRI image after perturbation.

[0017] Step 105: Perform interpolation calculations on the brain MRI images after perturbation to eliminate the influence of the transformed coordinates being decimals;

[0018] Step 106: Traverse all standard data images in the standard brain MRI dataset, repeat steps 101 to 105, and obtain the perturbation-added brain MRI images corresponding to all standard data images.

[0019] Step 107: Divide all the perturbated brain MRI images into training datasets, validation datasets, and test datasets for subsequent training.

[0020] Step 2, Self-Supervised Learning Model Training Process: Design and train a brain registration self-supervised learning model, which includes the following steps:

[0021] Step 201: Obtain the training dataset;

[0022] Step 202: Build a brain registration self-supervised learning model;

[0023] Step 203: Input the training set data into the constructed brain registration self-supervised learning model. Through feature extraction, fully connected dimensionality reduction, and regression calculation, obtain a 1×15 dimension vector pre. 15 =[p1,p2,……,p 15 ], where p1, P2, ..., p 12 The affine transformation matrix A that forms the prediction pre And by p 13 p 14 p 15 Then the nonlinear bias matrix B is formed. pre That is:

[0024]

[0025]

[0026] Step 204: The randomly generated perturbation affine matrix A... gt and nonlinear perturbation matrix B gt The perturbation vector Gt is obtained by stretching the feature to a 1×15 dimension. 15 Then we have Gt 15 =[a 11 ,a 12 ...a 34 [b1,b2,b3], construct a loss function and calculate the vector pre predicted by the brain registration self-supervised learning model based on the loss function. 15 With randomly added perturbation vector Gt 15 The loss value between the two is defined by the loss function as shown in equation (2):

[0027] Loss = (Pre 15 -Gt 15 ) 2 / 15 (2)

[0028] Step 205: Train the brain registration self-supervised learning model through forward propagation and backward propagation until the loss value no longer decreases, that is, the brain registration self-supervised learning model training is complete.

[0029] Step 3: Preprocessing of images to be registered: Perform appropriate preprocessing on the brain MRI images to be registered;

[0030] Step 4: Image registration process: Use a well-trained brain registration self-supervised learning model to register the brain MRI image after the preprocessing in Step 3 to the standard space.

[0031] Preferably, in step 202, the brain registration self-supervised learning model is built based on the deep neural network model, wherein the 2D convolution and pooling operations in the deep neural network model are changed to 3D convolution and pooling, and its last fully connected layer is changed to a size of 1×15, so that it corresponds to the 12 degrees of freedom of affine transformation and the 3-dimensional bias of nonlinear perturbation, respectively.

[0032] Preferably, step 3 specifically includes the following steps:

[0033] Step 301: Input the brain MRI image to be registered;

[0034] Step 302: Convert the captured raw IMA format image into a Nii format image.

[0035] Step 303: Swap axes to align the brain with the standard axis.

[0036] Step 304: Perform bias field correction on the brain MRI images processed in step 303;

[0037] Step 305: Remove redundant parts such as the neck and face from the brain MRI image, and remove the skull so that it can be input into the brain registration self-supervised learning model.

[0038] Preferably, step 4 specifically includes the following steps:

[0039] Step 401: Prepare the brain MRI images after the preprocessing in Step 3;

[0040] Step 402: Input the brain MRI image into the brain registration self-supervised learning model to obtain the 1×15 dimension feature vector predicted by the brain registration self-supervised learning model. And these form the corresponding affine transformation matrix for registration. and nonlinear bias matrix Then we have:

[0041]

[0042]

[0043] Step 403: Perform brain MRI image registration according to the following formula (3):

[0044]

[0045] In equation (3), These are the three channels of the registered brain MRI image, corresponding to x... org y org z org The input brain MRI image to be registered has three channels;

[0046] Step 404: Interpolate the registered brain MRI images to eliminate the influence of decimal coordinates.

[0047] This invention utilizes a self-supervised learning method to fully mine the inherent features of images, including semantic features, high-level features, and low-level features, effectively solving the problems of slow speed and cumbersome procedures in traditional brain MRI registration methods. This invention enables better comprehensive quantitative analysis of brain structure or can be used for other related tasks.

[0048] Compared to existing methods using tools such as FSL and ANS for brain MRI image registration, this invention significantly improves registration speed, enabling efficient completion of registration tasks when faced with large-scale registration demands. Furthermore, this invention incorporates self-supervised learning, effectively allowing the model to learn the inherent characteristics of brain MRI images and integrate their semantic information to complete the model's iteration and training process. This ensures the model fully preserves the features of the original images, facilitating quantitative analysis of brain structures by researchers. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating the overall process of the brain magnetic resonance image registration method based on self-supervised learning according to the present invention.

[0050] Figure 2 This is a flowchart of the preliminary tasks for this invention;

[0051] Figure 3 This is a flowchart of the training process for the self-supervised learning model of this invention;

[0052] Figure 4 This is a flowchart of the image preprocessing procedure for the present invention.

[0053] Figure 5 This is a flowchart of the image registration process for the present invention. Detailed Implementation

[0054] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0055] like Figure 1As shown in the figure, this embodiment discloses a brain MRI image registration method based on self-supervised learning, which specifically includes the following steps:

[0056] Step 1, Pre-task process: Design the pre-tasks for the brain registration self-supervised learning method and construct the training dataset.

[0057] like Figure 2 As shown, step 1 specifically includes the following steps:

[0058] Step 101: Prepare a publicly available standard dataset of brain MRI images. Select any image from the standard dataset of brain MRI images as the current standard data image.

[0059] Step 102: Randomly generate the perturbation affine matrix A gt Then we have:

[0060]

[0061] Perturbation affine matrix A gt The element in the i-th row and j-th column is represented as a ij a ij The perturbation is randomly generated on different channels, representing the transformation in four degrees of freedom (translation, rotation, scaling, and tilt) on each channel.

[0062] Step 103: Randomly generate the nonlinear perturbation matrix B gt Then we have:

[0063]

[0064] Nonlinear perturbation matrix B gt The element in the i-th row is represented as b i b i For nonlinear perturbations randomly generated on different channels;

[0065] Step 104: Calculate the brain MRI image after adding random perturbation to the current image, based on the following formula (1) for superimposing perturbation:

[0066]

[0067] In equation (1), x, y, and z are the three channels of the current standard data image, and x', y', and z' are the three channels of the brain MRI image after perturbation.

[0068] Step 105: Perform interpolation calculations on the brain MRI images after perturbation to eliminate the influence of the transformed coordinates being decimals;

[0069] Step 106: Traverse all standard data images in the standard brain MRI dataset, repeat steps 101 to 105, and obtain the perturbation-added brain MRI images corresponding to all standard data images.

[0070] Step 107: Divide all the perturbated brain MRI images into training datasets, validation datasets, and test datasets for subsequent training.

[0071] Step 2, Self-supervised learning model training process: Design and train a brain registration self-supervised learning model.

[0072] like Figure 3 As shown, step 2 specifically includes the following steps:

[0073] Step 201: Obtain the training dataset;

[0074] Step 202: Build a brain registration self-supervised learning model based on a deep neural network model. In this model, the 2D convolution and pooling operations in the deep neural network model are changed to 3D convolution and pooling, and the last fully connected layer is changed to 1×15 dimensions, so that they correspond to the 12 degrees of freedom of affine transformation and the 3-dimensional bias of nonlinear perturbation, respectively. The selected deep neural network model includes, but is not limited to, the ResNet model.

[0075] Step 203: Input the training set data into the constructed brain registration self-supervised learning model. Through feature extraction, fully connected dimensionality reduction, and regression calculation, obtain a 1×15 dimension vector pre. 15 =[p1,p2,……,p 15 ], where p1, p2, ..., p 12 The affine transformation matrix A that forms the prediction pre And by p 13 p 14 p 15 Then the nonlinear bias matrix B is formed. pre That is:

[0076]

[0077]

[0078] Step 204: The randomly generated perturbation affine matrix A... gt and nonlinear perturbation matrix B gt The perturbation vector Gt is obtained by stretching the feature to a 1×15 dimension. 15 Then we have Gt 15 =[a 11 ,a 12 ...a 34[b1,b2,b3], construct a loss function and calculate the vector pre predicted by the brain registration self-supervised learning model based on the loss function. 15 With randomly added perturbation vector Gt 15 The loss value between the two is defined by the loss function as shown in equation (2):

[0079] Loss = (Pre 15 -Gt 15 ) 2 / 15 (2)

[0080] Step 205: Train the brain registration self-supervised learning model through forward and backward propagation until the loss value no longer decreases, that is, the brain registration self-supervised learning model training is complete.

[0081] Step 3: Preprocessing of images to be registered: Perform appropriate preprocessing on the brain MRI images to be registered.

[0082] like Figure 4 As shown, step 3 specifically includes the following steps:

[0083] Step 301: Input the brain MRI image to be registered;

[0084] Step 302: Convert the captured raw IMA format image into a Nii format image.

[0085] Step 303: Swap axes to align the brain with the standard axis.

[0086] Step 304: Perform bias field correction on the brain MRI images processed in step 303;

[0087] Step 305: Remove redundant parts such as the neck and face from the brain MRI image, and remove the skull so that it can be input into the brain registration self-supervised learning model.

[0088] Step 4: Image registration process: Use a well-trained brain registration self-supervised learning model to register the brain MRI image after the preprocessing in Step 3 to the standard space.

[0089] like Figure 5 As shown, step 4 specifically includes the following steps:

[0090] Step 401: Prepare the brain MRI images after the preprocessing in Step 3;

[0091] Step 402: Input the brain MRI image into the brain registration self-supervised learning model to obtain the 1×15 dimension feature vector predicted by the brain registration self-supervised learning model. And these form the corresponding affine transformation matrix for registration. and nonlinear bias matrix Then we have:

[0092]

[0093]

[0094] Step 403: Perform brain MRI image registration according to the following formula (3):

[0095]

[0096] In equation (3), These are the three channels of the registered brain MRI image, corresponding to x... org y org z org The input brain MRI image to be registered has three channels;

[0097] Step 404: Interpolate the registered brain MRI images to eliminate the influence of decimal coordinates.

Claims

1. A method for registering brain magnetic resonance images based on self-supervised learning, characterized in that, Specifically, the following steps are included: Step 1, Pre-task process: Design the pre-tasks for the brain registration self-supervised learning method and construct the training dataset, specifically including the following steps: Step 101: Prepare a publicly available standard dataset of brain MRI images. Select any image from the standard dataset of brain MRI images as the current standard data image. Step 102: Randomly generate perturbation affine matrix Then we have: Perturbation affine matrix The first in Line number Column elements are represented as , For the perturbations randomly generated on different channels, represent the transformations in four degrees of freedom (translation, rotation, scaling, and tilt) on each channel; Step 103: Randomly generate the nonlinear perturbation matrix Then we have: Nonlinear perturbation matrix The first in Row element is represented as , For nonlinear perturbations randomly generated on different channels; Step 104: Calculate the brain MRI image after adding random perturbation to the current image, based on the following formula (1) for superimposing perturbation: (1) In equation (1), , , These are the three channels of the current standard data image, corresponding to... , , Three channels of a brain MRI image after perturbation; Step 105: Perform interpolation calculations on the brain MRI images after perturbation to eliminate the influence of the transformed coordinates being decimals; Step 106: Traverse all standard data images in the standard brain MRI dataset, repeat steps 101 to 105, and obtain the perturbation-added brain MRI images corresponding to all standard data images. Step 107: Divide all the perturbated brain MRI images into training datasets, validation datasets, and test datasets for subsequent training. Step 2, Self-Supervised Learning Model Training Process: Design and train a brain registration self-supervised learning model, which includes the following steps: Step 201: Obtain the training dataset; Step 202: Build a brain registration self-supervised learning model; Step 203: Input the training set data into the constructed brain registration self-supervised learning model. Through feature extraction, fully connected dimensionality reduction, and regression calculation, a vector of size 1×15 is obtained. , among which, by Composition of the predicted affine transformation matrix , and by , , Then a nonlinear bias matrix is ​​formed. That is: Step 204: The randomly generated perturbation affine matrix and nonlinear perturbation matrix Stretch the feature to a 1×15 dimension to obtain the perturbation vector. Then there is Construct a loss function and, based on the loss function, calculate the vector predicted by the brain registration self-supervised learning model. With randomly added perturbation vector The loss value between the two is defined by the loss function as shown in equation (2): (2) Step 205: Train the brain registration self-supervised learning model through forward propagation and backward propagation until the loss value no longer decreases, that is, the brain registration self-supervised learning model training is complete. Step 3: Preprocessing of images to be registered: Perform appropriate preprocessing on the brain MRI images to be registered; Step 4: Image registration process: Use a well-trained brain registration self-supervised learning model to register the brain MRI image after the preprocessing in Step 3 to the standard space.

2. The brain MRI image registration method based on self-supervised learning as described in claim 1, characterized in that, In step 202, the brain registration self-supervised learning model is built based on the deep neural network model. The 2D convolution and pooling operations in the deep neural network model are changed to 3D convolution and pooling, and the last fully connected layer is changed to a size of 1×15, so that they correspond to the 12 degrees of freedom of affine transformation and the 3-dimensional bias of nonlinear perturbation, respectively.

3. The brain MRI image registration method based on self-supervised learning as described in claim 1, characterized in that, Step 3 specifically includes the following steps: Step 301: Input the brain MRI image to be registered; Step 302: Convert the captured raw IMA format image into a nii format image; Step 303: Swap axes to align the brain with the standard axis. Step 304: Perform bias field correction on the brain MRI images processed in step 303; Step 305: Remove the neck and face from the brain MRI image and remove the skull so that it can be input into the brain registration self-supervised learning model.

4. The brain MRI image registration method based on self-supervised learning as described in claim 1, characterized in that, Step 4 specifically includes the following steps: Step 401: Prepare the brain MRI images after the preprocessing in Step 3; Step 402: Input the brain MRI image into the brain registration self-supervised learning model to obtain the 1×15 dimension feature vector predicted by the brain registration self-supervised learning model. And from these, form the corresponding affine transformation matrix for registration. and nonlinear bias matrix Then we have: Step 403: Perform brain MRI image registration according to the following formula (3): (3) In equation (3), , , These are the three channels of the registered brain MRI image, corresponding to... , , The input brain MRI image to be registered has three channels; Step 404: Interpolate the registered brain MRI images to eliminate the influence of decimal coordinates.

Images