A Comprehensive Brain Image Registration Method with Large Deformation Characteristics

Abstract

The present invention relates to a comprehensive brain image registration method based on progressive interpolation and diffeomorphism, with large deformation characteristics. Compared with the prior art, the present invention performs a progressive decomposition of the large deformation to be solved in the registration problem. For the initial global large deformation, a thin plate spline algorithm based on control points is designed to achieve a rapid estimation of the large deformation; for the local deformation, a B-spline algorithm based on image small panes is designed to achieve the adjustment of local details; for the remaining small and complex deformations, an algorithm based on partial differential equations is designed to achieve the precise registration of all complex deformations in the image. Such a progressive design effectively reduces the deformation calculation difficulty of the diffeomorphic mapping method steps with large computational amount and unstable calculation results, and at the same time improves the overall registration accuracy and efficiency, so that the global structure of the brain image and the tissues with complex structural differences locally have more accurate registration effects.

Description

Technical Field

[0001] The present invention relates to the technical field of image deformation configuration, and in particular to a comprehensive brain image registration method with large deformation characteristics. Background Art

[0002] The registration process of brain images is an essential step in almost all longitudinal and population studies of brain diseases. From a neuroanatomical perspective, there are differences in the global shape and local structure of the human brain anatomy among different individuals in a population. Large deformable registration is a key technology in brain image registration. The core of the technology is to accurately and effectively calculate the diffeomorphic deformation field, so as to align the corresponding neuroanatomical features of different brain images.

[0003] Existing common non-linear registration techniques and tools include D.Demons, SyN in the ANTs software, etc. Some comprehensive registration algorithms, such as a local region precise deformation registration algorithm with joint point registration disclosed in Chinese Patent CN201510008982.3, and a non-rigid image registration algorithm based on implicit shape representation and edge information fusion disclosed in Chinese Patent CN200910100749.2. Although these techniques can basically successfully achieve the non-linear registration of the global overall brain images, they cannot guarantee the accuracy of registration for the registration cases with large deformation requirements. In terms of details, for example, for the white matter with complex morphology, the registration results of the existing techniques can hardly meet the requirements of basic alignment. This has a great impact on the subsequent quantitative analysis based on local regions of interest or voxels.

[0004] The registration algorithm with large deformation characteristics has the highest current registration performance. The large deformation diffeomorphic mapping (LDDMM) is one of the most mainstream diffeomorphic registration models in current practical applications. It operates on the velocity field by designing a partial differential equation for deformation based on the flow method, and adds a constraint on the smoothness of the velocity field to ensure the smoothness of local small deformations and achieve diffeomorphic deformations at a large scale. The optimization goal of this model is to solve the velocity field that minimizes the mismatch index between the image to be registered and the target image under smooth regularization. However, to ensure numerical stability, this method requires a sufficiently small time step and has a high computational complexity for the integral solution of the velocity field. Therefore, in many practical application scenarios, a simplified method with a fixed velocity field is used to reduce the degrees of freedom of the model, such as the DARTEL algorithm. Another major problem of the LDDMM model is that when dealing with large deformations, due to the influence of the model regularization term, the estimation of the initial momentum may be inaccurate, which will cause the diffeomorphic property of the algorithm to be still destroyed under large deformations. Therefore, the use of LDDMM requires a good preprocessing step as a prerequisite to pre-eliminate the interference brought by large global deformations (including linear transformations such as displacement and rotation).

[0005] Regarding the above problem of calculating a large global transformation deformation field, the solution of the current technology is to decompose the overall registration process into two parts. In the first part, an initial deformation field is calculated for the large global transformation first, and in the second part, the remaining deformation is processed. In the prior art, algorithms based on machine learning are also used to estimate the remaining deformation in the more complex second part. However, whether it is LDDMM or the algorithm based on machine learning, they are all based on the modeling of the overall deformation field, which will result in the local deformation of the image not being accurately calculated. Summary of the Invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art, and provide a comprehensive brain image registration method based on progressive interpolation and diffeomorphism, with large deformation characteristics, which performs a progressive decomposition on the large deformation to be solved in the registration problem. For the initial global large deformation, a thin plate spline algorithm based on control points is designed to achieve a rapid estimation of the large deformation; for the local deformation, a B-spline algorithm based on image small panes is designed to achieve the adjustment of local details; for the remaining small and complex deformations, an algorithm based on partial differential equations is designed to achieve the precise registration of all complex deformations in the image. Such a progressive design effectively reduces the deformation calculation difficulty of the diffeomorphic mapping method steps with large computational amount and unstable calculation results, and at the same time improves the overall registration accuracy and efficiency, so that the global structure of the brain image and the tissues with complex structural differences locally have more accurate registration effects.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] A comprehensive brain image registration method with large deformation characteristics, which acquires the brain image to be registered and the target brain image; pre-registers the brain image to be registered and the target brain image to obtain the pre-registration result of the brain image to be registered; and uses a large deformation diffeomorphic mapping algorithm based on partial differential equations to register the pre-registration result of the brain image to be registered and the target brain image to obtain the registered brain image.

[0009] Furthermore, it includes the following steps:

[0010] S1. Acquire the brain image to be registered and the target brain image;

[0011] S2. Use a thin plate spline algorithm based on control point pairs to perform global registration on the brain image to be registered and the target brain image to obtain the initial pre-registration result of the brain image to be registered;

[0012] S3. Use a B-spline algorithm based on image small panes to perform local registration on the initial pre-registration result of the brain image to be registered and the target brain image to obtain the pre-registration result of the brain image to be registered;

[0013] S4. Use a large deformation diffeomorphic mapping algorithm based on partial differential equations, such as the DARTEL algorithm, to register the pre-registered result of the brain image to be registered with the target brain image, and obtain the registered brain image.

[0014] Further, step S2 includes:

[0015] Obtain the feature point set of the brain image to be registered and the feature point set of the target brain image, match the feature points in the feature point set of the brain image to be registered and the feature point set of the target brain image, obtain the corresponding relationship of the feature points in the two images, and form a control point pair with the corresponding two feature points to obtain multiple control point pairs;

[0016] Based on the control point pairs, use the thin plate spline algorithm to globally pre-register the brain image to be registered with the target brain image. For a control point pair, establish the mapping relationship between the feature points on the brain image to be registered and the feature points on the target brain image, and use the thin plate spline interpolation function to ensure the smoothness of the overall deformation field and minimize the bending energy of the spline at the same time, so as to obtain the globally smooth initial pre-registered result of the brain image to be registered.

[0017] Further, use the SIFT algorithm for automatically extracting local corner features to extract feature points from the brain image to be registered and the target brain image respectively, and extract the feature points with scale-invariant properties in the brain image to be registered and the target brain image to obtain the feature point set of the brain image to be registered and the feature point set of the target brain image.

[0018] Further, step S3 includes:

[0019] Obtain the pre-set pane size, perform windowing on the initial pre-registered result of the brain image to be registered and the target brain image, and make the panes of the initial pre-registered result of the brain image to be registered correspond to the panes of the target brain image one by one to obtain multiple pane pairs;

[0020] For each pane pair, use each voxel in the pane as the control object, use the cubic B-spline algorithm, and perform interpolation using the cubic B-spline function to determine the displacement vector of any voxel within the pane, and align the image within the pane of the initial pre-registered result with the image within the pane of the target brain image to obtain the pre-registered result of the local registration of the brain image to be registered.

[0021] Further, use the sliding window method to divide the initial pre-registered result of the brain image to be registered and the target brain image into multiple image panes respectively with a pre-set step size, and the size of the image pane is the pre-set pane size.

[0022] Further, the pre-set step size is 100 voxels, and the pre-set pane size is 128×128 voxels.

[0023] Furthermore, the normalized mutual information is used as the similarity measure, and the third-order B-spline algorithm is used to align the images within the window pane of the initial pre-registration result with the images within the window pane of the target brain image.

[0024] Furthermore, during the process of aligning the images within the window pane of the initial pre-registration result with the images within the window pane of the target brain image using the third-order B-spline algorithm, non-negativity constraints on the Jacobian coefficient of the deformation field and spatial constraints of isotropic total variation are set.

[0025] Furthermore, during the process of registering the pre-registration result of the brain image to be registered with the target brain image using the large deformation diffeomorphic mapping algorithm based on partial differential equations, a simplified method of a fixed velocity field is adopted to reduce the degrees of freedom of the large deformation diffeomorphic mapping model, thereby accelerating the calculation speed.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] (1) During the registration process, the thin plate spline algorithm based on control points, the B-spline algorithm based on image window panes, and the algorithm based on partial differential equations are combined to accurately estimate the global large deformation, local deformation, and tiny complex deformation respectively, so that both the global structure of the brain image and the tissues with complex structural differences locally have more accurate registration effects.

[0028] (2) Different from directly using the registration algorithm based on partial differential equations in the past, the present application provides an initial deformation field through two-step pre-registration to reduce the computational complexity of the registration algorithm based on partial differential equations, and greatly reduces the risk of initial momentum estimation error of this type of algorithm, further improving the efficiency and effectiveness of this type of method in dealing with the registration problem of large deformation brain images.

[0029] (3) The idea of progressive decomposition is proposed. For the initial global large deformation, a thin plate spline algorithm based on control points is designed to achieve a rapid estimation of the larger deformation; for the local deformation, a B-spline algorithm based on image window panes is designed to achieve an adjustment of local details; for the remaining tiny complex deformation, an algorithm based on partial differential equations is designed to achieve precise registration of all complex deformations in the image. The calculation amount of the diffeomorphic mapping method steps is large and the calculation results are unstable. The progressive design effectively reduces the deformation calculation difficulty of the diffeomorphic mapping method steps, and at the same time improves the overall registration accuracy and efficiency, so that both the global structure of the brain image and the tissues with complex structural differences locally have more accurate registration effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flowchart of the present invention;

[0031] Figure 2It is a comparison chart of the registration results between using the registration method of the present application and directly using the DARTEL algorithm in the simulation experiment;

[0032] Figure 3 It is a comparison chart of the registration results between using the registration method of the present application and using other algorithms in the real brain image dataset. Detailed implementation manners

[0033] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and gives detailed implementation manners and specific operation processes, but the protection scope of the present invention is not limited to the following embodiments.

[0034] Embodiment 1:

[0035] Image deformation registration generally matches and superimposes two or more images obtained at different times, by different sensors (imaging devices), or under different conditions (weather, illumination, camera position and angle, etc.). Image deformation registration essentially aims to find a point-to-point spatial transformation to deform the anatomical structure of one image (the image to be registered) onto another image (the target image).

[0036] A comprehensive brain image registration method with large deformation characteristics, which includes obtaining the brain image to be registered and the target brain image; pre-registering the brain image to be registered and the target brain image to obtain the pre-registration result of the brain image to be registered; using a large deformation diffeomorphic mapping algorithm based on partial differential equations to register the pre-registration result of the brain image to be registered and the target brain image to obtain the registered brain image; specifically, as Figure 1 shown, it includes the following steps:

[0037] S1. Obtain the brain image to be registered and the target brain image; in this step, methods such as Gaussian filtering can also be used to preprocess the brain image to remove the influence of noise;

[0038] S2. Use the thin plate spline algorithm based on control point pairs to globally register the brain image to be registered and the target brain image to obtain the initial pre-registration result of the brain image to be registered;

[0039] Specifically, step S2 includes:

[0040] 1) Obtain the feature point set of the brain image to be registered and the feature point set of the target brain image, match the feature points in the feature point set of the brain image to be registered and the feature point set of the target brain image to obtain the corresponding relationship of the feature points in the two images, and form a control point pair with two corresponding feature points to obtain multiple control point pairs;

[0041] In this embodiment, the SIFT algorithm for automatically extracting local corner features is used to extract feature points from the brain image to be registered and the target brain image respectively, and the feature points with scale-invariant properties in the brain image to be registered and the target brain image are extracted to obtain the feature point set of the brain image to be registered and the feature point set of the target brain image.

[0042] Compared with manually calibrating feature points, using the SIFT algorithm for feature point extraction has a higher degree of automation, faster speed, and more objective results. When matching feature points to determine the corresponding relationship of feature points in the brain image to be registered and the target brain image, common matching methods can be used to measure the similarity of feature points. In this embodiment, scale invariance is used as the similarity metric, and a feature point in the brain image to be registered is matched with a feature point in the target brain image to obtain a control point pair.

[0043] 2) Based on the control point pairs, the thin plate spline algorithm is used to globally pre-register the brain image to be registered and the target brain image. For a control point pair, a mapping relationship between the feature points on the brain image to be registered and the feature points on the target brain image is established, and the thin plate spline interpolation function is used to ensure the smoothness of the overall deformation field and minimize the bending energy of the spline at the same time, obtaining a globally smooth initial pre-registration result of the brain image to be registered.

[0044] The thin plate spline algorithm (or the thin plate spline registration method) is a commonly used image registration algorithm. In this embodiment, a common thin plate spline model can be used, and relevant practitioners can understand this, so no more details will be elaborated here.

[0045] S3. The B-spline algorithm based on image small panes is used to locally register the initial pre-registration result of the brain image to be registered and the target brain image to obtain the pre-registration result of the brain image to be registered;

[0046] Specifically, step S3 includes:

[0047] 1) Obtain the pre-set pane size, and perform panning on the initial pre-registration result of the brain image to be registered and the target brain image, and make the panes of the initial pre-registration result of the brain image to be registered correspond to the panes of the target brain image one by one to obtain multiple pane pairs;

[0048] In this embodiment, the sliding window method is used to divide the initial pre-registration result of the brain image to be registered and the target brain image into multiple image panes respectively with a pre-set step size (such as 100 voxels). The size of the image panes is the pre-set pane size (such as 128×128 voxels). After the division, the panes of the initial pre-registration result correspond to the panes of the target brain image one by one. For a pane of the initial pre-registration result, the image within the pane is aligned with the image within the corresponding pane of the target brain image, so as to perform local registration.

[0049] 2) Taking each voxel in the pane as a control object, the third-order B-spline algorithm is used for each pane pair respectively, and interpolation is performed using the third-order B-spline function to determine the displacement vector of any voxel within the pane. The image within the pane of the initial pre-registration result is aligned with the image within the pane of the target brain image to obtain the pre-registration result of the local registration of the brain image to be registered.

[0050] When performing image registration, the gray level of the registration image is regarded as a random variable, and mutual information is used as the similarity measure. When the spatial positions of the two images reach the maximum, the mutual information value is the largest. In order to reduce the influence of the image overlapping area on the mutual information, in this embodiment, normalized mutual information is used as the similarity measure, and the third-order B-spline algorithm is used to align the image within the pane of the initial pre-registration result with the image within the pane of the target brain image. A non-negativity constraint on the Jacobian coefficient of the deformation field and a spatial constraint of isotropic total variation are also set to maintain the diffeomorphic property of the deformation process, laying a foundation for the final registration in step S4.

[0051] The B-spline algorithm (or B-spline interpolation registration method) is also a commonly used technical means in image registration. Relevant practitioners can understand this and will not elaborate here.

[0052] S4. Based on the pre-registration result obtained by registering through the thin plate spline algorithm and the B-spline algorithm, the large deformation diffeomorphic mapping algorithm based on partial differential equations is used to register the pre-registration result of the brain image to be registered with the target brain image to obtain the registered brain image. In this embodiment, a simplified method with a fixed velocity field is adopted to reduce the degrees of freedom of the large deformation diffeomorphic mapping model, thereby accelerating the calculation speed. During the registration process, the deformation field obtained after the second registration is used as the initial deformation field, and the registration algorithm based on partial differential equations is normally used for registration.

[0053] Based on the pre-registration result obtained by registering through the thin plate spline algorithm and the B-spline algorithm, the subsequent steps of the large deformation diffeomorphic mapping algorithm (or diffeomorphic flow method) based on partial differential equations can adopt the commonly used algorithm steps. Relevant practitioners can understand this and will not elaborate here.

[0054] Brain image registration has high requirements for accuracy, especially for white matter with complex shapes. The registration results of the existing technologies can hardly meet the requirements of basic alignment. The registration algorithm with large deformation characteristics currently has the highest registration performance. The most popular one is the large deformation diffeomorphic mapping based on partial differential equations. However, it models the overall deformation field, resulting in inaccurate calculation of the local deformation of the image.

[0055] This application is based on the large deformation diffeomorphic mapping (LDDMM) based on partial differential equations, and combines the thin plate spline algorithm based on control points and the B-spline algorithm based on image patches. The thin plate spline algorithm is used for global large deformation registration, and the B-spline algorithm is used for local deformation registration.

[0056] Different from directly using the registration algorithm based on partial differential equations, this application provides an initial deformation field through two-step pre-registration, thereby reducing the computational complexity of the registration algorithm based on partial differential equations, and greatly reducing the risk of initial momentum estimation errors of this type of algorithm, and further improving the efficiency and effectiveness of this type of method in dealing with the large deformation brain image registration problem.

[0057] As Figure 2 shown, Figure 2 In (a), a large deformation registration task with a simple topological structure is simulated. The images to be registered and the target image are the upper and lower two images on the far left. It can be seen the image obtained by using this application after three-step registration (steps S2, S3, and S4), and the image obtained by directly using the large deformation diffeomorphic mapping algorithm based on partial differential equations (DARTEL algorithm) without using the thin plate spline algorithm and the B-spline algorithm is shown. It can be seen that the registration effect of using the DARTEL algorithm after pre-registration with the thin plate spline algorithm and the B-spline algorithm is better than directly using the DARTEL algorithm.

[0058] Figure 2 In (b), a large deformation registration task with a complex topological structure is simulated. Similar to (a), it can be seen that the registration effect of using the DARTEL algorithm after pre-registration with the thin plate spline algorithm and the B-spline algorithm is better than directly using the DARTEL algorithm, and the image to be registered can be basically accurately aligned with the target image.

[0059] This application provides an initial deformation field through two-step pre-registration, reducing the computational complexity of the registration algorithm based on partial differential equations, and also making the performance of this type of algorithm more stable when dealing with the large deformation registration problem. The designed simulation experiments ( Figure 2 a and b in) show that compared with directly using the registration algorithm DARTEL based on partial differential equations, the registration effect of this application is better, and at the same time, the success rate, accuracy, and speed of the overall registration process of the large deformation problem are improved.

[0060] As Figure 3As shown, based on real brain images, in two brain image datasets, namely HCP and ADNI, the registration images obtained using the present application were compared with the registration images obtained using other existing technologies (D. Demons, SyN, directly using the DARTEL algorithm). It can be seen that the registration effect of the present application is better. From the magnified registration result images, it can be seen that the present application can better register the local complex details of the images, demonstrating the superiority of its registration accuracy.

[0061] A comprehensive brain image registration method with large deformation characteristics proposed by the present application, compared with the existing commonly used registration techniques such as D. Demons and the SyN tool in ANTs software, in the registration tasks of three-dimensional brain images under different datasets, the registrations of gray matter and white matter have achieved higher DICE similarity coefficients. The registration effect of the local area of white matter is better. Refer to Figure 3 , and it can be clearly observed visually that the present application is more successful in registering the complex structures of the registration details.

[0062] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative labor. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field based on the concept of the present invention through logical analysis, reasoning, or limited experiments on the basis of the existing technology should be within the protection scope determined by the claims.

Claims

1. A comprehensive method for brain image registration with large deformation characteristics, characterized in that, Obtain the brain image to be registered and the target brain image; perform pre-registration on the brain image to be registered and the target brain image to obtain the pre-registration result of the brain image to be registered; use the large deformation diffeomorphic mapping algorithm based on the partial differential equation to register the pre-registration result of the brain image to be registered with the target brain image to obtain the registered brain image; It includes the following steps: S1. Obtain the brain image to be registered and the target brain image; S2. Use the thin plate spline algorithm based on control point pairs to globally register the brain image to be registered and the target brain image to obtain the initial pre-registration result of the brain image to be registered; S3. Use the B-spline algorithm based on image patches to locally register the initial pre-registration result of the brain image to be registered with the target brain image to obtain the pre-registration result of the brain image to be registered; S4. Use the large deformation diffeomorphic mapping algorithm based on the partial differential equation to register the pre-registration result of the brain image to be registered with the target brain image to obtain the registered brain image; Step S2 includes: Obtain the feature point set of the brain image to be registered and the feature point set of the target brain image, match the feature points in the feature point set of the brain image to be registered and the feature point set of the target brain image to obtain the corresponding relationship of the feature points in the two images, and form a control point pair with the two corresponding feature points to obtain multiple control point pairs; Based on the control point pairs, use the thin plate spline algorithm to globally pre-register the brain image to be registered and the target brain image to obtain the globally smooth initial pre-registration result of the brain image to be registered; Step S3 includes: Obtain the pre-set patch size, perform patching on the initial pre-registration result of the brain image to be registered and the target brain image, and make the patches of the initial pre-registration result of the brain image to be registered correspond to the patches of the target brain image one by one to obtain multiple patch pairs; For each patch pair, use each voxel in the patch as the control object, and use the third-order B-spline algorithm to align the image in the patch of the initial pre-registration result with the image in the patch of the target brain image to obtain the pre-registration result of the local registration of the brain image to be registered.

2. The comprehensive brain image registration method with large deformation characteristics according to claim 1, wherein, Use the SIFT algorithm to extract feature points from the brain image to be registered and the target brain image respectively to obtain the feature point set of the brain image to be registered and the feature point set of the target brain image.

3. A comprehensive method for brain image registration with large deformation characteristics according to claim 1, characterized in that, Use the sliding window method to divide the initial pre-registration result of the brain image to be registered and the target brain image into multiple image patches respectively with the pre-set step size, and the size of the image patch is the pre-set patch size.

4. A comprehensive brain image registration method with large deformation characteristics according to claim 3, characterized in that, The pre-set step size is 100 voxels, and the pre-set patch size is 128×128 voxels.

5. A comprehensive brain image registration method with large deformation characteristics according to claim 1, characterized in that Use the normalized mutual information as the similarity measure, and use the third-order B-spline algorithm to align the image in the patch of the initial pre-registration result with the image in the patch of the target brain image.

6. A comprehensive brain image registration method with large deformation characteristics according to claim 1, characterized in that, During the process of using the third-order B-spline algorithm to align the image in the patch of the initial pre-registration result with the image in the patch of the target brain image, the non-negativity constraint of the Jacobian coefficient of the deformation field and the spatial constraint of the isotropic total variation are set.

7. A comprehensive brain image registration method with large deformation characteristics according to claim 1, characterized in that, During the process of registering the pre-registration result of the brain image to be registered with the target brain image using the large deformation diffeomorphic mapping algorithm based on the partial differential equation, a simplified method with a fixed velocity field is adopted.

Images