A brain MRI registration method based on frequency domain and image domain features

Abstract

The application discloses a brain MRI registration method based on frequency domain and image domain features, and applies to the technical field of medical image processing, comprising: data preprocessing on brain MRI data set; a pair of images are randomly selected as floating image and fixed image, splicing and processing are carried out, wavelet transform and rotating window Transformer are used for feature extraction respectively, and the optimal feature extraction image is obtained; the optimal feature extraction image is decoded to obtain a registration deformation field; the floating image and the registration deformation field are input into a spatial transformation network to obtain a registration image; similarity measurement is carried out on the fixed image and the registration image, and the boundary alignment and folding point in the registration deformation field are optimized. The method uses the wavelet domain features and the image domain features of the double-channel extracted image, uses the structural similarity loss to promote the boundary alignment of the image anatomical region, and increases the Jacobian anti-folding optimization regular term in the objective function, so that the registration accuracy and authenticity are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, in particular to a brain MRI registration method based on frequency domain and image domain features. BACKGROUND

[0002] The brain is one of the most important organs of the human body, but the mortality rate of current brain-related diseases is still high. Brain diseases have the characteristics of strong suddenness and high mortality. Magnetic resonance imaging (MRI) as a good soft tissue imaging non-invasive auxiliary diagnostic tool plays an important role in brain disease screening, diagnosis, treatment guidance and evaluation. The registered brain MRI image fuses the information between the two images, which can provide more reliable information for doctors to judge the disease, and plays an important role in lesion positioning, surgical navigation and prognosis evaluation. In clinical treatment, doctors usually use manual registration to compare two or more images of patients at different times, different angles or even different imaging principles. However, the manual registration method is limited by the personal experience and state of the doctor, and the registration result varies from person to person. Moreover, this method consumes manpower and material resources, and wastes a lot of medical resources. Therefore, the clinical demand prompts us to invent a fully automatic registration algorithm to replace the time-consuming and laborious work of manual registration. With the emergence of convolutional neural network (CNN) and the rapid development of computer hardware, medical image registration algorithms based on deep learning have gradually appeared and become a research hotspot. However, due to the complexity of brain image content, the accuracy and authenticity of current brain image registration algorithms based on deep learning are difficult to meet the requirements of clinical application level.

[0003] Therefore, how to provide a brain MRI registration method based on frequency domain and image domain features to effectively improve the registration accuracy and authenticity is a problem that those skilled in the art need to solve. SUMMARY

[0004] Therefore, the present application provides a brain MRI registration method based on frequency domain and image domain features. By using dual-channel to extract the wavelet domain features and image domain features of the image, stronger image features are obtained to help improve the registration accuracy of the network; at the same time, the structural similarity loss is used to improve the registration accuracy of the network by promoting the boundary alignment of the anatomical regions of the image, and the Jacobian anti-aliasing optimization regular term is added in the objective function to enhance the reversible consistency of the deformation field, thereby improving the authenticity of the registration result.

[0005] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0006] A brain MRI registration method based on frequency domain and image domain features, comprising:

[0007] Step (1): Obtain a brain MRI dataset, and perform data preprocessing on the brain MRI dataset.

[0008] Step (2): Randomly select a pair of images in the brain MRI dataset as a floating image and a fixed image, perform splicing and processing, and perform feature extraction using wavelet transform and a rotating window Transformer, respectively, to obtain an optimal feature extraction image.

[0009] Step (3): Decode the optimal feature extraction image to obtain a registration deformation field.

[0010] Step (4): Input the floating image and the registration deformation field into a spatial transformation network to obtain a registration image.

[0011] Step (5): Perform similarity measurement on the fixed image and the registration image, and optimize boundary alignment and fold points in the registration deformation field.

[0012] Step (6): Perform iterative training using a registration network to obtain a network model with optimal weight parameters.

[0013] Step (7): Input an image to be registered into the network model with optimal weight parameters to obtain a registration result.

[0014] Optionally, in step (1), the data preprocessing includes skull removal processing, affine alignment, normalization, and size cropping using FreeSurfer.

[0015] Optionally, in step (2), a feature encoding module with a double-channel path is built, the floating image and the fixed image are spliced, and then input into the encoders of the wavelet transform and the rotating window Transformer for down-sampling processing.

[0016] Optionally, in step (2), the wavelet transform for feature extraction specifically includes:

[0017] Each layer of the forward propagation network based on the wavelet transform is defined as:

[0018]

[0019] ψ s =UG s U T ;

[0020]

[0021] wherein, G srepresents a feature map signal; s is a scale coefficient, which describes different scales of the wavelet base signal; ψ s represents a wavelet base, and U represents a matrix composed of Laplacian matrix eigenvectors. is a diagonal matrix, representing the convolution kernels available for learning at the kth layer; represents n nodes with f k channels. P is a dimension n x p of a feature vector , and q is a dimension n x q of a feature vector .

[0022] Optionally, in step (2), the feature extraction by the Rotated Window Transformer is specifically as follows:

[0023] The Rotated Window Transformer module is defined as follows:

[0024]

[0025]

[0026]

[0027]

[0028] wherein W-MSA represents a window-based multi-head attention module, MLP represents a multi-layer perception, SM-WSA represents a multi-head attention module using regularization and a rotated window, x l-1 and respectively represent the input and output features of the W-MSA module and the SM-WSA module, x l represents down-sampling output features, and LN represents layer normalization.

[0029] Optionally, in step (3), a decoding module for multi-scale information fusion is built to extract features with strong distinguishing ability from the optimal feature map, so as to obtain a registration deformation field.

[0030] Optionally, in step (4), trilinear interpolation is applied to transform the floating image, and the floating image and the registration deformation field are input into a spatial transformation network to obtain a registered image; the trilinear interpolation is defined as follows:

[0031]

[0032] wherein I M and respectively represent the floating image and the registered image, x represents a voxel in I M , y represents a voxel in an 8-pixel cubic neighborhood x+φ(y), d∈{i,j,k} represents three dimensions of the image, and φ represents the registration deformation field.

[0033] Optionally, in step (5), the boundary alignment is optimized by using structural similarity loss in the similarity measure objective function; the folding point is optimized by using a smoothing regularization term in the similarity measure objective function.

[0034] The similarity objective function is designed as:

[0035]

[0036] wherein, denotes the similarity measure objective function, denotes the structural similarity loss, which is a similarity measure between the fixed image I F and the registered image ; λ denotes the de-folding regularization term hyper-parameter.

[0037] The structural similarity loss between the fixed image I F and the registered image is defined as

[0038]

[0039] wherein, I F and are defined as the fixed image and the registered image, respectively; denotes the pixel intensity mean of the image I F over a neighborhood of size n = 11, denotes the pixel intensity mean of the image over a neighborhood of size n = 11; denotes the pixel intensity variance of the image I F over a neighborhood of size n = 11, denotes the pixel intensity variance of the image over a neighborhood of size n = 11; is the covariance of the two images over a neighborhood of size n = 11; C1 = (K1L) 2 C2 = (K2L) 2 are two variables used to maintain the stability of the equation; L is the dynamic range of the pixel.

[0040] The smoothing regularization term is defined as:

[0041]

[0042]

[0043] ​

[0044]

[0045] where, and represent the first and second order derivatives of the deformation field φ, respectively.

[0046] Optionally, in step (6), the registration network is iteratively trained, the Adam optimizer is used for optimization, the learning rate parameter is set to 2e-4, the batch size is set to 1, the training number is set to 1000 cycles, and each cycle is iterated 100 times. When the threshold of the training number is reached, the training is terminated.

[0047] Optionally, in step (7), the coincidence coefficient is further applied to evaluate the registration result.

[0048] The coincidence coefficient is defined as follows:

[0049]

[0050] M and W represent two images, respectively; wherein the closer the coincidence DSC is to 1, the more accurate the alignment of the anatomical structure of the image is.

[0051] According to the above technical solution, compared with the prior art, the brain MRI registration method based on frequency domain and image domain features disclosed in the present application uses the wavelet domain features and image domain features of the image extracted by the double channel to obtain image features with stronger distinguishing ability, thereby helping to improve the registration accuracy of the network. At the same time, the structural similarity loss is used to improve the registration accuracy of the network by promoting the boundary alignment of the anatomical regions of the image, and the Jacobian anti-aliasing optimization regular term is added in the objective function to enhance the reversible consistency of the deformation field, thereby improving the authenticity of the registration result. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creating any creative labor.

[0053] Figure 1 The flowchart of the present application.

[0054] Figure 2 The structural schematic diagram of the present application.

[0055] Figure 3 The rotation window Transformer module construction schematic diagram of the present application. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be apparently and completely described in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the protection scope of the present application.

[0057] The application discloses a brain MRI registration method based on frequency domain and image domain features, comprising:

[0058] Step (1): Obtain a brain MRI dataset, and perform data preprocessing on the brain MRI dataset, specifically: use FreeSurfer to perform skull removal processing and affine alignment to the same space; then normalize the obtained data to accelerate the convergence of the network; and cut the data to the same size of 128x128x128 to reduce the requirement of the network on computer hardware.

[0059] Step (2): Build a feature encoding module with a double-path channel, randomly select a pair of images in the brain MRI dataset as a floating image and a fixed image, splice and process them, and use wavelet transform and a rotating window Transformer for feature extraction respectively to obtain an optimal feature extraction image, specifically: build a feature encoding module with a double-path channel, and extract multi-scale features of the frequency domain and the image domain in the data respectively; the encoding module splices a pair of floating images and fixed images with dimensions being multiples of 16 for four times of down-sampling processing by the encoder; after the data is input into the feature extraction network, the feature map size is down-sampled to 1 / 16 of the original size by the encoders of the wavelet transform and the rotating window Transformer.

[0060] The wavelet transform for feature extraction is specifically:

[0061] Each layer of the forward propagation network based on the wavelet transform is defined as:

[0062]

[0063] Ψ s = UG s U T ;

[0064]

[0065] Wherein, G s represents a feature map signal; s is a scale coefficient describing different scales of the wavelet base signal; and ψ sis a wavelet basis, U is a Laplacian matrix eigenvector matrix. is a diagonal matrix, and represents the convolution kernel available for learning in the kth layer. represents n nodes with f k channels. P is a feature vector dimension n x p, and q is a feature vector dimension n x q.

[0066] Because the wavelet transform has the characteristics of multi-scale, it can decompose the signal into different frequency components, so using wavelet transform to extract the features of the image can provide rich global information for the network. Because the features after wavelet transform are sparse, and the eigenvectors of the Laplacian matrix are dense, the wavelet basis has higher sparsity and locality than the traditional Fourier basis, and the wavelet transform is more efficient than the Fourier transform.

[0067] In the process of image domain feature extraction, a long-distance image feature down-sampling module-rotating window Transformer is built to replace the traditional convolution down-sampling method to extract the long-distance feature dependency in the data, so as to extract the global features in the data; the rotating window Transformer promotes the alignment of the region of interest by the network, thereby improving the registration accuracy of the network.

[0068] The rotating window Transformer performs feature extraction as follows:

[0069] The rotating window Transformer module is defined as follows:

[0070]

[0071]

[0072]

[0073]

[0074] wherein W-MSA represents a window-based multi-head attention module, MLP represents a multi-layer perception machine, SM-WSA represents a multi-head attention module using regularization and rotating window, x l-1 and respectively represent the input and output features of the W-MSA module and the SM-WSA module, x l represents the down-sampling output feature, and LN represents layer normalization.

[0075] By using the dual-channel to extract the wavelet domain features and the image domain features of the image, the network can obtain more rich features, thereby improving the feature extraction capability of the network.

[0076] Step (3): A multi-scale information fusion decoding module is built to extract features with strong distinguishing ability from the optimal feature map, and a registration deformation field is obtained.

[0077] Step (4): The floating image and the registration deformation field are input into the spatial transformation network to obtain the registered image, specifically: the floating image is transformed by applying trilinear interpolation, the floating image and the registration deformation field are input into the spatial transformation network to obtain the registered image; the trilinear interpolation is defined as follows:

[0078]

[0079] wherein, I M and respectively represent the floating image and the registered image, x represents a voxel in I M , y represents a voxel in the 8-pixel cubic neighborhood x+φ(y), d∈{i,j,k} represents three dimensions of the image, and φ represents the registration deformation field.

[0080] Step (5): Similarity measurement is performed on the fixed image and the registered image, and the boundary alignment and the fold points in the registration deformation field are optimized; specifically: by using boundary loss in the similarity objective function, the penalty for the boundary mismatch part in the image is increased, the attention of the network on the boundary is increased, the boundary misalignment problem is focused on, the attention to the boundary is increased, and the boundary alignment in the registration deformation field is optimized; the fold points are optimized by using a smoothing regularization term in the similarity measurement objective function.

[0081] The similarity objective function is designed as:

[0082]

[0083] wherein, represents the similarity measurement objective function, represents the structural similarity loss, which is a similarity measurement between the fixed image I F and the registered image ; λ represents a fold regularization term hyperparameter.

[0084] The structural similarity loss between the fixed image I F and the registered image is defined as

[0085]

[0086] wherein, I F and are defined as the fixed image and the registered image, respectively; represents the image I F ​Mean of pixel intensity over a neighborhood of cubic size n = 11, representing an image Mean of pixel intensity over a neighborhood of cubic size n = 11; representing an image I F Variance of pixel intensity over a neighborhood of cubic size n = 11, representing an image Variance of pixel intensity over a neighborhood of cubic size n = 11; is the covariance of two images over a neighborhood of cubic size n = 11; C1 = (K1L) 2 C2 = (K2L) 2 are two variables used to maintain the stability of the equation; L is the dynamic range of the pixel.

[0087] using a smoothing regularization term to enhance the reversible consistency of the deformation field, thereby improving the effectiveness of the registration result, the smoothing regularization term is defined as follows:

[0088]

[0089]

[0090]

[0091] wherein, and respectively represent the first derivative and the second derivative of the deformation field φ, by designing a target function with an anti-aliasing optimization regularization term, promoting the reversible consistency of the deformation field of the registration network, and thereby improving the authenticity of the registration result.

[0092] Step (6): using the registration network for iterative training to obtain a network model with optimal weight parameters, specifically: iteratively training the registration network, using the Adam optimizer for optimization, and setting the learning rate parameter to 2e-4, the batch size to 1, the training number to 1000 cycles, and the iteration number to 100 times per cycle, and when the threshold of the training number is reached, the training is terminated.

[0093] Step (7): inputting the image to be registered into the network model with optimal weight parameters to obtain the registration result, and applying the coincidence coefficient to evaluate the registration result, specifically:

[0094] The coincidence coefficient is defined as follows:

[0095]

[0096] M and W represent two images respectively; wherein, the closer the coincidence degree DSC is to 1, the more accurate the alignment of the anatomical structure of the images is.

[0097] The application discloses a brain MRI registration method based on frequency domain and image domain features. The method uses a double-channel to extract wavelet domain features and image domain features of an image, so that more powerful image features are obtained, and registration accuracy of the network is improved. Meanwhile, a structural similarity loss is used to improve the registration accuracy of the network by promoting the boundary alignment of the anatomical regions of the image, and a Jacobian anti-aliasing optimization regular term is added to the objective function, so that the reversible consistency of the deformation field is enhanced, and the authenticity of the registration result is improved.

[0098] The various embodiments are described in the present specification in progressive order, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0099] The above description of disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A brain MRI registration method based on frequency domain and image domain features, characterized in that, The application relates to a brain image registration method and device. Step (1): obtaining a brain MRI data set, and performing data preprocessing on the brain MRI data set; Step (2): randomly selecting a pair of images in the brain MRI data set as a floating image and a fixed image, performing splicing and processing, and performing feature extraction by using wavelet transformation and a rotating window Transformer respectively to obtain an optimal feature extraction graph; Step (3): decoding the optimal feature extraction graph to obtain a registration deformation field; Step (4): inputting the floating image and the registration deformation field into a spatial transformation network to obtain a registration image; Step (5): performing similarity measurement on the fixed image and the registration image, and optimizing boundary alignment and folding points in the registration deformation field; Step (6): iteratively training a registration network to obtain a network model with optimal weight parameters; Step (7): inputting a to-be-registered image into the network model with optimal weight parameters to obtain a registration result.

2. The method of claim 1, wherein, In step (1), the data preprocessing is skull removal processing, affine alignment, normalization and size cropping by using FreeSurfer. 3.The brain MRI registration method based on frequency domain and image domain features according to claim 1, characterized in that, In step (2), a feature encoding module with a double-channel path is built, the floating image and the fixed image are spliced, and then are input into encoders of the wavelet transformation and the rotating window Transformer for down-sampling processing.

4. The method of claim 1, wherein, In step (2), the wavelet transformation performs feature extraction in the following manner: Each layer of the wavelet transformation-based forward propagation network is defined as: ； ； ； in, Represents a feature map signal; It is a scaling factor that describes different scales of the wavelet basis signal; Describes wavelet basis, This represents the matrix formed by the eigenvectors of the Laplacian matrix; It is a diagonal matrix, representing the first... Layers of convolutional kernels that can be learned; Indicates having Each channel One node; For feature vectors One dimension , Characteristic vector One dimension .

5. The method of claim 1, wherein, In step (2), the rotating window Transformer performs feature extraction in the following manner: The rotating window Transformer module is defined as follows: ； ； ； ； wherein, denotes a window-based multi-head attention module, denotes a multi-layer perceptron, denotes a multi-head attention module using regularization and a rotating window, and denote input and output features of a module, respectively, denotes down-sampled output features, denotes layer normalization.

6. The method of claim 1, wherein, In step (3), a multi-scale information fusion decoding module is built to extract features with strong distinguishing ability from the optimal feature graph to obtain the registration deformation field.

7. The method of claim 1, wherein, In step (4), a three-linear interpolation is applied to transform the floating image, and the floating image and the registration deformation field are input into a spatial transformation network to obtain a registration image; the three-linear interpolation is defined as follows: ； wherein, and denote the floating image and the registered image, respectively, denote a voxel in a voxel in an 8-pixel cubic neighborhood a voxel in three dimensions of the image, a registered deformation field.

8. The method of claim 1, wherein, In step (5), the boundary alignment is optimized by using a structural similarity loss in a similarity measurement objective function; the folding points are optimized by using a smoothing regular term in the similarity measurement objective function; The similarity measurement objective function is designed as: ； wherein, represents a similarity measure objective function, represents a structural similarity loss, is a fixed image and a registered image between them; represents an unrolling regularization term hyperparameter; The structural similarity loss between the fixed image and the registered image is defined as ​ ； where, and are defined as fixed and registered images, respectively; denotes the image the average pixel intensity over a neighborhood of cubic size ; denotes the image the average pixel intensity over a neighborhood of cubic size ; denotes the image the variance of pixel intensity over a neighborhood of cubic size ; denotes the image the variance of pixel intensity over a neighborhood of cubic size ; is the covariance of two images over a neighborhood of cubic size ; are two variables used to maintain stability of the equations; is the dynamic range of the pixel; the smoothness regularizer is defined as follows: ； ； ； wherein and represent the first and second derivatives of the deformation field respectively.

9. The method of claim 1, wherein, In step (6), the registration network is iteratively trained, an Adam optimizer is used for optimization, a learning rate parameter is set to 2e-4, a batch size is set to 1, a training number is set to 1000 cycles, each cycle is iterated 100 times, and the training is terminated when the threshold of the training number is reached.

10. The method of claim 1, wherein, In step (7), a coincidence degree coefficient is further applied to evaluate the registration result; The coincidence degree coefficient is defined as follows: ； M and W represent two images respectively; wherein, the closer the coincidence degree DSC is to 1, the more accurate the alignment of the anatomical structure of the image is.

Images