A Medical Image Registration Method Based on Edge Driving and Skeleton Inhibition

Abstract

The present invention relates to a medical image registration method based on edge driving and bone suppression, including: S1. Perform first preprocessing on the arterial-phase CT image and the venous-phase CT image to be registered in the pre-acquired original dataset respectively to obtain a second dataset; S2. Based on the pre-acquired trained coarse registration network structure and the original dataset, obtain a coarse registration dataset; S3. Perform second preprocessing on the coarse registration dataset to obtain the preprocessed coarse registration dataset; S4. Based on the pre-acquired trained fine registration network structure and the preprocessed coarse registration dataset, obtain the final fused image.

Description

Technical Field

[0001] The present invention relates to the technical field of medical images, and in particular to a medical image registration method based on edge driving and bone suppression. Background Art

[0002] CT enhanced scanning is to perform scanning after intravenous injection of water-soluble contrast agent. By using contrast agent mainly composed of iodine to increase the density difference between lesions and adjacent tissues, the lesion conditions can be reflected and the lesion display rate can be improved. The increase in the density of lesion tissue is called enhancement or intensification. According to different periods after the contrast agent for enhanced CT is injected into the blood vessel, enhanced CT can be divided into arterial phase image, venous phase image and plain scan phase image. MR imaging examination and nuclear magnetic resonance imaging examination are to place the human body in a strong magnetic field. By using hydrogen atoms contained in the human body, which can absorb energy in a specific magnetic field, generate nuclear magnetic signals during the process of releasing energy after being activated, and form images by receiving and processing the signals to achieve the purpose of medical diagnosis. When analyzing medical images, doctors need to analyze several images of the same patient together to obtain comprehensive information in multiple aspects of the patient. Since images in different phases are scanned at different times of the patient, the organs in the medical images will have different degrees of displacement, and the same organ cannot be completely consistent in the image space coordinates, which brings difficulties to clinicians for observing and analyzing the images.

[0003] Medical image registration refers to, for two medical images, seeking a (or a series of) spatial transformation for one medical image so that its structure in space can be consistent with that of the other image, that is, the same anatomical point of the human body has the same spatial position in the two images. The result of registration should make all anatomical points of the two images match, or the anatomical points with diagnostic significance and surgical interest can match. The traditional registration method optimizes an objective function for each pair of images, which is very time-consuming for large data sets or rich deformation models. At present, medical image registration based on deep learning has achieved remarkable results, and is superior to traditional iterative methods in both registration accuracy and speed. Guha Balakrishnan et al. proposed an unsupervised image registration method from voxel to voxel (VoxelMorph) based on convolutional neural network (CNN), which consists of a CNN network and a spatial transformer network (STN). The CNN network inputs two objects, one is called the floating image and the other is called the fixed image. The result output by the CNN network is a deformation field from the floating image to the fixed image, which represents the displacement amount of each spatial coordinate of the floating image. The spatial transformer network is responsible for deforming the floating image according to the deformation field output by the network and outputting the registered image. The network training optimizes the network parameters by maximizing the similarity measure between the registered image and the fixed image and minimizing the smoothness measure of the deformation field.

[0004] The VoxelMorph method can achieve good registration results when the corresponding anatomical points of two images are close and the overall anatomical structure is relatively regular. For example, in the registration of brain CT-MR images, when the network only needs to predict a small displacement, the registration effect is good. However, when the corresponding anatomical points of two images are far apart, there are large differences in anatomical structures, and there are many organs. For example, in the registration of abdominal CT images, the network needs to predict a large and complex displacement at the same time, and it is difficult to obtain good registration results. The reasons are as follows: First, the current similarity metric cannot accurately measure the alignment of image structures, and the alignment effect is poor. Second, the smoothness metric to prevent deformation field distortion will suppress the prediction of large displacements to avoid folding of the deformation field. Summary of the Invention

[0005] (1) Technical problems to be solved

[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a medical image registration method based on edge driving and bone suppression, which solves the technical problem of difficult to obtain good registration results in the prior art.

[0007] (2) Technical solutions

[0008] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0009] An embodiment of the present invention provides a medical image registration method based on edge driving and bone suppression, including:

[0010] S1. Perform first preprocessing on the to-be-registered arterial-phase CT image and venous-phase CT image in the pre-acquired original data set respectively to obtain a second data set;

[0011] The second data set includes: a floating image Mc corresponding to the to-be-registered arterial-phase CT image, a floating rough edge feature image Ms, a fixed image Fc corresponding to the venous-phase CT image, and a fixed rough edge feature image Fs;

[0012] S2. Obtain a rough registration data set based on the pre-acquired trained rough registration network structure and the original data set;

[0013] The rough registration network structure includes a rough registration network and a first spatial transformer connected to the rough registration network;

[0014] The rough registration network is a 3D U-Net network. The initial input channel of the 3D U-Net network is 4 channels, and the final output channel is 3 channels. All convolutional kernels of the 3D U-Net network are changed from ordinary convolutions to dilated convolutions;

[0015] S3. Perform a second preprocessing on the coarsely registered dataset to obtain the coarsely registered dataset after the second preprocessing;

[0016] S4. Based on the pre-obtained trained fine registration network structure and the coarsely registered dataset after the second preprocessing, obtain the final fused image;

[0017] The fine registration network structure includes a fine registration network and a second spatial transformer connected to the fine registration network;

[0018] The fine registration network is a VNet network. The initial input channels of the network are modified to include 4 input channels and 3 output channels, and an attention module is added to the skip connections.

[0019] Preferably,

[0020] The arterial-phase CT images to be registered in the original dataset have been pre-processed by affine transformation using the ANTs tool.

[0021] Preferably, S1 specifically includes:

[0022] S11. Set the window width and window level of the liver for the arterial-phase CT images and venous-phase CT images to be registered in the original dataset respectively, and convert the HU values of the arterial-phase CT images and venous-phase CT images to be registered in the original dataset to grayscale values respectively to obtain the first arterial-phase CT grayscale image and the first venous-phase CT grayscale image;

[0023] S12. According to formula (1), perform bone brightness attenuation on the first arterial-phase CT grayscale image and the first venous-phase CT grayscale image respectively to obtain the second arterial-phase CT grayscale image and the second venous-phase CT grayscale image;

[0024] The formula (1) is:

[0025] I = threshold+(I - threshold)*alpha, while I > threshold;

[0026] threshold is a preset attenuation threshold;

[0027] alpha is a preset attenuation factor;

[0028] S13. Perform denoising processing on the second arterial-phase CT grayscale image and the second venous-phase CT grayscale image respectively using the Gaussian low-pass filtering method to obtain the denoised arterial-phase CT image and the denoised venous-phase CT image;

[0029] S14. Fit the gray - level histogram of the arterial - phase CT denoised image to obtain a unimodal normal distribution curve of the arterial - phase CT denoised image, and further correct the unimodal normal distribution curve using formula (2) to obtain the third gray - level image of the arterial - phase CT;

[0030] The formula (2) is:

[0031]

[0032] where, μ M is the mean of the unimodal normal distribution curve of the fitting floating image Mc;

[0033] σ M is the variance of the unimodal normal distribution curve of the fitting floating image Mc;

[0034] μ F is the mean of the unimodal normal distribution curve of the fitting fixed image Fc;

[0035] σ F is the variance of the unimodal normal distribution curve of the fitting fixed image Fc;

[0036] S15. Sample the third gray - level image of the arterial - phase CT and the denoised image of the venous - phase CT at a size of 128*128*128 respectively to obtain the third gray - level image block of the arterial - phase CT and the denoised image block of the venous - phase CT;

[0037] S16. Use the Sobel operator to extract the rough edge features of the third gray - level image block of the arterial - phase CT and the denoised image block of the venous - phase CT respectively to obtain the floating rough edge feature image Ms and the fixed rough edge feature image Fs.

[0038] Preferably,

[0039] The first objective function of the rough registration network is:

[0040] L coarse =α1*L wt +α2*L sobel +α3*L ncc +α4*L gradient ;

[0041] where, α1 - α4 are weight coefficients;

[0042]

[0043] W [·] is the spectrogram calculated by the image in the wavelet domain;

[0044]

[0045] Among them, S [·] is the edge feature map of the image calculated based on the Sobel operator;

[0046]

[0047] Among them, Cov(F c , W c ) is the covariance of F c and W c ;

[0048] Var[·] is the variance;

[0049]

[0050] Among them, {x, y, z} represents the three directions of the three-dimensional image;

[0051] is the deformation field in the i direction;

[0052] is the first-order gradient

[0053] Preferably, before the S1, it further includes:

[0054] Using the pre-acquired first training data set to train the preset coarse registration network structure to obtain the trained coarse registration network structure;

[0055] The first training data set includes: the pre-acquired training floating image Mc, the training floating coarse edge feature image Ms, the training fixed image Fc, and the training fixed coarse edge feature image Fs;

[0056] Specifically, it includes:

[0057] Putting the 4 types of images, namely the training floating image Mc, the training floating coarse edge feature image Ms, the training fixed image Fc, and the training fixed coarse edge feature image Fs of the pre-acquired first training data set, into 4 input channels of the preset coarse registration network respectively, and the coarse registration network outputs the coarse deformation field;

[0058] Inputting the coarse deformation field and the training floating image Mc in the first training data set into the first spatial transformer, and the first spatial transformer outputs the coarsely registered image Wc;

[0059] Performing 64 rounds of gradient calculations according to the first objective function of the coarse registration network to obtain the trained coarse registration network structure.

[0060] Preferably, the S2 specifically includes:

[0061] S21. Put the data in the second dataset into the trained coarse registration network structure respectively, and the trained coarse registration network structure outputs the coarse registration image Wc corresponding to the second dataset;

[0062] S22. After respectively restoring the sizes of the coarse registration image Wc corresponding to the original dataset and the fixed image Fc in the original dataset, obtain the restored image of Wc and the restored image of Fc, and put them into the coarse registration dataset.

[0063] Preferably, the S3 specifically includes:

[0064] S31. Rotate, shear, horizontally and vertically flip each restored image of Wc and the restored image of Fc in the coarse registration dataset respectively to obtain the enhanced restored image of Wc and the enhanced restored image of Fc;

[0065] S32. Use the Canny algorithm to extract the fine edge features of the enhanced restored image of Wc and the enhanced restored image of Fc respectively to obtain the fine edge feature image of the enhanced restored image of Wc and the fine edge feature image of the enhanced restored image of Fc;

[0066] S33. Perform slider operations on the enhanced restored image of Wc, the enhanced restored image of Fc, the fine edge feature image of the enhanced restored image of Wc, and the fine edge feature image of the enhanced restored image of Fc respectively to obtain the second preprocessed coarse registration dataset;

[0067] The second preprocessed coarse registration dataset includes: the enhanced restored image block of Wc, the enhanced restored image block of Fc, the fine edge feature image block of the enhanced restored image of Wc, and the fine edge feature image block of the enhanced restored image of Fc.

[0068] Preferably, before the S1, it further includes:

[0069] Use the pre-obtained second training dataset to train the preset fine registration network structure to obtain the trained fine registration network structure;

[0070] The second training dataset includes: the pre-obtained trained enhanced restored image block of Wc, the trained enhanced restored image block of Fc, the trained fine edge feature image block of the enhanced restored image of Wc, and the trained fine edge feature image block of the enhanced restored image of Fc;

[0071] Specifically includes:

[0072] Put the four types of image patches, namely the enhanced Wc recovery image patches, the enhanced Fc recovery image patches, the fine edge feature image patches of the enhanced Wc recovery images, and the fine edge feature image patches of the enhanced Fc recovery images, which are trained in the pre-acquired second training dataset, into the four input channels of the preset fine registration network respectively. The fine registration network outputs a fine deformation field R;

[0073] According to the fine deformation field R, obtain the deformation field from the fixed image Fi to the floating image, and obtain the inverse deformation field R -1 ;

[0074] Input the fine deformation field R and the floating image patch M in the second training dataset i , into the second spatial transformer, and the second spatial transformer outputs a coarsely registered image;

[0075] Through the inverse deformation field R -1 and the fixed image F i and the input space transformer, transform the fixed image F i into an inverse floating image Calculate the second objective function and perform gradient calculation to update the network parameters, and obtain the trained fine registration network structure.

[0076] Preferably, the S4 specifically includes:

[0077] S41. Pass the second preprocessed coarsely registered dataset through the trained fine registration network structure to transform all floating image patches M i into finely registered arterial phase image patches S i ;

[0078] S42. Restore the size of the finely registered arterial phase image patches S i to the size of the arterial phase CT image to be registered, and obtain the registered arterial phase image S;

[0079] S43. Put the registered arterial phase image S into the red channel of the pre-set color image file, and put the venous phase CT image to be registered into the green channel to obtain the final fused image.

[0080] Preferably,

[0081] The fine registration network structure is specifically: the output of the decoding block of the VNet network passes through a convolutional layer and an upsampling layer to modify the number of channels of the feature map and the size of the feature map respectively, and then is added to the output of the encoding block and passes through a Relu activation layer and a convolutional layer in sequence. The output result is multiplied by the encoder output to obtain the result of the attention module;

[0082] The second objective function of the fine registration network is:

[0083] L refined = β1 * RL wt + β2 * RL canny + β3 * RL ncc + β4 * RL gradient + β5 * RL reverse

[0084] where β1 - β4 are weight coefficients;

[0085]

[0086] W [·] is the spectrogram calculated for the image in the wavelet domain;

[0087]

[0088] where Cov(S i , F i ) is the covariance of S i and F i ;

[0089] Var[·] is the variance;

[0090]

[0091]

[0092] where C[] is the edge feature map calculated for the image according to the Canny algorithm;

[0093]

[0094] (III) Advantageous Effects

[0095] The advantageous effects of the present invention are as follows: A medical image registration method based on edge driving and bone suppression according to the present invention preprocesses two images to eliminate interference signals and highlight key edges; constructs a cascaded neural network model from coarse registration to fine registration, the network outputs a deformation field, and obtains the registered image through the spatial transformation model STN, constructs an objective function including an edge feature metric-based similarity metric and a smoothness metric, and obtains the optimal network structure and parameters through training; fuses the registered image and the fixed image to obtain the final image. Description of the Drawings

[0096] Figure 1 is a flowchart of a medical image registration method based on edge driving and bone suppression according to the present invention;

[0097] Figure 2 is a schematic diagram of the coarse registration network structure in an embodiment of the present invention;

[0098] Figure 3 Schematic diagram of the refined matching network structure in the embodiment of the present invention. Specific implementation manner

[0099] To better explain the present invention for easy understanding, the present invention will be described in detail below in conjunction with the accompanying drawings through specific implementation manners.

[0100] To better understand the above technical solutions, the exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more clear and thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0101] Embodiment 1

[0102] See Figure 1 , this embodiment provides a medical image registration method based on edge driving and bone suppression, including:

[0103] S1. Perform first preprocessing on the to-be-registered arterial-phase CT image and venous-phase CT image in the pre-acquired original dataset respectively to obtain a second dataset.

[0104] The second dataset includes: a floating image Mc corresponding to the to-be-registered arterial-phase CT image, a floating rough edge feature image Ms, a fixed image Fc corresponding to the venous-phase CT image, and a fixed rough edge feature image Fs.

[0105] The to-be-registered arterial-phase CT image in the original dataset has been pre-processed by affine transformation using the ANTs tool.

[0106] In the specific application of this embodiment, S1 specifically includes:

[0107] S11. Set the window width and window level of the liver for the to-be-registered arterial-phase CT image and venous-phase CT image in the original dataset respectively, and convert the hu values of the to-be-registered arterial-phase CT image and venous-phase CT image in the original dataset into gray values respectively to obtain an arterial-phase CT first gray image and a venous-phase CT first gray image.

[0108] S12. According to formula (1), perform bone brightness attenuation on the arterial-phase CT first gray image and the venous-phase CT first gray image respectively to obtain an arterial-phase CT second gray image and a venous-phase CT second gray image.

[0109] The formula (1) is:

[0110] I = threshold + (I - threshold) * alpha, while I > threshold;

[0111] threshold is a preset attenuation threshold.

[0112] alpha is a preset attenuation factor.

[0113] S13. Respectively perform denoising processing on the second grayscale image of the arterial-phase CT and the second grayscale image of the venous-phase CT using the Gaussian low-pass filtering method to obtain the denoised arterial-phase CT image and the denoised venous-phase CT image.

[0114] S14. Fit the grayscale histogram of the denoised arterial-phase CT image to obtain the unimodal normal distribution curve of the denoised arterial-phase CT image, and further correct the unimodal normal distribution curve using formula (2) to obtain the third grayscale image of the arterial-phase CT.

[0115] The formula (2) is:

[0116]

[0117] where, μ M is the mean of the unimodal normal distribution curve of the fitted floating image Mc.

[0118] σ M is the variance of the unimodal normal distribution curve of the fitted floating image Mc.

[0119] μ F is the mean of the unimodal normal distribution curve of the fitted fixed image Fc.

[0120] σ F is the variance of the unimodal normal distribution curve of the fitted fixed image Fc.

[0121] S15. Respectively sample the third grayscale image of the arterial-phase CT and the denoised venous-phase CT image in a size of 128 * 128 * 128 to obtain the third grayscale image block of the arterial-phase CT and the denoised venous-phase CT image block.

[0122] S16. Use the Sobel operator to extract the rough edge features of the third grayscale image block of the arterial-phase CT and the denoised venous-phase CT image block respectively to obtain the floating rough edge feature image Ms and the fixed rough edge feature image Fs.

[0123] S2. Based on the pre-obtained trained rough registration network structure and the original data set, obtain the rough registration data set.

[0124] See Figure 2, in this embodiment, the rough registration network structure includes a rough registration network and a first spatial transformer connected to the rough registration network.

[0125] The rough registration network is a 3D U-Net network. The initial input channels of the 3D U-Net network are 4 channels, and the final output channels are 3 channels. All convolutional kernels of the 3D U-Net network are changed from ordinary convolutions to dilated convolutions.

[0126] The first objective function of the rough registration network is:

[0127] L coarse = α1 * L wt + α2 * L sobel + α3 * L ncc + α4 * L gradient .

[0128] Among them, α1 - α4 are weight coefficients.

[0129]

[0130] W [·] is the spectrogram calculated for the image in the wavelet domain.

[0131]

[0132] Among them, S [·] is the edge feature map calculated for the image based on the Sobel operator.

[0133]

[0134] Among them, Cov(F c , W c ) is the covariance of F c and W c .

[0135] Var[·] is the variance.

[0136]

[0137] Among them, {x, y, z} represents the three directions of the three-dimensional image.

[0138] is the deformation field in the i direction.

[0139] is the first-order gradient.

[0140] S3. Perform a second preprocessing on the rough registration dataset to obtain the preprocessed rough registration dataset.

[0141] S4. Obtain the final fused image based on the pre-acquired trained fine registration network structure and the second pre-processed coarse registration dataset;

[0142] See Figure 3 , in this embodiment, the fine registration network structure includes a fine registration network and a second spatial transformer connected to the fine registration network.

[0143] The fine registration network is a VNet network, the initial input channels of the network are modified to include 4 input channels and 3 output channels, and an attention module is added to the skip connection.

[0144] In the actual application of this embodiment, before the S1, it further includes:

[0145] Use the pre-acquired first training dataset to train the preset coarse registration network structure to obtain the trained coarse registration network structure.

[0146] The first training dataset includes: the pre-acquired training floating image Mc, the training floating coarse edge feature image Ms, the training fixed image Fc, and the training fixed coarse edge feature image Fs.

[0147] Specifically, it includes:

[0148] Put the 4 types of images, namely the training floating image Mc, the training floating coarse edge feature image Ms, the training fixed image Fc, and the training fixed coarse edge feature image Fs of the pre-acquired first training dataset, into the 4 input channels of the preset coarse registration network respectively, and the coarse registration network outputs a coarse deformation field.

[0149] Input the coarse deformation field and the training floating image Mc in the first training dataset into the first spatial transformer, and the first spatial transformer outputs the coarse registration image Wc.

[0150] Perform 64 rounds of gradient calculation according to the first objective function of the coarse registration network to obtain the trained coarse registration network structure.

[0151] Specifically, the S2 specifically includes:

[0152] S21. Put the data in the second dataset into the trained coarse registration network structure respectively, and the trained coarse registration network structure outputs the coarse registration image Wc corresponding to the second dataset.

[0153] S22. After restoring the sizes of the coarse registration image Wc corresponding to the original dataset and the fixed image Fc in the original dataset respectively, obtain the Wc restored image and the Fc restored image, and put them into the coarse registration dataset.

[0154] In a specific application, S3 specifically includes:

[0155] S31. Rotate, shear, horizontally and vertically flip each Wc restored image and Fc restored image in the rough registration dataset respectively to obtain the enhanced Wc restored image and the enhanced Fc restored image.

[0156] S32. Use the Canny algorithm to extract the fine edge features from the enhanced Wc restored image and the enhanced Fc restored image respectively to obtain the fine edge feature image of the enhanced Wc restored image and the fine edge feature image of the enhanced Fc restored image.

[0157] S33. Perform slider operations on the enhanced Wc restored image, the enhanced Fc restored image, the fine edge feature image of the enhanced Wc restored image, and the fine edge feature image of the enhanced Fc restored image simultaneously to obtain the second preprocessed rough registration dataset.

[0158] The second preprocessed rough registration dataset includes: the enhanced Wc restored image blocks, the enhanced Fc restored image blocks, the fine edge feature image blocks of the enhanced Wc restored image, and the fine edge feature image blocks of the enhanced Fc restored image.

[0159] In the actual application of this embodiment, before S1, it further includes:

[0160] Use the pre-obtained second training dataset to train the preset fine registration network structure to obtain the trained fine registration network structure.

[0161] The second training dataset includes: the pre-obtained trained enhanced Wc restored image blocks, trained enhanced Fc restored image blocks, trained fine edge feature image blocks of the enhanced Wc restored image, and trained fine edge feature image blocks of the enhanced Fc restored image.

[0162] Specifically, it includes:

[0163] Put the four types of image blocks of the trained enhanced Wc restored image blocks, trained enhanced Fc restored image blocks, trained fine edge feature image blocks of the enhanced Wc restored image, and trained fine edge feature image blocks of the enhanced Fc restored image in the pre-obtained second training dataset into the four input channels of the preset fine registration network respectively, and the fine registration network outputs the fine deformation field R.

[0164] According to the fine deformation field R, obtain the deformation field from the fixed image Fi to the floating image to obtain the inverse deformation field R -1 .

[0165] Input the refined deformation field R and the floating image patches M in the second training dataset i , into a second spatial transformer, which outputs a coarsely registered image.

[0166] Through the inverse deformation field R -1 and the fixed image Fi and the input spatial transformer, convert the fixed image Fi into an inverse floating image Calculate the second objective function and perform gradient calculation to update the network parameters and obtain a trained fine registration network structure.

[0167] Among them, in this embodiment, the specific steps of S4 include:

[0168] S41. Input the second pre-processed coarsely registered dataset into the trained fine registration network structure to convert all floating image patches M i into fine registered arterial phase image patches S i .

[0169] S42. Restore the size of the arterial phase CT image to be registered for the fine registered arterial phase image patches S i to obtain the registered arterial phase image S.

[0170] S43. Place the registered arterial phase image S into the red channel of a pre-set color image file, and place the venous phase CT image to be registered into the green channel to obtain the final fused image.

[0171] In this embodiment, the fine registration network structure is specifically as follows: The output of the decoding block of the VNet network passes through a convolutional layer and an upsampling layer to modify the number of channels of the feature map and the size of the feature map respectively, and then is added to the output of the encoding block and sequentially passes through a Relu activation layer and a convolutional layer. The output result is multiplied by the encoder output to obtain the result of the attention module.

[0172] The second objective function of the fine registration network is:

[0173] L refined =β1*RL wt +β2*RL canny +β3*RL ncc +β4*RL gradient +β5*RL reverse

[0174] Among them, β1-β4 are weight coefficients.

[0175]

[0176] W [·] is the spectrogram calculated for the image in the wavelet domain.

[0177]

[0178] Among them, Cov(S i , F i ) is the covariance of S i and F i .

[0179] Var[·] is the variance.

[0180]

[0181]

[0182] Among them, C [·] is the edge feature map calculated for the image · according to the Canny algorithm.

[0183]

[0184] A medical image registration method based on edge driving and bone suppression in this embodiment preprocesses two images to eliminate interference signals and highlight key edges; constructs a cascaded neural network model from coarse registration to fine registration, the network outputs a deformation field, and obtains the registered image through the spatial transformation model STN, constructs an objective function including an edge feature metric-based similarity metric and a smoothness metric, and obtains the optimal network structure and parameters through training; fuses the registered image and the fixed image to obtain the final image.

[0185] Embodiment 2

[0186] This Embodiment 2 provides a medical image registration method based on edge driving and bone suppression, including:

[0187] Step 1: Perform pre-registration on the to-be-registered arterial-phase CT image and venous-phase CT image, and perform affine transformation using the ANTs tool.

[0188] Step 2: Read the to-be-registered arterial-phase CT image and venous-phase CT image in the original dataset, and obtain the floating image Mc, fixed image Fc, floating rough edge feature image Ms, and fixed rough edge feature image Fs after the same preprocessing. The floating image Mc and the fixed image Fc are collectively referred to as the image I. The preprocessing steps are as follows:

[0189] Step 2.1: Set the window width and window level of the liver for the image I, and convert the hu value of the CT to a gray value.

[0190] Step 2.2: Set the attenuation threshold threshold and attenuation factor alpha, and perform bone brightness attenuation on the image I according to Equation 1 to reduce the gray value of the bone area.

[0191] Formula 1: I = threshold + (I - threshold) * alpha, while I > threshold

[0192] Step 2.3: Denoise the image I using the Gaussian low-pass filtering method.

[0193] Step 2.4: Fit the unimodal normal distribution curve of the image I, and correct the normal distribution curve of the floating image through the ξ formula (II) standard normal distribution.

[0194] The said Formula 2 is as follows:

[0195]

[0196] where μ M is the mean value of the unimodal normal distribution curve fitting the floating image Mc.

[0197] σ M is the variance of the unimodal normal distribution curve fitting the floating image Mc.

[0198] μ F is the mean value of the unimodal normal distribution curve fitting the fixed image Fc.

[0199] σ F is the variance of the unimodal normal distribution curve fitting the fixed image Fc.

[0200] Step 2.5: Reset the size of the image I, and uniformly resample it into a three-dimensional image block with a size of 128 * 128 * 128.

[0201] Step 2.6: Extract the rough edge feature image Is of the image I through the Sobel operator.

[0202] Step 3: Construct and train the rough registration network structure as shown in Figure 2 The specific steps are as follows:

[0203] Step 3.1: Construct a rough registration convolutional neural network, initialize the network weight parameters, and set the number of iterations.

[0204] Step 3.2: Construct a rough registration objective function, initialize the objective function parameters, and set the optimizer and learning rate.

[0205] Step 3.3: Put the floating image Mc, fixed image Fc, floating rough edge feature image Ms, and fixed rough edge feature image Fs used for training into the same channel and input them into the rough registration network, and obtain the predicted rough deformation field through network learning

[0206] Step 3.4: Obtain the coarsely registered image Wc through the coarse deformation field and the spatial transformer, calculate the objective function and perform gradient calculation, update the network parameters, and save the network model.

[0207] Step 4: Through the trained coarse registration network and the spatial transformer, convert all arterial phase images M in the original dataset into coarsely registered arterial phase images Wc to generate a coarsely registered dataset.

[0208] Step 5: Read the coarsely registered arterial phase CT images and venous phase CT images in the coarsely registered dataset, perform second preprocessing on them, and obtain the floating image patches Mi, fixed image Fi, floating fine-edge image patches Mic, and fixed fine-edge image patches Fic respectively.

[0209] Step 6: Construct and train a fine registration network. The structure of the fine registration network is as Figure 3 shown, and the specific steps are as follows:

[0210] Step 6.1: Construct a fine registration convolutional neural network, initialize the network weight parameters, and set the number of iterations.

[0211] Step 6.2: Construct a fine registration objective function, initialize the objective function parameters, and set the optimizer and learning rate.

[0212] Step 6.3: Put the floating image patches Mi, fixed image Fi, floating fine-edge image patches Mic, and fixed fine-edge image patches Fic used for training into the same channel and input them into the fine registration network to obtain the predicted fine deformation field through network learning.

[0213] Step 6.4: According to the predicted deformation field from the floating image to the fixed image, calculate the deformation field from the fixed image to the floating image, and call it the inverse deformation field.

[0214] Step 6.5: Obtain the finely registered image through the fine deformation field and the spatial transformer, and convert the fixed image into the inverse floating image through the inverse deformation field and the spatial transformer Calculate the objective function and perform gradient calculation, and update the network parameters.

[0215] Step 7: Through the trained fine registration network and the spatial transformer, convert all coarsely registered arterial phase image patches into finely registered arterial phase image patches.

[0216] Step 8: Restore the size of the finely registered arterial phase image patches to the original arterial phase image size.

[0217] Step 9: Define a color image, put the registered arterial phase image into the red channel, and the fixed image into the green channel to obtain the final fused image.

[0218] A medical image registration method based on edge driving and bone suppression in this embodiment. The image registration is divided into two steps: coarse registration and fine registration. The purpose of coarse registration is to align the edges of the skin, bones, fat, and large organs. The edges are relatively wide. The network directly inputs the whole image, and the network predicts large displacements so that the main structure and the organs with a relatively large area can be aligned first. The purpose of fine registration is to further align the internal tissues of the large organs and the edges of small organs. The edges are relatively narrow. First, the whole image is segmented into image blocks of the same size, and the network is responsible for predicting small displacements, which will more precisely align the fine edges and make the registration result more precise.

[0219] In this embodiment, the bone suppression method is used in both image preprocessings. By attenuating the brightness of the bones, it is avoided that the network focuses on the bone area due to the too large gray value of the bones, thus ignoring the organ area with a smaller gray value but belonging to the key area of registration.

[0220] In the method of this embodiment during the coarse registration preprocessing, the image is smoothed. Gaussian filtering is used for image smoothing, which can remove noise to a certain extent, reduce the resolution of the image, weaken the fine edges in the image, and leave the wide edges as the key points for registration to be highlighted.

[0221] In the method of this embodiment during both network preprocessings, the gray value correction method is used. Under the influence of the contrast agent, there is a certain gray value difference in the display of the liver gray value between the venous phase and arterial phase images. By fitting the gray value images of the arterial phase and venous phase to a normal distribution and then correcting the gray value image of the arterial phase to approach the gray value image of the venous phase, it helps to eliminate the gray value difference between the two images and reduce the network learning difficulty.

[0222] In the method of this embodiment, wavelet loss is used in both network trainings. Since the hu values of the key anatomical points of the enhanced CT image pairs caused by the contrast agent do not exactly correspond, directly calculating the mean square error cannot reflect the alignment situation. The wavelet loss can reflect the transformation of the hu values based on the wavelet spectrogram and can be used as a measure of the edge alignment degree to a certain extent, and at the same time better eliminates the influence of the hu value difference.

[0223] The method of this embodiment designs a fine registration network based on the improved VNet. A convolutional neural network based on the skip connection of the attention module is designed. The attention module calculates the attention scores of each channel output by the encoder, making the key feature channels with high scores more easily learned by the decoder, which improves the learning ability of the network.

[0224] In the method of this embodiment, a rough edge feature image is added as network input and a rough edge loss is added during the coarse registration process. The edge feature map is directly calculated by the Sobel operator for network input and the edge loss is obtained. The edges obtained from the calculated edge feature map are relatively wide, which is very suitable for aligning the edges of the overall structure to achieve the alignment effect of coarse registration.

[0225] In the method of this embodiment, a fine edge feature image is added as network input and a fine edge loss is added during the fine registration process. The edge feature map is calculated by the Canny algorithm for network input and the edge loss is obtained. The calculated edge feature map is refined and threshold-truncated, shifting the focus of network learning to the edges of internal tissues and small organs, which is very suitable for aligning fine edges to achieve the purpose of fine registration.

[0226] The method of this embodiment uses an inverse deformation field loss in fine registration. The inverse deformation field loss ensures that the predicted deformation field can maintain the property of diffeomorphism, further avoiding the distortion of the deformation field and reducing the folding of the deformation field.

[0227] The method of the embodiment establishes a step-by-step alignment registration method for edges from rough to fine, using different preprocessings, network structures and objective functions for rough registration and fine registration, and more pertinently and accurately completing the corresponding registration requirements. Among them, the convolutional network is improved by adding an attention mechanism, which improves the learning ability of the network, improves the registration accuracy, solves the complex problem of enhancing the registration image and simultaneously predicting the intricate deformation fields of large displacements and small displacements, and maintains the smoothness of the deformation field, avoiding distortion.

[0228] Generally speaking, the use of the medical image registration network model based on edge driving and bone suppression divides the prediction of the original complex deformation field into two sub-problems, and improves the registration accuracy step by step in a progressive manner, significantly improving the registration effect.

[0229] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system or a computer program product. Therefore, the present invention can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0230] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, and the combination of processes and / or blocks in the flowcharts and / or block diagrams, can be realized by computer program instructions.

[0231] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a claim listing several means, several of these means can be embodied by the same hardware. The use of the terms first, second, third, etc. is for convenience only and does not denote any order. These terms can be construed as part of the name of the element.

[0232] In addition, it should be noted that in the description of this specification, the descriptions of terms such as "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without conflict, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0233] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications after learning the basic creative concept. Therefore, the claims should be construed to cover the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0234] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention should also cover these modifications and variations.

Claims

1. A medical image registration method based on edge driving and bone suppression, characterized in that Including: S1. Perform first preprocessing on the arterial-phase CT images and venous-phase CT images to be registered in the pre-acquired original dataset respectively, and obtain a second dataset; The second dataset includes: a floating image Mc corresponding to the arterial-phase CT image to be registered, a floating rough edge feature image Ms, a fixed image Fc corresponding to the venous-phase CT image, and a fixed rough edge feature image Fs; S2. Based on the pre-acquired trained rough registration network structure and the original dataset, obtain a rough registration dataset; The rough registration network structure includes a rough registration network and a first spatial transformer connected to the rough registration network; The rough registration network is a 3D U-Net network. The initial input channels of the 3D U-Net network are 4 channels, and the final output channels are 3 channels. All convolutional kernels of the 3D U-Net network are changed from ordinary convolutions to dilated convolutions; S3. Perform second preprocessing on the rough registration dataset to obtain a second preprocessed rough registration dataset; S4. Based on the pre-acquired trained fine registration network structure and the second preprocessed rough registration dataset, obtain the final fused image; The fine registration network structure includes a fine registration network and a second spatial transformer connected to the fine registration network; The fine registration network is a VNet network. The initial input channels of the network are modified to include 4 input channels and 3 output channels, and an attention module is added to the skip connections; Before the S1, it further includes: Using the pre-acquired first training dataset to train a preset rough registration network structure to obtain a trained rough registration network structure; The first training dataset includes: a pre-acquired trained floating image Mc, a trained floating rough edge feature image Ms, a trained fixed image Fc, and a trained fixed rough edge feature image Fs; Specifically including: Put the 4 types of images, namely the trained floating image Mc, the trained floating rough edge feature image Ms, the trained fixed image Fc, and the trained fixed rough edge feature image Fs in the pre-acquired first training dataset, into the 4 input channels of the preset rough registration network respectively. The rough registration network outputs a rough deformation field; Input the rough deformation field and the trained floating image Mc in the first training dataset into the first spatial transformer, and the first spatial transformer outputs a rough registration image Wc; Perform 64 rounds of gradient calculation according to the first objective function of the rough registration network to obtain a trained rough registration network structure.

2. The method according to claim 1, wherein The arterial-phase CT image to be registered in the original dataset has been pre-processed by affine transformation using the ANTs tool.

3. The method according to claim 2, wherein The S1 specifically includes: S11. Set the window width and window level of the liver for the arterial-phase CT image and venous-phase CT image to be registered in the original dataset respectively, and convert the hu values of the arterial-phase CT image and venous-phase CT image to be registered in the original dataset into gray values respectively to obtain an arterial-phase CT first gray image and a venous-phase CT first gray image; S12. According to formula (1), perform bone brightness attenuation on the arterial-phase CT first grayscale image and the venous-phase CT first grayscale image respectively to obtain the arterial-phase CT second grayscale image and the venous-phase CT second grayscale image; The formula (1) is: I = threshold + (I - threshold) * alpha, while I > threshold; threshold is a preset attenuation threshold; alpha is a preset attenuation factor; S13. Perform denoising processing on the arterial-phase CT second grayscale image and the venous-phase CT second grayscale image respectively using the Gaussian low-pass filtering method to obtain the arterial-phase CT denoised image and the venous-phase CT denoised image; S14. Fit the grayscale histogram of the arterial-phase CT denoised image to obtain the unimodal normal distribution curve of the arterial-phase CT denoised image, and further correct the unimodal normal distribution curve using formula (2) to obtain the arterial-phase CT third grayscale image; The formula (2) is: where μ M is the mean of the unimodal normal distribution curve fitting the floating image Mc; σ M The variance for fitting the single-peak normal distribution curve of the floating image Mc; μ F is the mean of the single-peak normal distribution curve fitting the fixed image Fc; σ F is the variance of the unimodal normal distribution curve that fits the fixed image Fc; S15. Sample the arterial-phase CT third grayscale image and the venous-phase CT denoised image respectively with a size of 128*128*128 to obtain the arterial-phase CT third grayscale image block and the venous-phase CT denoised image block respectively; S16. Use the Sobel operator to extract the rough edge features of the arterial-phase CT third grayscale image block and the venous-phase CT denoised image block respectively to obtain the floating rough edge feature image Ms and the fixed rough edge feature image Fs.

4. The method according to claim 3, wherein The first objective function of the rough registration network is: L coarse = α1 * L wt + α2 * L sobel + α3 * L ncc + α4 * L gradient ; where, α1 - α4 are weight coefficients; W [·] is a spectrogram calculated in the wavelet domain for the image; Among them, S [·] is the edge feature map calculated based on the Sobel operator for the image; Among them, Cov(F c , W c ) is the covariance of F c and W c ; Var[·] is the variance; where, {x, y, z} represents the three directions of the three-dimensional image; The deformation field in the i direction; is a gradient.

5. The method according to claim 4, wherein The S2 specifically includes: S21. Put the data in the second data set into the trained rough registration network structure respectively, and the trained rough registration network structure outputs the rough registration image Wc corresponding to the second data set; S22. After restoring the sizes of the rough registration image Wc corresponding to the original data set and the fixed image Fc in the original data set respectively, obtain the Wc restored image and the Fc restored image, and put them into the rough registration data set.

6. The method according to claim 5, wherein The S3 specifically includes: S31. Perform rotation, shearing, horizontal and vertical flipping operations on each Wc restored image and Fc restored image in the rough registration data set respectively to obtain the enhanced Wc restored image and the enhanced Fc restored image; S32. Use the Canny algorithm to extract the fine edge features of the enhanced Wc restored image and the enhanced Fc restored image respectively to obtain the fine edge feature image of the enhanced Wc restored image and the fine edge feature image of the enhanced Fc restored image; S33. Perform slider operations on the enhanced Wc restored image, the enhanced Fc restored image, the fine edge feature image of the enhanced Wc restored image, and the fine edge feature image of the enhanced Fc restored image respectively at the same time to obtain the second preprocessed rough registration data set; The second pre-processed roughly registered data set includes: enhanced Wc restored image patches, enhanced Fc restored image patches, fine edge feature image patches of the enhanced Wc restored image, and fine edge feature image patches of the enhanced Fc restored image.

7. The method according to claim 6, characterized in that, Before the S1, it further includes: Using a pre-acquired second training data set to train a preset fine registration network structure to obtain a trained fine registration network structure; The second training data set includes: pre-acquired trained enhanced Wc restored image patches, trained enhanced Fc restored image patches, trained fine edge feature image patches of the enhanced Wc restored image, and trained fine edge feature image patches of the enhanced Fc restored image; Specifically, it includes: Putting the four types of image patches of the pre-acquired trained enhanced Wc restored image patches, trained enhanced Fc restored image patches, trained fine edge feature image patches of the enhanced Wc restored image, and trained fine edge feature image patches of the enhanced Fc restored image in the second training data set into four input channels of a preset fine registration network respectively, and the fine registration network outputs a fine deformation field R; Obtain a fixed image F according to the fine deformation field R i The deformation field from the floating image to obtain an inverse deformation field R -1 ; Input the refined deformation field R and the floating image patch M in the second training dataset i into a second spatial transformer, which outputs a coarsely registered image; Through the inverse deformation field R -1 and the fixed image F i and the input space converter to convert the fixed image F i into an inverse floating image Calculate the second objective function and perform gradient calculation, update the network parameters, and obtain the trained fine registration network structure.

8. The method according to claim 7, characterized in that The S4 specifically includes: S41. Convert all floating image patches M in the second pre-processed roughly registered data set through the trained fine registration network structure i into fine registered arterial phase image patches S i ; S42. Restore the size of the arterial phase CT image to be registered for the image block S after fine registration to obtain the registered arterial phase image S; i ​ S43: Putting the registered arterial phase image S into the red channel of a pre-set color image file, and putting the unregistered venous phase CT image into the green channel to obtain a final fused image.

9. The method according to claim 8, wherein The fine registration network structure is specifically: the output of the decoding block of the VNet network passes through a convolutional layer and an upsampling layer, respectively modifying the number of channels of the feature map and the size of the feature map, and then adding it to the output of the encoding block and passing through a Relu activation layer and a convolutional layer in sequence, and the output result is multiplied by the encoder output to obtain the result of the attention module; The second objective function of the fine registration network is: L refined = β1 * RL wt + β2 * RL canny + β3 * RL ncc + β4 * RL gradient + β5 * RL reverse wherein, β1-β4 are weight coefficients; w [·] is a spectrogram calculated in the wavelet domain for the image; where Cov(S i , F i ) is the covariance of S i and F i ; Var[·] is variance; Among them, C [·] is the edge feature map calculated based on the Canny algorithm for the image;

Images