A feature-based liver registration method for CT and ultrasound images

Abstract

The present invention discloses a feature-based liver registration method for CT and ultrasound images. The method first preprocesses the CT image and extracts liver organ contour features from it. An ultrasound image is acquired and extracted from it. The CT and ultrasound images are then quickly and roughly matched based on the liver organ area features to select several candidate CT slices. The candidate CT slices are then registered one by one with the ultrasound image to obtain a spatial transformation matrix, which is then used to transform the candidate CT slices into the ultrasound image coordinate system. The image similarity between the transformed candidate CT slices and the two-dimensional ultrasound image is calculated to obtain a CT slice that accurately matches the two-dimensional ultrasound image. The present invention can improve the accuracy and efficiency of registration.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image registration, and in particular to a feature-based liver registration method for CT images and ultrasound images. Background Art

[0002] With the rapid development of computer technology and medical imaging technology, the processing of medical images using image processing technology has shown important application value and development prospects in clinical practice. Different medical images contain different diagnostic information. Clinically, it is necessary to fuse and compare different medical images and comprehensively utilize the information contained in different images, so that doctors can understand the condition from all angles and make the diagnosis more accurate. As a prerequisite for achieving image fusion, image registration technology is widely used in the diagnosis of clinical diseases, the determination of treatment plans, and the evaluation of treatment effects, and is a popular research direction.

[0003] Medical image registration technology is an important component of medical image processing. By fusing and comparing information from different medical images, registration plays a vital role in clinical diagnosis, surgical navigation, surgical planning, and postoperative evaluation. With the rapid development of medical imaging technology, multimodal medical image registration plays a crucial role in many clinical applications. Therefore, the study of multimodal medical image registration algorithms has important theoretical and clinical value.

[0004] Medical image registration can be categorized into grayscale-based and feature-based methods. Grayscale-based registration methods directly use the grayscale of the entire image to measure the similarity between two images. Search methods (such as maximum mutual information, correlation, conditional entropy, and joint entropy) are used to find the point where the similarity measure is maximized or minimized, thereby determining the transformation model parameters between the two images. Grayscale-based registration methods have disadvantages such as high computational complexity, long registration time, sensitivity to scaling, rotation, and distortion, and neglect of spatially relevant image information. Feature-based registration methods generally extract a feature from the two images. Commonly used image features include point features, line segments, edges, contours, closed regions, and statistical moments. While feature-based registration methods offer advantages such as simple operation, fast registration speed, and high accuracy, they also have disadvantages such as the need for manual intervention and the difficulty in acquiring feature points. Summary of the Invention

[0005] The present invention aims to solve the problem that existing feature-based registration methods require manual intervention and are difficult to obtain feature points, and provides a feature-based liver registration method for CT images and ultrasound images.

[0006] To solve the above problems, the present invention is achieved through the following technical solutions:

[0007] A feature-based liver registration method for CT images and ultrasound images includes the following steps:

[0008] Step 1: Acquire a CT image of the human liver and extract the contour features of the liver organ from each CT slice of the CT image;

[0009] Step 2: Acquire an ultrasonic image of the human liver and extract the contour features of the liver organ from the ultrasonic image;

[0010] Step 3: Based on the contour features of the liver organ extracted in steps 1 and 2, all CT slices of the CT image are roughly registered with the ultrasound image to obtain roughly registered CT slices, namely;

[0011] Step 3.1, calculating the area of ​​the contour features of the liver organ in each CT slice of the CT image to obtain the surface area of ​​the liver organ in each CT slice; and simultaneously calculating the area of ​​the contour features of the liver organ in the ultrasound image to obtain the surface area of ​​the liver organ in the ultrasound image;

[0012] Step 3.2, first calculating the absolute value of the difference between the surface areas of the liver organ between every two adjacent CT slices of the CT image, and then calculating the average of the absolute values ​​of the difference between the surface areas of the liver organ between every two adjacent CT slices as the average difference of the CT image;

[0013] Step 3.3, calculating the absolute value of the difference between the surface area of ​​the liver organ in each CT slice of the CT image and the surface area of ​​the liver organ in the ultrasound image, as the comparative difference value of each CT slice;

[0014] Step 3.4, comparing the comparison difference value of each CT slice of the CT image with the average difference value of the CT image, and retaining the CT slices in the CT image whose comparison difference value of the CT slices is smaller than the average difference value of the CT image, thereby obtaining coarsely registered CT slices;

[0015] Step 4: Based on the contour features of the liver organ extracted in steps 1 and 2, the coarsely registered CT slice of the CT image obtained in step 3 is finely registered with the ultrasound image to obtain a finely registered CT slice, that is;

[0016] Step 4.1, performing image space transformation on each coarsely registered CT slice using the mean square error between the binary matrix of the CT slice image and the binary matrix of the ultrasound image as the objective function, so as to transform each coarsely registered CT slice into the ultrasound space coordinate system, thereby obtaining a spatially transformed coarsely registered CT slice;

[0017] Step 4.2: Calculate the overlap ratio between the coarsely registered CT slice after spatial transformation and the ultrasound image using the dice coefficient, and use the coarsely registered CT slice after spatial transformation with the highest overlap ratio as the finely registered CT slice.

[0018] The specific process of step 1 above is as follows:

[0019] Step 1.1, adjust the window for each CT slice;

[0020] Step 1.2, resample each windowed CT slice using bilinear interpolation;

[0021] Step 1.3: Input each resampled CT slice into the UNet convolutional neural network to obtain a binary mask of the liver organ, thereby extracting the contour features of the liver organ in each CT slice.

[0022] The specific process of step 2 above is as follows:

[0023] Step 2.1, adjusting the window of the ultrasound image;

[0024] Step 2.2, resampling the windowed ultrasound image using bilinear interpolation;

[0025] Step 2.3: Use a pixel gradient segmentation method to segment the resampled ultrasound image to obtain a binary mask, i.e., a contour feature, of the liver organ, thereby extracting the contour feature of the liver organ in the ultrasound image.

[0026] The specific process of step 2.3 above is as follows: first, use two median filters to denoise the resampled ultrasound image; then use the Canny edge detection algorithm to extract the edge features of the liver organ contour; then binarize the contour edge image and fill the closed area to obtain the binary mask of the liver organ in the ultrasound image.

[0027] In step 4.2 above, the calculation formula for the overlap rate Dice is:

[0028]

[0029] Where, I US is the binary matrix of the ultrasound image, I′ CT is the binary matrix of the coarsely registered CT slice after spatial transformation.

[0030] Compared with the prior art, the present invention has the following characteristics:

[0031] 1. A major difficulty in multimodal image registration is that for the same structure, images of different modalities have no obvious correlation in pixel values. The present invention uses a segmentation method based on liver organ features to extract binary masks of the liver organs from CT slices and ultrasound images. The common feature description of the binary mask makes the pixel values ​​of the liver organs in CT slices and two-dimensional ultrasound correlated, thereby simplifying the multimodal registration problem to a single-modal registration problem.

[0032] 2. A coarse registration is added before fine registration. By comparing the liver area in the ultrasound image with the liver area in the CT slice, CT slices with similar liver areas to the ultrasound image can be searched efficiently, and most CT slices can be excluded, thereby greatly reducing the number of subsequent registrations and improving the computational efficiency of the overall framework.

[0033] 3. Considering that the essence of binary image registration is registration based on the correlation of pixel values ​​between two images, compared with grayscale images, the correlation of pixel values ​​of binary images is more obvious. The present invention adopts mean square error (MSE) as the similarity measure of the registration framework, which can take into account both computational efficiency and registration effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of a feature-based liver registration method for CT and ultrasound images. DETAILED DESCRIPTION

[0035] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to specific examples.

[0036] A feature-based liver registration method for CT and ultrasound images, such as Figure 1 As shown, it includes the following steps:

[0037] Step 1: In the preoperative stage, a CT image of the human liver is acquired, and the contour features of the liver organ are extracted from each CT slice of the CT image.

[0038] A CT image is composed of multiple CT slices. Each CT slice is a two-dimensional image, and the CT image composed of all CT slices is a three-dimensional image. To this end, it is necessary to perform feature extraction on each CT slice of the CT image to obtain the contour features of the liver organ in each CT slice. The specific process is as follows:

[0039] Step 1.1: Adjust the window of the CT slice. Since this is liver registration, adjust the window width and window level to 400 Hu and 60 Hu to improve the distinction between the liver and surrounding organs.

[0040] Step 1.2: Resample the windowed CT slices using bilinear interpolation to restore the true size of the organ in physical space.

[0041] Step 1.3: Input the resampled CT slices into a UNet convolutional neural network to obtain a binary mask of the liver. Because the resolution of CT slices is significantly different from the pixel values ​​of the liver region and the surrounding tissue, the present invention uses an existing UNet convolutional neural network to segment the liver region in CT slices. Specifically, the windowed and resampled CT slices are input into the UNet network to obtain a binary mask of the liver. This binary mask of the liver is the contour feature of the liver.

[0042] Step 2: During the operation, an ultrasound image of the human liver is collected, and contour features of the liver organ are extracted from the ultrasound image.

[0043] The ultrasound image of the human liver is composed of a pair of ultrasound images, which are two-dimensional images. Therefore, it is only necessary to perform feature extraction on this ultrasound image to obtain the contour features of the liver organ in this ultrasound image. The specific process is as follows:

[0044] Step 2.1: Adjust the ultrasound image window. Since liver registration is being performed, adjust the window width and window level to 400 Hu and 60 Hu to improve the distinction between the liver and surrounding organs.

[0045] Step 2.2: Resample the windowed ultrasound image using bilinear interpolation to restore the true size of the organ in physical space.

[0046] Step 2.3: Use the pixel gradient segmentation method to obtain a binary mask of the liver organ on the resampled ultrasound image. The process of the pixel gradient segmentation method is as follows: First, due to the presence of noise shadows in the ultrasound image, the present invention uses two median filters for denoising. The first filter uses a small-size kernel to remove the overall white noise of the image while ensuring the gradient information of the contour edge as much as possible. The second filter uses a large-size kernel to remove the noise on the contour edge. Then, the Canny edge detection algorithm is used to extract the contour edge features of the liver organ. Finally, the contour edge image is binarized and the closed area is filled to obtain a binary mask of the liver organ in the ultrasound image. The binary mask of the liver organ is the contour feature of the liver organ.

[0047] Step 3: perform coarse registration on the 3D CT image and the 2D ultrasound image to obtain a CT slice in the 3D CT image that is coarsely registered with the 2D ultrasound image.

[0048] A major difficulty in multimodal image registration is that for the same structure, images of different modalities have no obvious correlation in pixel values. To this end, the present invention extracts binary masks of the liver organs from CT slices and ultrasound images based on a segmentation method based on liver organ features. The common feature description of the binary mask makes the pixel values ​​of the liver organs in CT slices and two-dimensional ultrasound correlated, thereby simplifying the multimodal registration problem to a binary image registration problem.

[0049] To improve the registration speed of 3D CT to 2D ultrasound, this paper proposes a new method for fast coarse slice matching based on liver organ area features. Since the liver organ contour features are binary masks, the surface area of ​​the contour features can be obtained by counting the number of pixels in the contour features. This allows the matching relationship between the CT slices in the 3D CT image and the 2D ultrasound image to be preliminarily determined by comparing the liver area similarities between the 3D CT and 2D ultrasound images without the need for registration. The 3D CT to 2D ultrasound coarse matching process is as follows:

[0050] Step 3.1: Calculate the area of ​​the contour features of the liver organ in each CT slice of the CT image to obtain the surface area of ​​the liver organ in each CT slice. Where i = 1, 2, ..., n, n is the number of CT slices in the CT image. At the same time, the area of ​​the contour feature of the liver organ in the ultrasound image is calculated to obtain the surface area S of the liver organ in the ultrasound image. US .

[0051] Step 3.2: Calculate the mean difference of CT images

[0052] First, take adjacent CT slices one by one and calculate the absolute value of the difference in the surface area of ​​the liver organ between each two adjacent CT slices of the CT image.

[0053] Then calculate the average of the absolute values ​​of the differences in the surface areas of the liver organs between every two adjacent CT slices as the average difference of the CT images.

[0054] Step 3.3: Comparison of the difference between the CT slices

[0055] Calculate the absolute value of the difference between the surface area of ​​the liver organ in each CT slice of the CT image and the surface area of ​​the liver organ in the ultrasound image

[0056] Step 3.4: Compare the difference of each CT slice of the CT image The mean difference between the CT images For comparison: If , the i-th CT slice is retained, and the retained CT slice is the coarsely registered CT slice; otherwise, the i-th CT slice is deleted.

[0057] In the coarse matching process, the liver organ area difference between the ultrasound image and the CT slice is compared with the average liver organ area difference of the CT slice. This can efficiently search for the candidate CT slice that is closest to the ultrasound image and exclude most CT slices, thereby greatly reducing the number of subsequent alignments and improving the computational efficiency of the overall framework.

[0058] Step 4: Perform precise registration on the three-dimensional CT image and the two-dimensional ultrasound image to obtain a CT slice in the three-dimensional CT image that is precisely registered with the two-dimensional ultrasound image.

[0059] After obtaining several retained CT slices (roughly registered CT slices) through 3D CT-2D ultrasound coarse matching, it is necessary to perform fine registration of these coarsely registered CT slices with the ultrasound one by one. Existing image registration methods can be used for fine registration. The specific process of the fine registration method used in the present invention is as follows:

[0060] Step 4.1: First, the mean square error between the binary matrix of the CT slice image and the binary matrix of the ultrasound image is used as the objective function. The Adam optimizer updates the registration parameters according to the gradient of the objective function until the objective function converges to the global optimal value or the number of iterations reaches the set value. Then, the rigid transformation matrix from the CT slice to the ultrasound image is obtained. Then, the rigid transformation matrix from the CT slice to the ultrasound image is used to transform each coarsely registered CT slice into the ultrasound space coordinate system to obtain the coarsely registered CT slice after spatial transformation.

[0061] Considering the nature of binary image registration, which is based on the correlation of pixel values ​​between two images, binary images exhibit more pronounced correlations than grayscale images. Based on this characteristic, the present invention uses the mean square error (MSE) of the binary matrices of CT slice and ultrasound images as the similarity metric for the registration framework, which balances computational efficiency and registration effectiveness. Based on this selected similarity metric, the present invention employs a rigid transformation approach to transform the CT slice-to-ultrasound image space.

[0062] Assume that the ultrasound image is the fixed image and the CT slice is the moving image. The rigid transformation matrix from the CT slice to the ultrasound image is obtained by iteratively optimizing Equation (1). The Adam optimizer updates the registration parameters according to the gradient of the mean square error (MSE) of the objective function in Equation (2) until the objective function converges to the global optimal value or the number of iterations reaches the set maximum value.

[0063]

[0064]

[0065] Where, I US is the binary matrix of the ultrasound image, I CT is the binary matrix of the CT slice before spatial transformation, is the rigid transformation matrix from CT slice to ultrasound image, MSE(I CT ,I US ) is the mean square error of the binary matrices of the CT slice and the ultrasound image, N is the number of pixels (since the actual size of the liver organ in the CT slice and ultrasound image has been restored by image resampling in the previous step, the number of pixels in the CT slice and ultrasound image is the same, both N). is the i-th grayscale value in the binary matrix of the CT slice, is the grayscale value of the i-th element in the binary matrix of the ultrasound image.

[0066] After the rigid transformation matrix between ultrasound and the selected CT slice is calculated by the Adam optimizer, the rigid transformation matrix can be used to transform the coarsely registered CT slice into the ultrasound space coordinate system, that is,

[0067]

[0068] Where, I CT is the binary matrix of the coarsely registered CT slice after spatial transformation, I′ CT is the binary matrix of the coarsely registered CT slice after spatial transformation, is the rigid transformation matrix from CT slice to ultrasound image.

[0069] Step 4.2: Calculate the overlap ratio between the coarsely registered CT slice after spatial transformation and the ultrasound image, and use the coarsely registered CT slice after spatial transformation with the highest overlap ratio as the finely registered CT slice.

[0070] In order to measure the similarity of liver organ features between the two images, the Dice Coefficient (DC) was used to calculate the overlap rate of the binary mask of the coarsely registered CT slice and ultrasound image after spatial transformation. The formula is as follows:

[0071]

[0072] Where, I US is the binary matrix of the ultrasound image, I′ CT is the binary matrix of the coarsely registered CT slice after spatial transformation.

[0073] CT slice I′ with the highest Dice overlap CT That is, the CT slice that is closest to the ultrasound image (precisely registered CT slice), thereby determining the exact matching relationship between the three-dimensional CT image and the two-dimensional ultrasound image.

[0074] To address the lack of real-time and high-resolution medical imaging guidance for ultrasound-guided liver cancer ablation surgery, the present invention proposes to register and fuse preoperative 3D CT images with real-time, intraoperative 2D ultrasound images. Combining the advantages of high resolution CT images and the real-time nature of ultrasound images, this method uses CT-ultrasound fusion image guidance to clearly and in real time locate the lesion, thereby guiding clinicians through the puncture procedure and improving puncture accuracy. Preoperatively, the present invention preprocesses the CT image and extracts liver organ contour features from it. During the intraoperative phase, ultrasound images are acquired in real time and extracted from them. CT and ultrasound images are quickly and roughly matched based on liver organ area characteristics to select several candidate CT slices. The candidate CT slices are then registered with the ultrasound image one by one to obtain a spatial transformation matrix, which is used to transform the candidate CT slices into the ultrasound image coordinate system. By calculating the image similarity between the transformed candidate CT slices and the 2D ultrasound image, a CT slice that precisely matches the 2D ultrasound image is obtained. Image fusion techniques can then be used to fuse the precisely registered CT slices with the ultrasound image. This image fusion process is not the focus of this invention and can be implemented using existing fusion algorithms.

[0075] It should be noted that although the embodiments of the present invention described above are illustrative, they are not intended to limit the present invention. Therefore, the present invention is not limited to the above-mentioned specific embodiments. Without departing from the principles of the present invention, any other embodiments obtained by those skilled in the art under the guidance of the present invention are deemed to be within the protection of the present invention.

Claims

1. A feature-based liver registration method for CT images and ultrasound images, characterized in that: The steps are as follows: Step 1: Acquire a CT image of the human liver and extract the contour features of the liver organ, i.e., a binary mask, from each CT slice of the CT image; Step 2: Acquire an ultrasound image of the human liver, and extract the contour features of the liver organ, i.e., a binary mask, from the ultrasound image; Step 3: Based on the contour features of the liver organ extracted in steps 1 and 2, all CT slices of the CT image are roughly registered with the ultrasound image to obtain roughly registered CT slices, namely; Step 3.1, calculating the area of ​​the contour features of the liver organ in each CT slice of the CT image to obtain the surface area of ​​the liver organ in each CT slice; and simultaneously calculating the area of ​​the contour features of the liver organ in the ultrasound image to obtain the surface area of ​​the liver organ in the ultrasound image; Step 3.2, first calculating the absolute value of the difference between the surface areas of the liver organ between every two adjacent CT slices of the CT image, and then calculating the average of the absolute values ​​of the difference between the surface areas of the liver organ between every two adjacent CT slices as the average difference of the CT image; Step 3.3, calculating the absolute value of the difference between the surface area of ​​the liver organ in each CT slice of the CT image and the surface area of ​​the liver organ in the ultrasound image, as the comparative difference value of each CT slice; Step 3.4, comparing the comparison difference value of each CT slice of the CT image with the average difference value of the CT image, and retaining the CT slices in the CT image whose comparison difference value of the CT slices is smaller than the average difference value of the CT image, thereby obtaining coarsely registered CT slices; Step 4: Perform fine registration on the coarsely registered CT slice of the CT image obtained in step 3 and the ultrasound image to obtain a finely registered CT slice, that is; Step 4.1, performing image space transformation on each coarsely registered CT slice using the mean square error of the binary matrices of the CT slice image and the ultrasound image as the objective function, so as to transform each coarsely registered CT slice into the ultrasound space coordinate system, thereby obtaining a coarsely registered CT slice after spatial transformation; Step 4.2: Calculate the overlap ratio between the coarsely registered CT slice after spatial transformation and the ultrasound image using the dice coefficient, and use the coarsely registered CT slice after spatial transformation with the highest overlap ratio as the finely registered CT slice.

2. The feature-based liver registration method for CT images and ultrasound images according to claim 1, characterized in that: The specific process of step 1 is as follows: Step 1.1, adjust the window for each CT slice; Step 1.2, resample each windowed CT slice using bilinear interpolation; Step 1.3: Input each resampled CT slice into the UNet convolutional neural network to obtain a binary mask of the liver organ, thereby extracting the contour features of the liver organ in each CT slice.

3. The feature-based liver registration method for CT images and ultrasound images according to claim 1, wherein: The specific process of step 2 is as follows: Step 2.1, adjusting the window of the ultrasound image; Step 2.2, resampling the windowed ultrasound image using bilinear interpolation; Step 2.3: Use a pixel gradient segmentation method to segment the resampled ultrasound image to obtain a binary mask, i.e., a contour feature, of the liver organ, thereby extracting the contour feature of the liver organ in the ultrasound image.

4. The feature-based liver registration method for CT images and ultrasound images according to claim 3, wherein: The specific process of step 2.3 is as follows: first, use two median filters to denoise the resampled ultrasound image; then use the Canny edge detection algorithm to extract the edge features of the liver organ contour; then binarize the contour edge image and fill the closed area to obtain the binary mask of the liver organ in the ultrasound image.

5. The feature-based liver registration method for CT images and ultrasound images according to claim 1, wherein: In step 4.2, the calculation formula for the overlap rate Dice is: Where, I US is the binary matrix of the ultrasound image, I′ CT is the binary matrix of the coarsely registered CT slice after spatial transformation.

Images