CT image and C-shaped arm X-ray image fusion method for spinal surgical navigation

Abstract

The invention belongs to the technical field of medical image processing, and provides a CT image and C-shaped arm X-ray image fusion method for spinal surgical navigation in order to solve the technical problem that a CT image and a C-shaped arm X-ray image cannot be aligned due to the change of the body position of a patient and further enhance the registration accuracy of the CT image and the C-shaped arm X-ray image. Affine transformation factors required for matching the feature vertebral body to the surface of the spine area of the C-shaped arm in a three-dimensional space are calculated in combination with a Point Pair Feature algorithm, further optimization is performed through an ICP algorithm, compared with a two-dimensional C-shaped arm X-ray image, information of the coronal plane, the sagittal plane and the cross section of the spine is fully utilized, a more accurate matching effect can be achieved by using three-dimensional features, and the matching efficiency is improved. And the registration accuracy between the CT image and the C-shaped arm X-ray image is further enhanced.

Description

Technical Field

[0001] The invention belongs to the technical field of medical image processing. Background Art

[0002] Using a CT machine or a C-arm X-ray machine, three-dimensional images of the patient's lesion area can be taken as the basis for spinal surgery navigation. These three-dimensional images can be used for pre-operative planning and real-time assistance during surgery, thereby ensuring that the doctor accurately locates the lesion and performs accurate surgical operations.

[0003] C-arm X-ray machines can be used during surgery due to their small size, speed and convenience compared to CT machines. They can collect three-dimensional images through 360° continuous scanning. However, the C-arm X-ray images are not clear enough. If there are metal objects in the patient's body, such as metal nails used for spinal correction, metal artifacts will be generated in the image, which will seriously interfere with the doctor's judgment and affect the effect of the operation. Large CT machines can continuously scan patients with thin layers of 1~2mm and collect clearer three-dimensional images, which can be used to locate the patient's lesion area and plan the surgical plan before surgery.

[0004] An article entitled "Comparison of lumbar pedicle puncture guided by three-dimensional C-arm navigation system and CT navigation system" was published in Chinese Medical Journal (2007, 87(37): 2606-2609). The article introduced the experiments of lumbar pedicle puncture using C-arm and CT navigation respectively, and compared and analyzed the effects of the two navigation methods on the results of lumbar pedicle puncture. In order to improve the quality of C-arm images and enhance the effect of intraoperative surgical navigation, the shortcomings of intraoperative C-arm X-ray images can be solved by fusing preoperative high-quality CT images. However, due to the inevitable differences in the patient's body position before and during surgery, the preoperative CT images cannot be directly fused with the intraoperative C-arm X-ray images, and a certain registration algorithm is needed to align the two.

[0005] A Chinese patent application with application number 201310384723.1 and titled “A method for completely corresponding fusion of preoperative CT data and intraoperative X-ray images” discloses a scheme which proposes to reconstruct the preoperative CT image through the MC algorithm, and then use the three-dimensional image as input to simulate the imaging process of the C-arm according to the external dimension data of the C-arm X-ray machine, such as the position of the emission source, the position of the receiving surface, and the position of the rotation center, so as to achieve the purpose of corresponding fusion of the CT image and the C-arm image. The patent only simulates the intraoperative C-arm imaging with the preoperative reconstructed CT image according to the external dimension data of the C-arm X-ray machine, and does not consider the influence of factors such as the patient's body position changes and breathing before and during the operation on the matching.

[0006] A Chinese patent application with application number 202111167612.6 and titled "A real-time registration algorithm for chest CT and X-ray based on deep learning" discloses a scheme involving a registration method for three-dimensional CT images and two-dimensional C-arm images, which uses the center point of the spine as a matching feature, establishes feature association information between the C-arm two-dimensional image and the CT image, and then calculates the candidate poses required for matching. Finally, a robust estimation unit is used to judge all candidate poses and aggregate them to obtain the final pose. The patent selects the center of the spine that is less affected by the patient's breathing as a feature point, and combines a certain registration algorithm to match the CT data with the C-arm data; however, the deficiency of fusing the two-dimensional C-arm image features with the three-dimensional CT image is that the two-dimensional C-arm collects the frontal and lateral images, and can only use the two-dimensional images of the coronal and sagittal planes of the spine, and lacks the cross-sectional image. Therefore, the registration accuracy of the existing technology for three-dimensional CT images and C-arm X-ray images needs to be further improved. Summary of the invention

[0007] In order to solve the technical problem that CT images and C-arm X-ray images cannot be aligned due to changes in patient position, and to further enhance the registration accuracy of CT images and C-arm X-ray images, the present invention proposes a "CT image and C-arm X-ray image fusion method for spinal surgery navigation".

[0008] CT image and C-arm X-ray image fusion method for spinal surgery navigation, such as Figure 1 As shown, the following steps are included:

[0009] Step 1: segment and extract the spinal vertebral area in the preoperative CT image, and extract the spinal area in the intraoperative C-arm X-ray image;

[0010] Step 2: Select a characteristic vertebra for the CT image and extract characteristic points of the characteristic vertebra or the surface of the spinal region: divide the segmentation result of step 1 into multiple slices in two dimensional directions, perform contour extraction and post-analysis and filtering on each slice, and merge the filtering results in two dimensional directions as characteristic points of the characteristic vertebra or the surface of the spinal region;

[0011] Step 3: Calculate the normal vectors of the surface feature points of the characteristic vertebra and spine area, and construct surface feature point cloud data;

[0012] Step 4: Calculate the affine transformation relationship between the characteristic vertebra and the spine region in three-dimensional space: Use the characteristic vertebra in the CT image as the model and the spine in the C-arm X-ray image as the scene to calculate the spatial rotation matrix required to match the model to the scene. and the spatial translation matrix , the space rotation matrix and the spatial translation matrix Constructing an affine transformation relationship for matching the feature vertebra to the spine;

[0013] Step 5, performing affine transformation on the CT image according to the affine transformation relationship;

[0014] Step six, weighted fusion of the C-arm X-ray image and the affine transformed CT image.

[0015] Technical effect:

[0016] like Figure 2 , Figure 3 As shown, compared with the prior art, the present invention:

[0017] 1. Using the selected characteristic vertebra as the basis for image matching can overcome the problem of misalignment between CT images and C-arm X-ray images caused by changes in patient position.

[0018] 2. By using three-dimensional C-arm X-ray images, characteristic points of characteristic vertebrae and the surface of the spinal region are used as matching features, and the Point Pair Feature algorithm is combined to calculate the affine transformation factors required to match the characteristic vertebrae to the surface of the C-arm spinal region in three-dimensional space, and further optimized by the ICP algorithm. Compared with the use of two-dimensional C-arm X-ray images, the present invention makes full use of the coronal, sagittal and cross-sectional information of the spine. The use of three-dimensional features can achieve more accurate matching effects, further enhancing the registration accuracy between CT images and C-arm X-ray images.

[0019] 3. A 3D CT image of the patient's torso is collected by a CT machine, and a C-arm X-ray machine takes an image of part of the patient's lumbar region. The fourth lumbar vertebra is used as a matching feature vertebra for the CT image and the C-arm X-ray image. The result of fusing the CT image and the C-arm X-ray image using the technical solution of the present invention is as follows: Figure 3 As shown; in step five, when performing affine transformation on the CT image, the value of the rotated voxel is estimated by the neighboring points in the three-dimensional space. The rotated three-dimensional CT image is smoother and clearer. The transformed three-dimensional CT image and the C-arm X-ray image are weighted fused to achieve the purpose of enhancing the intraoperative C-arm X-ray image by combining the preoperative CT image, and can improve the problem of unclear imaging of the C-arm X-ray image, thereby effectively improving the accuracy of spinal surgery navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is the overall flow chart of the present invention.

[0021] Figure 2 It is a schematic diagram of the experimental results of the present invention of performing affine transformation on a part of the spinal image.

[0022] Figure 3It is a schematic diagram of the weighted fusion matching result of the CT image and the C-arm X-ray image of the present invention. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field without making creative work by adopting the embodiments of the present invention are within the scope of protection of the present invention.

[0024] In the step 1, the spinal vertebral region segmentation in the CT image is achieved by using a pre-trained 3D semantic segmentation neural network model 3D VGG: the preoperative CT image is represented as , and its three-dimensional dimensions are recorded as ,in Indicates the number of plain scan images of CT images, and Represents the height and width of each plain scan image. The CT image is semantically segmented by the spinal vertebrae through the pre-trained 3D VGG model to obtain the vertebral segmentation results that constitute the spine. The three-dimensional size of each vertebral segmentation result is consistent with the size of the CT image, and the non-zero voxel corresponds to the vertebral area.

[0025] The extraction of the spine region in the C-arm X-ray image is achieved by using the pre-trained 3D segmentation neural network model nnUnet: the C-arm X-ray image collected during surgery is represented as , and its three-dimensional dimensions are recorded as ,in Indicates the number of plain scan images of the C-arm X-ray image. and Represents the height and width of each plain scan image. The C-arm X-ray image is segmented by the pre-trained nnUnet model to obtain the 3D spine segmentation result. The data shape of the spine segmentation result is consistent with the C-arm X-ray image, where the non-zero voxels correspond to the spine area.

[0026] In the step 2, the method for extracting the characteristic vertebral surface feature points of the CT image is consistent with the method for extracting the surface feature points of the spine region in the C-arm X-ray image. The following takes the extraction of characteristic vertebral surface feature points of the CT image as an example:

[0027] First, in the first dimension, the three-dimensional feature vertebra segmentation result is split into a two-dimensional image set, that is, The size is Two-dimensional images, each of which is a slice of the characteristic cone in the direction corresponding to the first dimension, wherein pixels with non-zero values ​​represent pixels constituting the characteristic cone;

[0028] Secondly, the contour of the characteristic vertebral area is detected on each split 2D image. Due to the uneven density of the characteristic vertebral body and the hollow vertebral body, multiple contours will be detected, so it is necessary to filter the contours by analyzing the overlap between different contours. The position information of the feature points constituting the contour of the characteristic vertebral area is expressed as ,in , Respectively represent the horizontal and vertical coordinate information of each feature point, represents the number of feature points, combined with the order information of the two-dimensional image in the first dimension: , the feature points on the vertebral surface are expressed as ;

[0029] Third, in the second dimension, the three-dimensional feature vertebral segmentation result is split into The size is Similarly, the contour of the characteristic vertebral area is detected on each split two-dimensional image and redundant contours are filtered out. The position information of the feature points constituting the contour of the characteristic vertebral area is expressed as ,in , Respectively represent the horizontal and vertical coordinate information of each feature point, Represents the number of feature points, combined with the order information of the two-dimensional image in the second dimension, it is expressed as , the feature points on the vertebral surface are expressed as ;

[0030] Finally, the feature point detection results in two dimensional directions are merged to form the vertebral surface feature points.

[0031] The spine region in the C-arm X-ray image of step 1 is extracted and contour filtering is performed in the same manner as above to complete the extraction of surface feature points of the spine region, which will not be repeated here.

[0032] In step 3, the method for calculating the normal vector of the surface feature points of the characteristic vertebra and the spine region is as follows: setting the range radius and the maximum number of selected points in the three-dimensional space, using the KDTree algorithm to calculate the neighboring points, and then calculating the normal vector of each surface feature point, recorded as , all normal vector information and surface feature point information are combined to form feature vertebral and spinal region surface point cloud data. Preferably, the range radius in the three-dimensional space is set to 50, and the maximum number of selected points is set to 30.

[0033] In step 4, the calculation method of the affine transformation relationship is specifically as follows: combining the characteristic vertebrae and the surface point cloud data of the spinal region obtained in step 3, taking the characteristic vertebrae in the CT image as the model, and taking the spine in the C-arm X-ray image as the scene, and calculating the spatial rotation matrix required to match the model to the scene through the Point Pair Feature algorithm and the ICP algorithm and the spatial translation matrix , the space rotation matrix and the spatial translation matrix Construct an affine transformation relationship that matches the feature vertebra to the spine.

[0034] In step 5, the affine transformation process of the CT image is specifically as follows:

[0035] The affine transformation of CT images requires affine transformation of each voxel that constitutes the three-dimensional image. The new position information after the transformation may not be an integer, that is, it cannot correspond to the three-dimensional matrix position. In order to make the matching results directly available for medical scenarios, the results need to be saved in the form of a three-dimensional matrix in a file format that can be read by applications such as 3D Slicer and ITK SNAP. In order to correspond the affine transformation results to the three-dimensional matrix position, the process is:

[0036] According to the CT image size and the spatial rotation matrix calculated in step 4 and the spatial translation matrix , calculate the size of the three-dimensional matrix occupied by the CT image after affine transformation ,

[0037] , ,

[0038] ,

[0039] in, Represents the vertex position information matrix of the original CT image, Indicates the number of plain scan images of the original CT image, and Indicates the height and width of each flat scan image. represents the vertex position information matrix of the CT image after affine transformation, represents the number of plain scan images of the CT image after affine transformation, and Indicates the height and width of each flat scan image;

[0040] The three-dimensional matrix after affine transformation is recorded as ,for For each voxel in , the inverse transformation is performed according to the inverse matrix of the rotation matrix and the translation matrix to obtain the position information corresponding to this position in the original CT three-dimensional image, which is expressed as:

[0041]

[0042] in, Indicates the position information of the voxel in the original CT image. , The specific value at the location is ; Indicated in voxel position information, ;judge Whether it is within the original CT image, that is, ,

[0043] like Not in the original CT image, The value is 0;

[0044] like In the original CT image, four adjacent points are taken in each dimension, namely

[0045] ,in Represents the coordinates in a certain dimension. A total of 64 neighboring points are taken in the three dimensional directions. The neighboring points are represented as , which is used to weight and determine The specific value of the location and the original CT image The value of As Middle position The value of

[0046] ,

[0047] ,

[0048] ,

[0049] Among them, the neighboring point value is , is a weight parameter, preferably, a weight parameter The value is -0.5.

[0050] The above affine transformation operation is performed on a part of the CT images. The transformed plain scan image of the spine in the sagittal plane and the visualization result through 3D Slicer are shown as follows: Figure 2 shown.

[0051] In step six, the C-arm X-ray image and the affine transformed CT image are weighted fused:

[0052] Through the affine transformation of the original CT three-dimensional data in step five, the CT image and the C-arm X-ray image have been aligned and located in the same coordinate system. In order to correspond the alignment result to the three-dimensional matrix result and further save it as a target file, the process is as follows:

[0053] First, the minimum and maximum values ​​of the affine transformed CT image coordinate information and the C-arm X-ray image coordinate information are compared in three dimensions to determine the size of the fused image three-dimensional matrix. , expressed as:

[0054]

[0055]

[0056]

[0057]

[0058]

[0059]

[0060]

[0061]

[0062]

[0063] in, , and They represent the minimum coordinates of the coordinate systems of the CT image and the C-arm X-ray image in three dimensions. , and Respectively represent the maximum coordinates;

[0064] Secondly, unify the size of the CT image matrix and the C-arm X-ray image matrix after affine transformation:

[0065] First, adjust the size of the CT image matrix after affine transformation: calculate the starting and ending positions of the CT image after affine transformation in the three-dimensional matrix, and then embed the transformed CT image into the three-dimensional matrix. The starting position of the CT image in the three-dimensional matrix is ​​the distance of the CT image from the minimum coordinate in the coordinate system in each dimension, and the ending position corresponds to the starting and ending positions in each dimension plus the length in this dimension. The process is expressed as:

[0066]

[0067]

[0068]

[0069]

[0070]

[0071]

[0072]

[0073] in, It represents the result of adjusting the size of the CT image matrix after affine transformation;

[0074] Second, the size of the C-arm X-ray image matrix is ​​adjusted in the same way as the size of the CT image matrix after affine transformation. The process is expressed as:

[0075]

[0076]

[0077]

[0078]

[0079]

[0080]

[0081]

[0082] in, It represents the result of adjusting the size of the C-arm X-ray image matrix after affine transformation;

[0083] Finally, the CT image matrix and the C-arm X-ray image matrix after weighted unified matrix shape obtain the final fusion result, which is expressed as:

[0084]

[0085] in represents the fusion result, represents a weighting factor. Preferably, The value is 0.5.

[0086] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, a variety of simple modifications can be made to the technical solution of the present invention, and these simple modifications all belong to the protection scope of the present invention.

Claims

1. A method for fusing CT images and C-arm X-ray images for spinal surgery navigation, characterized in that: The following steps are involved: Step 1: segment and extract the spinal vertebral area in the preoperative CT image, and extract the spinal area in the intraoperative C-arm X-ray image; Step 2: Select a characteristic vertebra for the CT image and extract characteristic points of the characteristic vertebra or the surface of the spinal region: divide the segmentation result of step 1 into multiple slices in two dimensional directions, perform contour extraction and post-analysis and filtering on each slice, and merge the filtering results in the two dimensional directions as characteristic points of the characteristic vertebra or the surface of the spinal region; Step 3: Calculate the normal vectors of the surface feature points of the characteristic vertebra and spine area, and construct surface feature point cloud data; Step 4: Calculate the affine transformation relationship between the characteristic vertebra and the spine region in three-dimensional space: Use the characteristic vertebra in the CT image as the model and the spine in the C-arm X-ray image as the scene to calculate the spatial rotation matrix required to match the model to the scene. and the spatial translation matrix , the space rotation matrix and the spatial translation matrix Constructing an affine transformation relationship for matching the feature vertebra to the spine; Step 5, performing affine transformation on the CT image according to the affine transformation relationship; Step six, weighted fusion of the C-arm X-ray image and the affine transformed CT image.

2. The method for fusing CT images and C-arm X-ray images for spinal surgery navigation according to claim 1, characterized in that: In step 1, the segmentation of the spinal vertebral region in the CT image is achieved through the pre-trained three-dimensional semantic segmentation neural network model 3D VGG, in which the voxels with non-zero values ​​correspond to the spinal vertebral region; the extraction of the spinal region in the C-arm X-ray image is achieved through the pre-trained three-dimensional segmentation neural network model nnUnet, in which the voxels with non-zero values ​​correspond to the spinal region.

3. The method for fusing CT images and C-arm X-ray images for spinal surgery navigation according to claim 1, characterized in that: The specific extraction of characteristic vertebral surface feature points of the CT image in step 2 is as follows: CT images are represented as , and its three-dimensional dimensions are recorded as ,in Indicates the number of plain scan images of CT images, and Represents the height and width of each plain scan image. First, in the first dimension, the three-dimensional feature vertebral segmentation result is split into a two-dimensional image set, that is, The size is Two-dimensional images, each of which is a slice of the characteristic cone in the direction corresponding to the first dimension, wherein pixels with non-zero values ​​represent pixels constituting the characteristic cone; Secondly, the contour of the characteristic vertebral area is detected on each split two-dimensional image and the contour is filtered; the position information of the characteristic points constituting the contour of the characteristic vertebral area is expressed as ,in , Respectively represent the horizontal and vertical coordinate information of each feature point, represents the number of feature points, combined with the order information of the two-dimensional image in the first dimension: , the feature points on the vertebral surface are expressed as ; Third, in the second dimension, the three-dimensional feature vertebral segmentation result is split into The size is The contour of the characteristic vertebral area is detected on each split two-dimensional image and redundant contours are filtered. The position information of the characteristic points constituting the contour of the characteristic vertebral area is expressed as ,in , Respectively represent the horizontal and vertical coordinate information of each feature point, Represents the number of feature points, combined with the order information of the two-dimensional image in the second dimension, it is expressed as , the feature points on the vertebral surface are expressed as ; Finally, the feature point detection results in two dimensional directions are merged to form the vertebral surface feature points.

4. The method for fusing CT images and C-arm X-ray images for spinal surgery navigation according to claim 1, characterized in that: In step 3, the method for calculating the normal vectors of the surface feature points of the characteristic vertebra and spine region is as follows: set the range radius in the three-dimensional space to 50, select a maximum of 30 points, use the KDTree algorithm to calculate the neighboring points, and then calculate the normal vector of each surface feature point, recorded as , all normal vector information and surface feature point information are combined to form the surface point cloud data of the characteristic vertebra and spine area.

5. The method for fusing CT images and C-arm X-ray images for spinal surgery navigation according to claim 1, characterized in that: In step 5, the affine transformation of the CT image is specifically as follows: According to the CT image size and the spatial rotation matrix calculated in step 4 and the spatial translation matrix , calculate the size of the three-dimensional matrix occupied by the CT image after affine transformation , ， ， ， in, Represents the vertex position information matrix of the original CT image, Indicates the number of plain scan images of the original CT image, and Indicates the height and width of each flat scan image. represents the vertex position information matrix of the CT image after affine transformation, represents the number of plain scan images of the CT image after affine transformation, and Indicates the height and width of each flat scan image; The three-dimensional matrix after affine transformation is recorded as ,for For each voxel in , the inverse transformation is performed according to the inverse matrix of the rotation matrix and the translation matrix to obtain the position information corresponding to this position in the original CT three-dimensional image, which is expressed as: ,in, Indicates the position information of the voxel in the original CT image. , The specific value at the location is ; Indicated in voxel position information, ;judge Whether it is within the original CT image, that is, , like Not in the original CT image, The value is 0; like In the original CT image, four adjacent points are taken in each dimension, namely ,in Represents the coordinates in a certain dimension. A total of 64 neighboring points are taken in the three dimensional directions. The neighboring points are represented as , which is used to weight and determine The specific value of the location and the original CT image The value of As Middle position The value of ， ， , where the neighboring point value is , is the weight parameter.

6. The method for fusing CT images and C-arm X-ray images for spinal surgery navigation according to claim 1, characterized in that: In step 6, the weighted fusion of the C-arm X-ray image and the affine transformed CT image is specifically as follows: the minimum and maximum values ​​of the affine transformed CT image coordinate information and the C-arm X-ray image coordinate information are compared in three dimensions to determine the size of the fused image three-dimensional matrix; secondly, the size of the affine transformed CT image matrix and the C-arm X-ray image matrix is ​​unified; finally, the weighted unified matrix shape of the CT image matrix and the C-arm X-ray image matrix is ​​used to obtain the final fusion result, which is expressed as: ， in represents the fusion result, represents the weighting factor, Represents the result of adjusting the size of the CT image matrix after affine transformation. Represents the result of adjusting the size of the C-arm X-ray image matrix after affine transformation.

Images