Method for fusing ct images and c-arm x-ray images for spinal surgery navigation

Abstract

The present application belongs to the technical field of medical image processing, in order to solve the technical problem that CT image and C-arm X-ray image cannot be aligned due to the change of patient position, and further enhance the registration accuracy of CT image and C-arm X-ray image, the present application proposes a "CT image and C-arm X-ray image fusion method for spine surgery navigation", which combines Point Pair Feature algorithm to calculate the affine transformation factor required to match the characteristic vertebra to the surface of the C-arm spine region in three-dimensional space, and further optimizes through ICP algorithm. Compared with using two-dimensional C-arm X-ray image, the present application fully utilizes the coronal, sagittal and transverse information of the spine, and uses three-dimensional features to achieve more accurate matching effect, further enhancing the registration accuracy between CT image and C-arm X-ray image.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of medical image processing. BACKGROUND

[0002] Using CT machine or C-arm X-ray machine can shoot three-dimensional images of patients' lesion area, which can be used as the basis of spine surgery navigation, and can be used for preoperative planning and real-time assistance during surgery, so as to ensure the accurate positioning of the lesion position by the doctor and accurate operation.

[0003] C-arm X-ray machine can be used in surgery due to its small size and fast and convenient characteristics compared with CT machine, and can collect three-dimensional images through 360° continuous scanning of C-arm X-ray machine, but the deficiency is that the C-arm X-ray image is not clear enough, and if there is metal in the patient's body, such as metal steel nails for spine correction, metal artifacts will be generated in the image, which will seriously interfere with the doctor's judgment and affect the operation effect; while large CT machine can continuously scan the patient with 1~2mm thin layer to collect clearer three-dimensional images for positioning the patient's lesion area and planning the operation scheme before surgery.

[0004] An article entitled "Comparison of three-dimensional C-arm navigation system and CT navigation system guided lumbar pedicle puncture" published in Chinese Medical Journal (2007, 87(37): 2606~2609) introduces an experiment of using C-arm and CT navigation for lumbar pedicle puncture, and compares and analyzes the influence of the two navigation methods on lumbar pedicle puncture; in order to improve the quality of C-arm image and enhance the effect of intraoperative navigation, the preoperative high-quality CT image can be fused to solve the problem of insufficient C-arm X-ray image during surgery, but due to the inevitable difference between the patient's position before and during surgery, the preoperative CT image cannot be directly fused with the intraoperative C-arm X-ray image, and a certain registration algorithm is needed to align them.

[0005] A Chinese patent application with application number 201310384723.1 entitled "A method for completely corresponding fusion of preoperative CT data and intraoperative X-ray image" discloses a scheme which proposes to reconstruct the preoperative CT image by MC algorithm, and then input the three-dimensional image, and simulate the imaging process of C-arm according to the shape and size data of C-arm X-ray machine, such as the position of the emission source, the position of the receiving surface and the position of the rotation center, so as to achieve the purpose of corresponding fusion of CT image and C-arm image; this patent only simulates the intraoperative C-arm imaging of preoperative reconstructed CT image according to the shape and size data of C-arm X-ray machine, and does not consider the influence of factors such as change of patient's position before and during surgery and respiration on matching.

[0006] A patent application with the title "A chest CT and X-ray real-time registration algorithm based on deep learning" with the application number 202111167612.6 discloses a scheme related to a registration method of three-dimensional CT image and two-dimensional C-arm image, taking the center point of the spine as the matching feature, establishing the feature association information between the C-arm two-dimensional image and the CT image, then calculating the candidate poses required for matching, and finally judging all candidate poses through a robust estimation unit to obtain the final pose. The patent selects the center of the spine as the feature point which is less affected by the patient's breathing, and matches the CT data and C-arm data by combining a certain registration algorithm; but the shortage of the fusion of two-dimensional C-arm image features and three-dimensional CT image is that two-dimensional C-arm collects frontal and lateral images, only two-dimensional images of the coronal and sagittal planes of the spine can be used, and the transverse plane image is missing, so the registration accuracy of the existing technology for three-dimensional CT image and C-arm X-ray image needs to be further improved. SUMMARY

[0007] In order to solve the technical problem that the CT image and the C-arm X-ray image cannot be aligned due to the change of the patient's body position, and further enhance the registration accuracy of the CT image and the C-arm X-ray image, the present application proposes a "CT image and C-arm X-ray image fusion method for spinal surgery navigation".

[0008] The CT image and C-arm X-ray image fusion method for spinal surgery navigation, as shown in Figure 1 , comprises the following steps:

[0009] Step one, segment and extract the spinal vertebra region in the preoperative CT image, and extract the spinal region in the intraoperative C-arm X-ray image;

[0010] Step two, select a feature vertebra for the CT image, and extract feature points on the surface of the feature vertebra or the spinal region: divide the segmentation result of step one into multiple slices in two dimension directions, analyze and filter after contour extraction on each slice, and combine the filtering results in two dimension directions as the feature points on the surface of the feature vertebra or the spinal region;

[0011] Step three, calculate the normal vector of the feature vertebra and the surface feature points of the spinal region, and construct the surface feature point cloud data;

[0012] Step four, calculate the affine transformation relationship between the feature vertebra and the spinal region in three-dimensional space: take the feature vertebra in the CT image as a model, take the spine in the C-arm X-ray image as a scene, calculate the spatial rotation matrix and the spatial translation matrix required for matching the model to the scene, the spatial rotation matrix and the spatial translation matrix affine transformation relationship between the feature vertebra and the spine is established;

[0013] Step five, affine transformation is performed on the CT image according to the affine transformation relationship;

[0014] Step six, the C-arm X-ray image and the affine transformed CT image are fused.

[0015] Technical effects:

[0016] As shown in Figure 2 , Figure 3 Compared with the prior art, the present application has the following advantages:

[0017] 1. Using the selected feature vertebra as the image matching basis, the problem that the CT image and the C-arm X-ray image cannot be aligned due to the change of the patient's body position can be overcome.

[0018] 2. By using the three-dimensional C-arm X-ray image, the feature vertebra and the feature points on the surface of the spine region are used as matching features, the affine transformation factor required for matching the feature vertebra to the surface of the C-arm spine region in three-dimensional space is calculated by using the Point Pair Feature algorithm, and is further optimized by the ICP algorithm. Compared with using the two-dimensional C-arm X-ray image, the present application fully utilizes the coronal, sagittal and transverse information of the spine, and uses three-dimensional features to achieve more accurate matching effect, thereby further enhancing the registration accuracy between the CT image and the C-arm X-ray image.

[0019] 3. By using the three-dimensional CT image of the patient's torso collected by the CT machine and the image of the patient's partial lumbar region collected by the C-arm X-ray machine, the fourth lumbar vertebra is used as the matching feature vertebra of the CT image and the C-arm X-ray image, and the fusion result of the CT image and the C-arm X-ray image by using the technical solution of the present application is shown in Figure 3 When affine transformation is performed on the CT image in step five, the value of the voxel after rotation is estimated by using the adjacent points in the three-dimensional space, the three-dimensional CT image after rotation is smoother and clearer, the three-dimensional CT image after transformation is fused with the C-arm X-ray image by weighting, the purpose of enhancing the C-arm X-ray image during the operation by combining the preoperative CT image is achieved, and the problem of unclear imaging of the C-arm X-ray image is improved, thereby effectively improving the accuracy of the spinal surgery navigation. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is the overall flowchart of the present application.

[0021] Figure 2 is the experimental result diagram of the affine transformation of part of the spinal image of the present application.

[0022] Figure 3It is a CT image and C-arm X-ray image weighted fusion matching result schematic diagram of the present application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. All other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.

[0024] In the step one, the spinal vertebra region segmentation in the CT image is realized by a pre-trained three-dimensional semantic segmentation neural network model 3D VGG: the preoperative CT image is represented as , and the three-dimensional size is recorded as , wherein represents the number of plain scan images of the CT image, and represent the height and width of each plain scan image, the spinal vertebra semantic segmentation of the CT image is obtained by the pre-trained 3D VGG model, the three-dimensional size of each vertebra segmentation result is consistent with the size of the CT image, and the non-zero voxel corresponds to the vertebra region.

[0025] The spinal region extraction in the C-arm X-ray image is realized by a pre-trained three-dimensional segmentation neural network model nnUnet: the intraoperative C-arm X-ray image is represented as , and the three-dimensional size is recorded as , wherein represents the number of plain scan images of the C-arm X-ray image, and represent the height and width of each plain scan image, the spinal segmentation of the C-arm X-ray image is obtained by the pre-trained nnUnet model, the three-dimensional spinal segmentation result is consistent with the C-arm X-ray image, and the non-zero voxel corresponds to the spinal region.

[0026] In the step two, the extraction method of the feature vertebra surface feature point of the CT image is consistent with the extraction method of the spinal region surface feature point in the C-arm X-ray image. The following takes the extraction of the feature vertebra surface feature point of the CT image as an example:

[0027] Firstly, in the first dimension direction, the three-dimensional feature vertebra segmentation result is split into a two-dimensional image set, that is , and each two-dimensional image has a size of , and each two-dimensional image is a layer of slice of the feature vertebra in the first dimension corresponding direction, wherein the non-zero pixel represents the pixel constituting the feature vertebra.

[0028] Secondly, the contour of the feature vertebra region is detected on each of the two-dimensional images obtained by splitting, and since the density of the feature vertebra is uneven and the vertebra is hollow, multiple contours are detected, so the contour needs to be filtered by analyzing the coincidence between different contours; the position information of the feature points constituting the contour of the feature vertebra region is represented as , wherein , represents the horizontal coordinate and vertical coordinate information of each feature point, respectively, represents the number of feature points, and the sequence information of the two-dimensional image in the first dimension is , the surface feature points of the vertebra are represented as ;

[0029] Thirdly, the three-dimensional feature vertebra segmentation result is split into two-dimensional images with a size of in the second dimension direction, and the contour of the feature vertebra region is detected on each of the two-dimensional images obtained by splitting and the redundant contour is filtered, the position information of the feature points constituting the contour of the feature vertebra region is represented as , wherein , represents the horizontal coordinate and vertical coordinate information of each feature point, respectively, represents the number of feature points, and the sequence information of the two-dimensional image in the second dimension is , the surface feature points of the vertebra are represented as ;

[0030] Finally, the feature point detection results in the two dimension directions are merged to constitute the surface feature points of the vertebra.

[0031] The spine region in the C-arm X-ray image in step one is extracted and subjected to contour filtering processing in the same way, and the extraction of the surface feature points of the spine region is completed, which will not be described here.

[0032] In the step three, the method for calculating the normal vector of the surface feature points of the feature vertebra and the spine region is as follows: the range radius in the three-dimensional space and the maximum number of selected points are set, the KDTree algorithm is used to calculate the adjacent points, and then the normal vector of each surface feature point is calculated, denoted as , and all the normal vector information and surface feature point information are combined to constitute the point cloud data of the surface points of the feature vertebra and the spine region. Preferably, the range radius in the three-dimensional space is set to 50, and the maximum number of selected points is set to 30.

[0033] In the fourth step, the affine transformation relationship is calculated in the following way: combining the feature vertebra and the surface point cloud data of the spine region obtained in the third step, taking the feature vertebra in the CT image as a model and taking the spine in the C-arm X-ray image as a scene, the spatial rotation matrix and the spatial translation matrix required for matching the model to the scene are calculated by using the Point Pair Feature algorithm and the ICP algorithm. and the spatial translation matrix , the spatial rotation matrix and the spatial translation matrix constitute the affine transformation relationship of matching the feature vertebra to the spine.

[0034] In the fifth step, the affine transformation process of the CT image is as follows:

[0035] The affine transformation of the CT image needs to be performed on each voxel constituting the three-dimensional image. The new position information after the transformation may not be an integer, i.e., it cannot be mapped to the three-dimensional matrix position. In order to make the matching result directly available for the medical scene, the result needs to be saved in the form of a three-dimensional matrix in a file format readable by application programs such as 3D Slicer and ITK SNAP. In order to map the affine transformation result to the three-dimensional matrix position, the process is as follows:

[0036] According to the CT image size and the spatial rotation matrix and the spatial translation matrix calculated in the fourth step, the three-dimensional matrix size occupied by the CT image after the affine transformation is calculated. ,

[0037] , ,

[0038] ,

[0039] wherein, represents the vertex position information matrix of the original CT image, represents the number of plain images of the original CT image, and represent the height and width of each plain image, represents the vertex position information matrix of the CT image after the affine transformation, represents the number of plain images of the CT image after the affine transformation, and represent the height and width of each plain image;

[0040] The three-dimensional matrix after the affine transformation is denoted as for Each voxel in the position, according to the inverse matrix of the rotation matrix and the translation matrix, get the corresponding position information in the original CT three-dimensional image, expressed as:

[0041]

[0042] wherein, represents the corresponding position information of the voxel in the original CT image, , the specific value of the position is ; represents the position information of the voxel in ; judge whether is in the original CT image, that is, ,

[0043] If is not in the original CT image, the value is 0;

[0044] If is in the original CT image, take the adjacent four points in each dimension direction, that is,

[0045] , wherein represents the coordinate in a certain dimension, and a total of 64 adjacent points are taken in three dimension directions, and the adjacent points are expressed as , which is used for weighting to determine the specific value of the position of , and the value of in the original CT image is taken as the value of the position in ,

[0046] ,

[0047] ,

[0048] ,

[0049] wherein, the value of the adjacent point is , is a weight parameter, preferably, the weight parameter takes the value of-0.5.

[0050] The above affine transformation operation is performed on a part of the CT image, and the transformed one layer of the sagittal plane scanning image of the spine and the visualization result by 3D Slicer are shown in Figure 2 .

[0051] ​​In the step six, the C-arm X-ray image and the CT image after affine transformation are fused by weighted fusion:

[0052] By the affine transformation of the original CT three-dimensional data in step five, the CT image and the C-arm X-ray image are aligned and located in the same coordinate system. In order to correspond the alignment result to the three-dimensional matrix result, it is further saved in the form of a target file, the process is as follows:

[0053] Firstly, the minimum and maximum values of the coordinate information of the CT image after affine transformation and the coordinate information of the C-arm X-ray image are compared in three dimensions respectively, and the size of the three-dimensional matrix of the fused image is determined , which is expressed as:

[0054]

[0055]

[0056]

[0057]

[0058]

[0059]

[0060]

[0061]

[0062]

[0063] wherein, , and respectively represent the minimum coordinates of the coordinate system in which the CT image and the C-arm X-ray image are located in three dimensions, , and respectively represent the maximum coordinates;

[0064] Secondly, the size of the CT image matrix after affine transformation and the C-arm X-ray image matrix is unified:

[0065] 1. Adjust the size of the CT image matrix after affine transformation: calculate the start and end positions of the CT image in the three-dimensional matrix after affine transformation, and then embed the transformed CT image into the three-dimensional matrix. The starting position of the CT image in the three-dimensional matrix is the distance of the CT image in each dimension direction from the minimum coordinate in the coordinate system, and the end position corresponds to the start and end positions in each dimension direction plus the length in this dimension direction. The process is expressed as:

[0066]

[0067]

[0068]

[0069]

[0070]

[0071]

[0072]

[0073] wherein, represents the result of adjusting the size of the CT image matrix after affine transformation;

[0074] Secondly, the size of the C-arm X-ray image matrix is adjusted in the same way as the size of the CT image matrix after affine transformation, and the process is represented as:

[0075]

[0076]

[0077]

[0078]

[0079]

[0080]

[0081]

[0082] wherein, represents the result of adjusting the size of the C-arm X-ray image matrix after affine transformation;

[0083] Finally, the final fusion result is obtained by weighting the CT image matrix and the C-arm X-ray image matrix after the shape of the matrix is unified, and is represented as:

[0084]

[0085] wherein represents the fusion result, represents the weighting factor. Preferably, the value is 0.5.

[0086] The preferred embodiments of the present application are described in detail above with reference to the drawings, but the present application is not limited to the specific details of the above-described embodiments, and various simple modifications can be made to the technical solutions of the present application within the technical concept of the present application, and these simple modifications all belong to the protection scope of the present application.

Claims

1. A method for fusing CT images and C-arm X-ray images for spinal surgery navigation, characterized in that, The method comprises the following steps: Step one, segment the spine vertebra region in the preoperative CT image, and extract the spine region in the intraoperative C-arm X-ray image; Step two, select a feature vertebra for the CT image, and extract feature points on the surface of the feature vertebra or the spine region: divide the segmentation result of step one into multiple slices in two dimension directions, perform contour extraction, analysis and filtering on each slice, and combine the filtering results in two dimension directions as the feature points on the surface of the feature vertebra or the spine region; Step three, calculate the normal vector of the feature vertebra and the surface feature points of the spine region, and construct the surface feature point cloud data; Step four, calculating the affine transformation relationship between the feature vertebra and the spine region in three-dimensional space: taking the feature vertebra in the CT image as a model and taking the spine in the C-arm X-ray image as a scene, calculating the spatial rotation matrix required for matching the model to the scene and the spatial translation matrix , the spatial rotation matrix and the spatial translation matrix constitute the affine transformation relationship of matching the feature vertebra to the spine; Step five, affine transform the CT image according to the affine transformation relationship; Step six, weighted fusion of the C-arm X-ray image and the affine transformed CT image; The affine transformation of the CT image in step five is specifically: According to the CT image size and the spatial rotation matrix calculated in step four and the spatial translation matrix , calculate the size of the three-dimensional matrix occupied by the CT image after affine transformation , ， ， ， wherein, represents a matrix of vertex position information of the original CT image, represents the number of plain scan images of the original CT image, and represents the height and width of each plain scan image, represents a matrix of vertex position information of the CT image after affine transformation, represents the number of plain scan images of the CT image after affine transformation, and represents the height and width of each plain scan image. The three-dimensional matrix after the affine transformation is denoted as For each voxel in the matrix, the inverse of the rotation matrix and the translation matrix are used to obtain the corresponding position information in the original CT three-dimensional image, which is denoted as: For each voxel in the matrix, the inverse of the rotation matrix and the translation matrix are used to obtain the corresponding position information in the original CT three-dimensional image, which is denoted as: ， in, This indicates the location information of the voxel in the original CT image. , The specific value at the location ； Indicates in Midvoxel location information, ;judge Whether it is within the original CT image, i.e. , If not in the original CT image, is 0; If In the original CT image, four adjacent points are taken in each dimension direction, namely wherein represents the coordinate in a certain dimension, and there are 64 neighboring points in total in three dimensions, and the neighboring points are represented as , which is used for weighting to determine the specific value of the location of , and the value of in the original CT image is taken as the value of the location of , and , and , ， ， wherein the neighboring points take values , are weight parameters.

2. The CT image and C-arm X-ray image fusion method for spinal surgery navigation according to claim 1, characterized in that: In step one, the segmentation of the spine vertebra region in the CT image is realized by using a pre-trained three-dimensional semantic segmentation neural network model 3D VGG, wherein the non-zero voxels correspond to the spine vertebra region; the extraction of the spine region in the C-arm X-ray image is realized by using a pre-trained three-dimensional segmentation neural network model nnUnet, wherein the non-zero voxels correspond to the spine region.

3. The CT image and C-arm X-ray image fusion method for spinal surgery navigation as claimed in claim 1, wherein: The feature vertebra surface feature point extraction in step two is specifically as follows: the CT image is represented as , and the three-dimensional size is recorded as , wherein represents the number of plain scan images of the CT image, and represent the height and width of each plain scan image. Firstly, in the first dimension direction, the three-dimensional feature vertebra segmentation result is split into a two-dimensional image set, that is two-dimensional images with a size of , and each two-dimensional image is a slice of the feature vertebra in the first dimension direction, wherein the non-zero value pixel represents the pixel constituting the feature vertebra. Secondly, the contour of the feature vertebra region is detected on each of the two-dimensional images obtained by splitting, and the contour is filtered; the position information of the feature points constituting the contour of the feature vertebra region is expressed as , wherein , respectively represent the horizontal coordinate and vertical coordinate information of each feature point, represents the number of feature points, and the order information of the two-dimensional images in the first dimension is combined as , and the feature points on the surface of the vertebra are expressed as ; Third, in the second dimension direction, the three-dimensional feature vertebra body segmentation result is split into two-dimensional images, the contour of the feature vertebra body region is detected on each of the split obtained two-dimensional images, and the redundant contour is filtered, the feature point position information of the contour of the feature vertebra body region is expressed as , wherein , respectively represent the horizontal coordinate and vertical coordinate information of each feature point, the number of feature points, and the order information of the two-dimensional images in the second dimension direction is expressed as , and the vertebra body surface feature point is expressed as ; Finally, the feature point detection results in two dimension directions are combined to form the surface feature points of the vertebra.

4. The CT image and C-arm X-ray image fusion method for spinal surgery navigation as claimed in claim 1, wherein: In step three, the feature vertebra and spine region surface feature point normal vector calculation method is specifically as follows: setting the range radius in the three-dimensional space as 50, the maximum number of selected points as 30, using the KDTree algorithm to calculate the adjacent points, and then calculating the normal vector of each surface feature point, denoted as Combining all the normal vector information and surface feature point information to form the feature vertebra and spine region surface point cloud data.

5. The CT image and C-arm X-ray image fusion method for spinal surgery navigation as claimed in claim 1, wherein: In step six, the weighted fusion of the C-arm X-ray image and the affine transformed CT image is specifically: compare the minimum and maximum values of the coordinate information of the affine transformed CT image and the coordinate information of the C-arm X-ray image in three dimensions respectively, to determine the size of the fusion image three-dimensional matrix; secondly, unify the size of the affine transformed CT image matrix and the C-arm X-ray image matrix; finally, weighted unify the CT image matrix and the C-arm X-ray image matrix after the shape is unified to obtain the final fusion result, which is represented as: ， wherein represents the fusion result, represents the weighting factor, represents the result of adjusting the matrix size of the CT image after the affine transformation, represents the result of adjusting the matrix size of the C-arm X-ray image after the affine transformation.

Images