Coordinate transformation registration method and system based on CT image and X-ray image

Abstract

The invention discloses a coordinate transformation registration method and system based on a CT image and an X-ray image, and the method comprises the steps: carrying out the rendering processing based on preoperative CT data, and obtaining an X-ray image with a calibration target mark; feature preprocessing is carried out based on the X-ray image with the calibration target mark, two-dimensional X-ray image coordinate features are obtained, and a correction matrix for eliminating X-ray image distortion is obtained; and obtaining a transformation matrix of a two-dimensional X-ray feature image coordinate system and a real three-dimensional space coordinate system based on the two-dimensional X-ray image coordinate features, and completing coordinate transformation registration of a three-dimensional real space and a CT three-dimensional space in combination with a registration method of the two-dimensional X-ray image and the CT image and the correction matrix. According to the invention, the harm of a doctor exposed in a radiation environment can be reduced, the conversion of image space coordinates is simplified, and the registration precision of a CT image and an X-ray image is further improved. The coordinate transformation registration method and system based on the CT image and the X-ray image can be widely applied to the technical field of image registration.

Description

Technical Field

[0001] The present invention relates to the technical field of image registration, and in particular to a coordinate transformation registration method and system based on CT images and X-ray images. Background Art

[0002] At present, surgical navigation systems are basically divided into the following categories: navigation systems based on preoperative three-dimensional CT images, navigation systems based on intraoperative two-dimensional X-ray images, and navigation systems based on intraoperative three-dimensional X-ray images. The navigation system based on preoperative three-dimensional CT images uses preoperative CT data to render a three-dimensional model, but due to the displacement and rotation of the lesion during the operation, the bone structure relationship in the relevant area of ​​the lesion will change during the operation. The bone structure relationship reflected by the three-dimensional model rendered by the preoperative CT data does not match the real situation in real time. Therefore, the navigation system based on three-dimensional CT data generally focuses on helping doctors to formulate a complete, scientific and practical surgical execution plan for patients before the operation, and the execution process of the operation still depends on the doctor's empirical judgment. The navigation system based on intraoperative two-dimensional X-ray images uses intraoperative X-ray images to provide doctors with two-dimensional image information of the intraoperative lesion site, but it requires a large number of X-rays, which will prolong the operation time and increase the doctor's radiation exposure time, which is potentially harmful. Different from the traditional two-dimensional C-arm X-ray acquisition device, the navigation system based on the intraoperative three-dimensional C-arm realizes the acquisition of X-ray images in another way. Relying on its unique imaging principle, the three-dimensional C-arm can reconstruct a three-dimensional image from a two-dimensional image, achieving the effect of taking a single shot with both a two-dimensional image and a corresponding three-dimensional image model. It can be said that it has the advantages of both CT image-based and X-ray image navigation systems. However, the main manufacturers of three-dimensional C-arm equipment are foreign manufacturers, the cost is relatively high, and there are no large number of clinical examples to verify the effect, resulting in unsatisfactory final image registration. In summary, the surgical navigation system in the relevant technology has problems such as reliance on single data, non-intuitive surgical process information, unclear impact of surgical process on surgical results, and high intraoperative radiation. Summary of the invention

[0003] In order to solve the above technical problems, the purpose of the present invention is to provide a coordinate transformation and registration method and system based on CT images and X-ray images, which can reduce the harm of doctors' exposure to radiation environment and simplify the transformation of image space coordinates, thereby improving the registration accuracy of CT images and X-ray images.

[0004] The first technical solution adopted by the present invention is: a coordinate transformation registration method based on CT images and X-ray images, comprising the following steps:

[0005] Based on the preoperative CT data, rendering processing is performed to obtain an X-ray image with calibration target marks, wherein the X-ray image with calibration target marks includes an X-ray image with calibration target marks in the frontal position and an X-ray image with calibration target marks in the lateral position;

[0006] Perform feature preprocessing based on the X-ray image with calibration target marks to obtain the coordinate features of the two-dimensional X-ray image and obtain a correction matrix for eliminating the distortion of the X-ray image;

[0007] Based on the coordinate features of the two-dimensional X-ray image, the transformation matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system is obtained, and the coordinate transformation and registration between the three-dimensional real space and the CT three-dimensional space is completed by combining the registration method and correction matrix of the two-dimensional X-ray image and CT image.

[0008] Furthermore, the step of performing rendering processing based on the preoperative CT data to obtain an X-ray image with a calibration target marker specifically includes:

[0009] Turn on the C-arm X-ray machine and the optical positioning device, wherein a calibration target is installed on one side of the image intensification end of the C-arm X-ray machine, and the optical positioning mark of the optical positioning device is within a preset working range;

[0010] Import preoperative CT data, render the 3D model through the VTK open source library algorithm and connect the optical positioning device so that the optical positioning markers can be detected, import the surgical probe for calibration and install the calibration target;

[0011] Based on the installed and fixed calibration target, the C-arm X-ray machine is used to take pictures to obtain an X-ray image with calibration target marks.

[0012] Furthermore, the step of performing feature preprocessing based on the X-ray image with the calibration target mark to obtain the two-dimensional X-ray image coordinate features and obtain the correction matrix for eliminating the X-ray image distortion specifically includes:

[0013] Performing format conversion preprocessing on the X-ray image with the calibration target mark to obtain a converted X-ray image with the calibration target mark;

[0014] Performing feature extraction processing on the converted X-ray image with the calibration target mark to obtain features of the X-ray image with the calibration target mark;

[0015] Performing feature sorting processing on the X-ray image features with the calibration target mark to obtain sorted two-dimensional X-ray image features with the calibration target mark;

[0016] The ideal image is established and the point set is matched on the arranged two-dimensional X-ray image features with calibration target marks to obtain a correction matrix for eliminating X-ray image distortion.

[0017] Furthermore, the step of performing format conversion preprocessing on the X-ray image with the calibration target mark to obtain the converted X-ray image with the calibration target mark specifically includes:

[0018] Based on the SimpleITK library, the X-ray image with calibration target marks is converted to obtain a SimpleITK format image with calibration target marks;

[0019] Convert the SimpleITK format image with calibration target marks into a NumPy array to obtain the two-dimensional data of the X-ray image with calibration target marks;

[0020] Normalizing the two-dimensional data of the X-ray image with the calibration target mark to obtain a normalized X-ray image with the calibration target mark;

[0021] The normalized X-ray image with the calibration target mark is subjected to pixel expansion and storage processing to obtain a converted X-ray image with the calibration target mark.

[0022] Furthermore, the step of performing feature extraction processing on the converted X-ray image with the calibration target mark to obtain the features of the X-ray image with the calibration target mark specifically includes:

[0023] Performing gamma enhancement processing on the converted X-ray image with the calibration target mark to obtain an enhanced X-ray image with the calibration target mark;

[0024] Performing binary segmentation processing on the enhanced X-ray image with the calibration target mark to obtain a segmented X-ray image with the calibration target mark;

[0025] Performing region of interest extraction processing on the segmented X-ray image with the calibration target mark to obtain an extracted X-ray image with the calibration target mark;

[0026] Calculating the circular confidence of the extracted X-ray image with the calibration target mark to obtain the circular confidence value of the X-ray image with the calibration target mark;

[0027] Based on the circular confidence value of the X-ray image with the calibration target mark, the minimum circumscribed circle of the shape is obtained and stored, and the feature extraction processing is cyclically performed on the converted X-ray image with the calibration target mark to obtain the features of the X-ray image with the calibration target mark.

[0028] Furthermore, the step of performing feature sorting processing on the X-ray image features with the calibration target mark to obtain sorted two-dimensional X-ray image features with the calibration target mark specifically includes:

[0029] The X-ray image features with calibration target marks are processed into feature point sets, and a point set containing mark points of the anteroposterior lower mark plate, a point set containing mark points of the anteroposterior upper and lower mark plates, a point set containing mark points of the lateral lower mark plate, and a point set containing mark points of the lateral upper and lower mark plates are obtained;

[0030] The point set containing the marking points of the lower marking plate in the correct position and the point set containing the marking points of the upper and lower marking plates in the correct position are judged by the distance between the centers of the circles and classified to obtain the point set containing the upper marking in the correct position;

[0031] The point set containing the marking points of the lateral lower marking plate and the point set containing the marking points of the lateral upper and lower marking plates are judged and classified to obtain the point set containing the lateral upper marking;

[0032] The point set containing the marking points of the anteroposterior lower marking plate, the point set containing the anteroposterior upper marking, the point set containing the marking points of the lateral lower marking plate and the point set containing the lateral upper marking are classified according to the center coordinates, sorted and grouped from small to large according to the y coordinate, and the sorted two-dimensional X-ray image features with calibration target markings are output.

[0033] Furthermore, the step of performing ideal image establishment and point set matching on the sorted two-dimensional X-ray image features with calibration target marks to obtain a correction matrix for eliminating X-ray image distortion specifically includes:

[0034] Based on the upper and lower marking plate marking points in the sorted two-dimensional X-ray image features with calibration target markings, a real point set is constructed;

[0035] Extract the center point of the real point set to generate a virtual distortion-free point set;

[0036] The mapping relationship between the real point set and the virtual distortion-free point set is obtained by the least square method, and the correction matrix for eliminating the distortion of the X-ray image is obtained.

[0037] Furthermore, the step of obtaining a conversion matrix between a two-dimensional X-ray feature image coordinate system and a real three-dimensional space coordinate system based on the two-dimensional X-ray image coordinate feature, and combining the registration method and correction matrix of the two-dimensional X-ray image and the CT image to complete the coordinate conversion registration between the three-dimensional real space and the CT three-dimensional space specifically includes:

[0038] Based on the coordinate features of the two-dimensional X-ray image, a one-to-one correspondence between the two-dimensional coordinates of the features in the X-ray image and the three-dimensional coordinates of the features in the real space is established, and the transformation matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system is solved by the least squares method;

[0039] Determine the conversion matrix from the calibration target coordinate system to the optical positioning mark coordinate system on the side of the calibration target through mechanical dimensions;

[0040] Determine the conversion matrix from the optical positioning mark coordinate system on the side of the calibration target to the optical positioning device coordinate system by the optical positioning device;

[0041] Superimpose the conversion matrix of the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system, the conversion matrix of the calibration target coordinate system to the optical positioning mark coordinate system on the side of the calibration target, and the conversion matrix of the optical positioning mark coordinate system on the side of the calibration target to the optical positioning device coordinate system, and obtain the transformation matrix from the two-dimensional coordinates of the X-ray image to the optical positioning device coordinate system;

[0042] Based on the transformation matrix from the two-dimensional coordinates of the X-ray image to the coordinate system of the optical positioning device, combined with the registration method and correction matrix of the CT image and X-ray image, the coordinate conversion and registration between the three-dimensional real space and the CT three-dimensional space is completed.

[0043] The second technical solution adopted by the present invention is: a registration system for CT images and X-ray images based on coordinate transformation, comprising:

[0044] The first module is used for performing rendering processing based on the preoperative CT data to obtain an X-ray image with a calibration target mark, wherein the X-ray image with the calibration target mark includes an X-ray image with a calibration target mark in an anteroposterior position and an X-ray image with a calibration target mark in a lateral position;

[0045] The second module is used to perform feature preprocessing based on the X-ray image with the calibration target mark, obtain the coordinate features of the two-dimensional X-ray image and obtain the correction matrix for eliminating the distortion of the X-ray image;

[0046] The third module is used to obtain the transformation matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system based on the two-dimensional X-ray image coordinate features, and combine the registration method and correction matrix of the two-dimensional X-ray image and CT image to complete the coordinate transformation and registration between the three-dimensional real space and the CT three-dimensional space.

[0047] The beneficial effects of the method and system of the present invention are as follows: the present invention uses preoperative CT data to perform rendering processing to obtain an X-ray image with calibration target marks, and combines the preoperative CT three-dimensional data. The intraoperative effect can be directly presented on the three-dimensional model, which is convenient for doctors to intuitively observe the effects of the surgical process, reduce the number of times doctors take X-ray images, and reduce the hazards of doctors being exposed to radiation environments. Further, feature preprocessing is performed based on the X-ray image with calibration target marks to obtain two-dimensional X-ray feature images and three-dimensional X-ray feature images, which can more effectively obtain real-time information during the operation and judge the actual implementation effect of the operation. By obtaining the conversion matrix between the two-dimensional coordinates of the X-ray image and the three-dimensional coordinates of the world coordinate system, and combining the registration of the CT image and the X-ray image, the alignment and registration between the world coordinate system and the CT virtual coordinate system are completed, and the conversion of spatial coordinates is completed more simply. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flow chart of steps of a coordinate transformation registration method based on CT images and X-ray images of the present invention;

[0049] Figure 2 It is a structural block diagram of a registration system of CT images and X-ray images based on coordinate transformation of the present invention;

[0050] Figure 3 It is a schematic diagram of the framework of the registration process of CT images and X-ray images provided by a specific embodiment of the present invention;

[0051] Figure 4 It is a schematic diagram of the operation flow of connecting an optical device and acquiring a surgical probe provided by a specific embodiment of the present invention;

[0052] Figure 5 is a schematic diagram of a virtual space rendering result provided by a specific embodiment of the present invention;

[0053] Figure 6 It is a schematic diagram of a process of collecting images with a calibration target provided by a specific embodiment of the present invention;

[0054] Figure 7 It is a schematic diagram of the process of DCM image format conversion provided by a specific embodiment of the present invention;

[0055] Figure 8 It is a schematic diagram of the process of image feature extraction provided by a specific embodiment of the present invention;

[0056] Fig. 9 is a schematic diagram of extracting an image region of interest provided by a specific embodiment of the present invention;

[0057] Fig.10 is a schematic diagram of feature extraction of an image provided by a specific embodiment of the present invention;

[0058] Fig.11 is a schematic diagram of a process of sorting image features provided by a specific embodiment of the present invention;

[0059] Fig.12 is a schematic diagram of the result of feature arrangement of an orthotopic image provided by a specific embodiment of the present invention;

[0060] Fig.13 is a schematic diagram of the result of lateral image feature arrangement provided by a specific embodiment of the present invention;

[0061] Fig.14 is a schematic diagram of a process of image correction provided by a specific embodiment of the present invention;

[0062] Fig.15 is a schematic diagram of applying a correction matrix to a distorted image provided by a specific embodiment of the present invention;

[0063] Fig.16 It is a schematic diagram of a two-dimensional coordinate to three-dimensional coordinate mapping process provided by a specific embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only provided for the convenience of explanation and description, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0065] First, the technical terms in the embodiments of the present invention are explained:

[0066] 1) C-arm X-ray machine: The current mainstream X-ray imaging equipment is structurally divided into an X-ray transmitting end, an image enhancing end, and a C-arm structure. When working, X-rays are emitted from one side of the transmitting end, penetrate human tissue, and are received at the image enhancing end. After being processed by a computer, an image of the internal structure of the human body is formed. The C-arm structure can flexibly rotate and move around the patient, making it convenient to perform X-ray imaging from different angles. With its real-time imaging and high flexibility, it is often used for intraoperative image acquisition in orthopedic clinical surgery.

[0067] 2) Calibration target: installed on the image enhancement end of the C-arm X-ray machine, with upper and lower calibration plates installed, with several marking points arranged in sequence attached to the plates, and optical positioning marks installed on the side of the target to capture the coordinates in the optical positioning device coordinate system. The main functions are: to achieve X-ray image calibration, eliminate distortion; to obtain the coordinate conversion between two-dimensional X-ray images and three-dimensional real space.

[0068] 3) Optical positioning markers and optical positioning equipment: An optical positioning marker is a device that is strictly designed and produced. It is usually in the shape of an "X" cross, and is equipped with more than three small marking balls coated with special coatings that can reflect near-infrared light. Optical positioning markers are usually attached to equipment in the surgical scene to position it, including robotic arms in surgical robot scenes and probe tools in surgical navigation. Optical positioning equipment is mostly used in surgical navigation. It is a device that uses optical principles to determine the position and rotation matrix of an object. It can emit high-frequency near-infrared light and capture the near-infrared light reflected by the optical positioning marker through a binocular camera. It reconstructs the three-dimensional coordinates of the marker in the coordinate system of the positioning device, and the positioning accuracy can reach the millimeter level. The optical positioning marker must be within the working range of the optical positioning device to be captured normally. Any obstruction of the marker during surgery is considered an abnormal situation and should be avoided.

[0069] 4) Real space coordinate system: that is, the real surgical scene coordinate system, which is a three-dimensional coordinate system, generally refers to the optical positioning device coordinate system.

[0070] 5) Virtual space coordinate system: refers to the virtual space in the computer, which is a three-dimensional coordinate system. It generally refers to the CT image rendering space coordinate system, that is, the space in which the CT image renders a three-dimensional model.

[0071] 6) X-ray image coordinate system: that is, the image coordinate system after taking the X-ray image, which is a two-dimensional coordinate system.

[0072] 7) VTK: VTK is an open source software system for data visualization and image processing. It provides a large number of algorithms and tools that can help developers efficiently process and visualize various data, such as scientific computing data, medical image data, etc. The rendering algorithm used in the present invention is the Marching Cubes rendering algorithm, which is mainly used to extract isosurfaces from three-dimensional scalar field data (such as density values ​​in medical CT scan data). The core idea is to divide the three-dimensional space into small cubes (voxels). For each small cube, the intersection of the isosurface and the cube is determined by analyzing the relationship between the scalar value of its vertex (such as density value) and the given isosurface value. According to the pre-defined lookup table, the intersection points of the isosurface and the edge of the cube can be obtained, and these intersection points can be connected to approximately construct the geometric shape of the isosurface.

[0073] 8) Least Squares Method: It is a mathematical optimization technique. It finds the best function matching data by minimizing the sum of squares of errors. Simply put, when we have a set of data points and assume that these data points conform to a certain functional relationship (such as linear function, polynomial function, etc.), the least squares method can help us find the best parameters of this function so that this function is as close to these data points as possible.

[0074] Reference Figure 1 The present invention provides a coordinate transformation registration method based on CT images and X-ray images, the method comprising the following steps:

[0075] S100, performing rendering processing based on preoperative CT data to obtain an X-ray image with calibration target marks, wherein the X-ray image with calibration target marks includes an X-ray image with calibration target marks in the frontal position and an X-ray image with calibration target marks in the lateral position;

[0076] In this embodiment, the C-arm X-ray machine is first turned on, and a calibration target is installed on one side of the image enhancement end of the C-arm X-ray machine; the optical positioning device is turned on to ensure that the optical positioning mark is within the specified working range and then import the preoperative CT data, and use the VTK open source library algorithm to render the three-dimensional model; the optical positioning device is connected to ensure that the optical positioning mark is detected, import the surgical probe tool and perform relevant calibration, and finally use the C-arm X-ray machine to take two images with calibration target marks in the anteroposterior and lateral positions, and record the coordinates and rotation matrix of the optical mark attached to the calibration target under the optical positioning device in the anteroposterior (0°) and lateral (90°) positions respectively.

[0077] Furthermore, it should be noted that if Figure 4 As shown, connect the optical device to ensure normal transmission of device data, determine whether the surgical probe and the calibration target positioning mark are within the working range, distinguish the two optical marks with the help of the collection file, and bind the data transmitted by the optical positioning device to the corresponding data structure respectively, import the probe, import the probe in the virtual space, update the position of the probe by binding the data transmitted by the optical positioning device to reflect the movement of the probe in the real space, and complete the tip calibration by sampling around the needle tip with the probe: find the coordinate transformation from the probe tip to the probe center, reflect the direction of the probe in the real space (probe center to needle tip) in the virtual space, and render the entire probe. At this time, the probe direction and movement in the virtual space (CT data space) can reflect the events in the real space, such as Figure 5 shown.

[0078] Furthermore, it should be noted that Figure 6 As shown, collecting the anteroposterior and lateral images with calibration targets refers to collecting X-ray images of the upper and lower plates of the calibration target in the anteroposterior and lateral positions respectively, that is, four X-ray images need to be collected, among which the first X-ray image is an anteroposterior image containing only the lower calibration plate, the second X-ray image is an anteroposterior image containing both the upper and lower calibration plates, the third X-ray image is a lateral image containing both the upper and lower calibration plates, and the fourth X-ray image is a lateral image containing only the lower calibration plate. The purpose of such collection is to complete the collection of the image marking features of the upper and lower calibration plates in the anteroposterior and lateral positions respectively without disassembling the calibration plates multiple times. The collection process is as follows:

[0079] 1) Install the lower calibration plate and collect the first X-ray image;

[0080] 2) Install the upper calibration plate and collect the second X-ray image;

[0081] 3) Rotate the C-arm 90° to collect the third X-ray image;

[0082] 4) Remove the upper calibration plate and collect the fourth X-ray image;

[0083] 5) Restore the C-arm machine and remove the lower calibration plate.

[0084] S200, performing feature preprocessing based on the X-ray image with the calibration target mark to obtain the coordinate features of the two-dimensional X-ray image and obtain a correction matrix for eliminating the distortion of the X-ray image;

[0085] In this embodiment, firstly, the format conversion, feature extraction and feature arrangement operations are performed on the two images with calibration target marks in the anteroposterior and lateral positions, wherein the feature extraction is divided into the following steps: image enhancement, threshold segmentation, region of interest extraction, and minimum circumscribed circle extraction to obtain the marking features of the upper and lower plates of the calibration target. Then, the extracted marking features are arranged respectively, and the X-ray image distortion correction matrix is ​​obtained by polynomial fitting, which can be used to correct the X-ray image to eliminate the distortion caused by the imaging principle. Finally, the coordinates of the marking points on the calibration target in the anteroposterior and lateral positions are obtained by combining the two sets of rotation matrices with the mechanical dimensions of the calibration target. In the coordinate system of the optical positioning device,

[0086] S210, performing format conversion preprocessing on the X-ray image with the calibration target mark to obtain a converted X-ray image with the calibration target mark;

[0087] Specifically, based on the SimpleITK library, the X-ray image with calibration target marks is converted to obtain a SimpleITK format image with calibration target marks; the SimpleITK format image with calibration target marks is converted into a NumPy array to obtain the two-dimensional data of the X-ray image with calibration target marks; the two-dimensional data of the X-ray image with calibration target marks is normalized to obtain the normalized X-ray image with calibration target marks; the normalized X-ray image with calibration target marks is pixel expanded and stored to obtain the converted X-ray image with calibration target marks.

[0088] In some embodiments, such as Figure 7As shown, determine whether the file format is a .dcm file, and use the SimpleITK library (sitK) in Python to read the DICOM file. Read the DICOM file of the specified path and convert it into a SimpleITK image object, convert the SimpleITK image object into a NumPy array, obtain its shape information, and then resize it into a two-dimensional array. This is usually because the DICOM file may contain multiple slices, and only the image data of one of the slices is needed here, so convert the three-dimensional array into a two-dimensional array, obtain the maximum and minimum values ​​of the image, and create an array containing the lower and upper limits of the window. Convert the maximum and minimum values ​​to floating-point numbers to ensure the accuracy of subsequent calculations, and normalize the input DICOM image data. Map the grayscale values ​​of the image data to a range of 0 to 1. Normalization is achieved by subtracting the lower limit of the window from each pixel value and then dividing it by the window range, extending the pixel value to [0,255]. The pixel values ​​of the normalized image data are expanded to the range of 0 to 255, and then the data type is converted to uint8 to meet the image storage format requirements. The processed files are stored in the specified directory, and all files in the folder are traversed. The format conversion processing steps are repeated for the X-ray image with the calibration target mark to obtain the processed image.

[0089] S220, performing feature extraction processing on the converted X-ray image with the calibration target mark to obtain features of the X-ray image with the calibration target mark;

[0090] Specifically, gamma enhancement processing is performed on the converted X-ray image with calibration target marks to obtain an enhanced X-ray image with calibration target marks; binary segmentation processing is performed on the enhanced X-ray image with calibration target marks to obtain a segmented X-ray image with calibration target marks; region of interest extraction processing is performed on the segmented X-ray image with calibration target marks to obtain an extracted X-ray image with calibration target marks; circular confidence calculation is performed on the extracted X-ray image with calibration target marks to obtain a circular confidence value of the X-ray image with calibration target marks; based on the circular confidence value of the X-ray image with calibration target marks, the shape minimum circumscribed circle is obtained and stored, and feature extraction processing is cyclically performed on the converted X-ray image with calibration target marks to obtain features of the X-ray image with calibration target marks.

[0091] In some embodiments, such as Figure 8 As shown in the figure, firstly, the image is gamma enhanced to increase the contrast of the image, the grayscale histogram of the image is calculated and the upper and lower bounds of the threshold segmentation are determined, the image is binary segmented, and then the region of interest is extracted from the image. The extraction result is shown in Fig. 9As shown, the number of points of each shape in the image is counted. If the number is greater than a certain value, the shape is considered to be a polygon (including a circle). The contour area and perimeter of the shape are calculated, and the confidence that the shape is a circle is calculated. The expression is:

[0092]

[0093] Among them, S is the area of ​​the shape, d is the perimeter of the shape, if the confidence is within the confidence range, the shape is considered to be a circle, it is added to the set A, the minimum circumscribed circle of the shape contour in the set Q is calculated, the coordinates of the center of the circle and the radius of the circle are obtained and stored in the set In the above example, the feature extraction process of the converted X-ray image with calibration target markers is repeated for the processed image, and the point set of the storage marker circumscribed circle features is obtained as A1, A2, A3, A4, as shown in Fig.10 shown.

[0094] S230, performing feature sorting processing on the X-ray image features with the calibration target mark to obtain sorted two-dimensional X-ray image features with the calibration target mark;

[0095] Specifically, the X-ray image features with calibration target marks are grouped and processed into feature point sets to obtain a point set containing marking points of the anteroposterior lower marking plate, a point set containing marking points of the anteroposterior upper and lower marking plates, a point set containing marking points of the lateral lower marking plate, and a point set containing marking points of the lateral upper and lower marking plates; the point set containing the marking points of the anteroposterior lower marking plate and the point set containing the marking points of the anteroposterior upper and lower marking plates are subjected to circle center distance judgment and classification processing to obtain a point set containing the anteroposterior upper mark; the point set containing the marking points of the lateral lower marking plate and the point set containing the marking points of the lateral upper and lower marking plates are subjected to circle center distance judgment and classification processing to obtain a point set containing the lateral upper mark; the point set containing the marking points of the anteroposterior lower marking plate, the point set containing the anteroposterior upper mark, the point set containing the marking points of the lateral lower marking plate, and the point set containing the lateral upper mark are respectively classified according to the center coordinates, sorted and grouped according to the y coordinate from small to large, and the sorted two-dimensional X-ray image features with calibration target marks are output.

[0096] In some embodiments, such as Fig.11 As shown, the point set after feature extraction is divided into two groups: group a contains point set A1 of the marking points of the lower marking plate in the anteroposterior position, and point set A2 of the marking points of the upper + lower marking plate in the anteroposterior position; group b contains point set A3 of the marking points of the lower marking plate in the lateral position, and point set A4 of the marking points of the upper + lower marking plate in the lateral position.

[0097] Determine the distance between the centers of two point sets in the same group. If the center distance is less than a specific value, then this circle is the circle jointly owned by the lower marker plate in the frontal position and the upper + lower marker plates in the frontal position, that is, the marker on the lower marker plate. If the center distance is less than the specific value, then this circle is considered to be the marker on the upper plate, and it is added to the new point set B1. The same operation is performed on another group of point sets to obtain the new point set B2. The following four point sets are obtained: the lower frontal marker A1, the upper frontal marker B1, the lower lateral marker A3, and the upper lateral marker B2. Classify the four point sets according to the center coordinates, specifically: first sort them into several groups according to the x coordinate, and then sort them in ascending order of the y coordinate within each group. The finally obtained markers are grouped by column, and the y coordinate within the group is in ascending order. The expression is as follows:

[0098] C i ={c1,c2,...c n}, i = 1, 2, 3, 4

[0099] where c j ={((x1,y1)),((x2,y2)),((x3,y3)),...((x m ,y m ))|y1 < y2 <... < y m ,|x o -x p |< ε}, 0 < j ≤ n, and the result is as Fig.12 and Fig.13 shown.

[0100] S240. Establish an ideal image and perform point set matching on the features of the processed two-dimensional X-ray image with calibration target markers to obtain a correction matrix for eliminating X-ray image distortion.

[0101] Specifically, based on the upper and lower marker points in the features of the processed two-dimensional X-ray image with calibration target markers, construct a real point set; extract the center points of the real point set to generate a virtual distortion-free point set; obtain the mapping relationship between the real point set and the virtual distortion-free point set by the least squares method to obtain a correction matrix for eliminating X-ray image distortion.

[0102] In some embodiments, as Fig.14 shown, use the upper + lower marker points as the real point set The arrangement of extracting the center points to generate a virtual distortion-free point set Solve the mapping relationship between the real point set and the virtual point set by the least squares method. The obtained relationship is the correction matrix, and its principle is shown in the following formula:

[0103]

[0104] Where (p, q) is the coordinate of the undistorted point set, S is the correction parameter, and its expression is

[0105]

[0106] That is, the overdetermined equation is:

[0107]

[0108] Solve it using the least squares method, A is what we want to find, and finally, Fig.15 As shown, the correction matrix is ​​applied to the original image and the effect comparison is observed.

[0109] S300, based on the coordinate features of the two-dimensional X-ray image, a conversion matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system is obtained, and the coordinate conversion and registration between the three-dimensional real space and the CT three-dimensional space is completed by combining the registration method and correction matrix of the two-dimensional X-ray image and the CT image.

[0110] In this embodiment, a one-to-one correspondence is established between the obtained two-dimensional coordinates and the obtained three-dimensional coordinates, and the conversion matrix from the two-dimensional coordinates (X-ray image) to the three-dimensional coordinates (real space) is solved by the least squares method, and the calibration plate on the calibration target is unloaded. After unloading, intraoperative X-ray photography can be performed, and the lesion site can be photographed in the frontal and lateral positions. Combined with the two-dimensional and three-dimensional registration modules of CT images and X-ray images, the preoperative planning process can be completed: point planning is performed on the model rendered by CT data, and the point planning made is mapped to the real space through: CT space->X-ray image->optical positioning device coordinate system; intraoperative effect visualization: intraoperative cutting and other operations on the lesion site can be mapped to the virtual space through: X-ray image->calibration target coordinate system->optical positioning device coordinate system->CT space coordinate system; postoperative effect evaluation: the data obtained by surgery through postoperative CT and preoperative CT are compared to evaluate the execution of surgical planning and surgical effect.

[0111] Specifically, based on the coordinate features of the two-dimensional X-ray image, a one-to-one correspondence between the two-dimensional coordinates of the features in the X-ray image and the three-dimensional coordinates of the features in the real space is established, and the conversion matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system is solved by the least squares method; the conversion matrix from the calibration target coordinate system to the optical positioning mark coordinate system on the side of the calibration target is determined by the mechanical size; the conversion matrix from the optical positioning mark coordinate system on the side of the calibration target to the optical positioning device coordinate system is determined by the optical positioning device; the conversion matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system, the conversion matrix from the calibration target coordinate system to the optical positioning mark coordinate system on the side of the calibration target, and the conversion matrix from the optical positioning mark coordinate system on the side of the calibration target to the optical positioning device coordinate system are superimposed to obtain the transformation matrix from the two-dimensional coordinates of the X-ray image to the coordinate system of the optical positioning device; based on the transformation matrix from the two-dimensional coordinates of the X-ray image to the coordinate system of the optical positioning device, combined with the registration method and correction matrix of CT images and X-ray images, the coordinate conversion and registration between the three-dimensional real space and the CT three-dimensional space is completed.

[0112] In some embodiments, such as Fig.16 As shown, the rotation and translation matrix T of the calibration target coordinate system to the optical positioning mark coordinate system on the side of the calibration target is determined by the mechanical dimensions. The coordinates of the mark points in the upper plate and the lower plate in the calibration target coordinate system can be obtained by the following method:

[0113] TD i ,i=1,2

[0114] Where D1 represents the set of marked points on the upper plate, and D2 represents the set of marked points on the lower plate.

[0115] Determine the transformation matrix T1 and T2 from the calibration target mark coordinate system to the optical positioning device coordinate system, where T1 is the transformation matrix when the optical positioning device is in the upright position, and T2 is the transformation matrix when the optical positioning device is in the side position. The coordinates of the mark points on the calibration target in the optical positioning system coordinate system can be obtained by coordinate superposition, specifically:

[0116] T1TD i ,i=1,2

[0117] T2TD i ,i=1,2

[0118] Determine the one-to-one correspondence between two-dimensional and three-dimensional point sets:

[0119] When upright: C i →T1TD i ,i=1,2;

[0120] Lateral position: C j →T2TD j ,j=1,2;

[0121] The least square method is used to calculate the mapping relationship M1 from the two-dimensional image to the upper plate in the anteroposterior position, the mapping relationship M2 from the two-dimensional image to the lower plate in the anteroposterior position, the mapping relationship N1 from the two-dimensional image to the upper plate in the lateral position, and the mapping relationship N2 from the two-dimensional image to the lower plate in the lateral position. The expressions are:

[0122]

[0123] Based on the imaging principle of the C-arm X-ray machine, the mapping of the two-dimensional image coordinates of a point P during surgery to the three-dimensional coordinates (under the optical positioning system) can be calculated as follows: respectively calculate the intersection points of the X-ray passing through this point P with the upper and lower plates in the anteroposterior position, as well as the spatial equation of this ray; respectively calculate the intersection points of the X-ray image irradiating this point with the upper and lower plates in the lateral position, as well as the spatial equation of this ray. Solve for the point closest to these two straight lines, which is the coordinate of point P in space. For example, if the two-dimensional X-ray image coordinates of a point P in the anteroposterior and lateral positions under the C-arm X-ray machine are known to be (x1, y1) and (x2, y2), respectively, the three-dimensional coordinates of this point under the optical positioning device can be calculated in the following way:

[0124] When in the correct position, the intersection of the X-ray rays passing through this point and the upper and lower plates of the calibration target is:

[0125] (X1,Y1,Z1)=M1(x1,y1,1) T

[0126] (X2,Y2,Z2)=M2(x1,y1,1) T

[0127] The intersection of the X-ray rays passing through this point and the upper and lower plates of the calibration target in the lateral position is:

[0128] (X3,Y3,Z3)=N1(x2,y2,1) T

[0129] (X4,Y4,Z4)=N2(x2,y2,1) T

[0130] The equations of the X-ray lines passing through this point in the anteroposterior and lateral positions are:

[0131]

[0132] tidy:

[0133]

[0134] The point to be sought is: the point closest to the two straight lines, that is, to find the distance between the two closest points on the two straight lines:

[0135]

[0136] Right now:

[0137]

[0138] At this point, the conversion of the two-dimensional coordinates of the X-ray image to the three-dimensional coordinates of the real space (optical navigation device) has been completed. Combined with the registration of the CT image and the X-ray image, the alignment between the real space coordinate system and the virtual space coordinate system (CT space coordinate system) can be completed.

[0139] In summary, if Figure 3 As shown, the embodiment of the present invention realizes the navigation system based on CT images and X-ray images by designing a calibration target installed on a C-arm X-ray machine based on the advantages of the CT image and X-ray image navigation system. The present invention realizes the correction of X-ray images, and the unification of the X-ray image coordinate system, the CT image coordinate system, and the optical positioning system coordinate system (real space coordinate system). Through preoperative registration, the surgery can be planned, the intraoperative information can be intuitively reflected, and the impact of the surgical process on the results can be analyzed.

[0140] Therefore, compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0141] 1) Compared with the traditional navigation system based on X-ray images, the present invention combines the preoperative CT three-dimensional data, and the intraoperative effect can be directly presented on the three-dimensional model, which is convenient for doctors to intuitively observe the effect of the surgical process and analyze which step of the operation affects the surgical effect. It can reduce the number of times doctors take X-ray images and reduce the harm of doctors being exposed to radiation environments.

[0142] 1) Compared with the traditional navigation system based on CT images, the present invention combines intraoperative X-ray images, which can more effectively obtain real-time information during the operation, judge the actual effect of the operation, and compare it with the preoperative plan to evaluate the real-time effect of the operation. At the same time, it reduces the difficulty of operation for doctors and streamlines the surgical process.

[0143] 2) The present invention performs anomaly detection and judgment in the data acquisition stage after connecting the optical positioning device, which can ensure that the abnormal data of the optical positioning device is eliminated without affecting the stability of the system.

[0144] 3) The present invention distinguishes different optical markers through the unique geometry file of each optical marker, and can update the geometry file, ensuring that the distinction after the optical positioning device data is acquired is rapid and effective. In the import and rendering of the probe and the real-time tracking part, the present invention integrates the needle tip calibration function of the probe tool, which can find the coordinate transformation from the probe tip to the probe center, and can be manually updated multiple times after importing, ensuring the real-time and accuracy of the probe tracking before each operation.

[0145] 4) Two tasks were accomplished by designing the calibration target: 1) the conversion relationship between the intraoperative X-ray image two-dimensional coordinate system and the three-dimensional real space coordinate system (optical positioning coordinate system); 2) calibration of the intraoperative X-ray image to reduce the surgical accuracy problems caused by image distortion caused by the imaging principle of the C-arm X-ray machine. The calibration target has been designed and verified by field tests many times to ensure that it can fit the structure of the C-arm X-ray machine, and the double-layer structure and the arrangement of the marks on each layer are designed, and the corresponding algorithm is matched based on the design.

[0146] 5) The intraoperative information is extracted by designing the upper and lower layers of markers on the calibration target and designing a matching automated marker extraction algorithm. The automated marker extraction algorithm includes image format conversion, image enhancement, binary segmentation, region of interest extraction, shape determination, marker point organization and other steps. Corresponding response methods are provided for some abnormal situations that may occur, and the algorithm as a whole can accurately meet the information extraction requirements.

[0147] 6) The method proposed in the present invention of using a double-layer calibration target to complete the conversion of X-ray image two-dimensional coordinates to three-dimensional real space coordinates can easily complete the conversion of space coordinates, build a bridge between two-dimensional space and real three-dimensional space, and provide basic technical support for surgical navigation. At the same time, it greatly reduces the number of X-ray exposures during surgery, reduces the amount of radiation during surgery, reduces the potential surgical risks for on-site surgical personnel, and reduces the duration of surgery, reducing the consumption of doctors by surgery.

[0148] Reference Figure 2 , a registration system for CT images and X-ray images based on coordinate transformation, comprising:

[0149] The first module 201 is used for performing rendering processing based on preoperative CT data to obtain an X-ray image with calibration target marks, wherein the X-ray image with calibration target marks includes an X-ray image with calibration target marks in the frontal position and an X-ray image with calibration target marks in the lateral position;

[0150] The second module 202 is used to perform feature preprocessing based on the X-ray image with the calibration target mark to obtain the coordinate features of the two-dimensional X-ray image and obtain a correction matrix for eliminating the distortion of the X-ray image;

[0151] The third module 203 is used to obtain the transformation matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system based on the two-dimensional X-ray image coordinate features, and combine the registration method and correction matrix of the two-dimensional X-ray image and CT image to complete the coordinate transformation and registration between the three-dimensional real space and the CT three-dimensional space.

[0152] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0153] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A coordinate transformation registration method based on CT images and X-ray images, characterized in that: The following steps are involved: Based on the preoperative CT data, rendering processing is performed to obtain an X-ray image with calibration target marks, wherein the X-ray image with calibration target marks includes an X-ray image with calibration target marks in the frontal position and an X-ray image with calibration target marks in the lateral position; Perform feature preprocessing based on the X-ray image with calibration target marks to obtain the coordinate features of the two-dimensional X-ray image and obtain a correction matrix for eliminating the distortion of the X-ray image; Based on the coordinate features of the two-dimensional X-ray image, the transformation matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system is obtained, and the coordinate transformation and registration between the three-dimensional real space and the CT three-dimensional space is completed by combining the registration method and correction matrix of the two-dimensional X-ray image and CT image.

2. The coordinate transformation registration method based on CT images and X-ray images according to claim 1, characterized in that: The step of performing rendering processing based on the preoperative CT data to obtain an X-ray image with a calibration target marker specifically includes: Turn on the C-arm X-ray machine and the optical positioning device, wherein a calibration target is installed on one side of the image intensification end of the C-arm X-ray machine, and the optical positioning mark of the optical positioning device is within a preset working range; Import preoperative CT data, render the 3D model through the VTK open source library algorithm and connect the optical positioning device so that the optical positioning markers can be detected, import the surgical probe for calibration and install the calibration target; Based on the installed and fixed calibration target, the C-arm X-ray machine is used to take pictures to obtain an X-ray image with calibration target marks.

3. The coordinate transformation registration method based on CT images and X-ray images according to claim 2, characterized in that: The step of performing feature preprocessing based on the X-ray image with the calibration target mark to obtain the coordinate features of the two-dimensional X-ray image and obtain the correction matrix for eliminating the distortion of the X-ray image specifically includes: Performing format conversion preprocessing on the X-ray image with the calibration target mark to obtain a converted X-ray image with the calibration target mark; Performing feature extraction processing on the converted X-ray image with the calibration target mark to obtain features of the X-ray image with the calibration target mark; Performing feature sorting processing on the X-ray image features with the calibration target mark to obtain sorted two-dimensional X-ray image features with the calibration target mark; The ideal image is established and the point set is matched on the arranged two-dimensional X-ray image features with calibration target marks to obtain a correction matrix for eliminating X-ray image distortion.

4. The coordinate transformation registration method based on CT images and X-ray images according to claim 3, characterized in that: The step of performing format conversion preprocessing on the X-ray image with the calibration target mark to obtain the converted X-ray image with the calibration target mark specifically includes: Based on the SimpleITK library, the X-ray image with calibration target marks is converted to obtain a SimpleITK format image with calibration target marks; Convert the SimpleITK format image with calibration target marks into a NumPy array to obtain the two-dimensional data of the X-ray image with calibration target marks; Normalizing the two-dimensional data of the X-ray image with the calibration target mark to obtain a normalized X-ray image with the calibration target mark; The normalized X-ray image with the calibration target mark is subjected to pixel expansion and storage processing to obtain a converted X-ray image with the calibration target mark.

5. The coordinate transformation registration method based on CT images and X-ray images according to claim 4, characterized in that: The step of performing feature extraction processing on the converted X-ray image with the calibration target mark to obtain the features of the X-ray image with the calibration target mark specifically includes: Performing gamma enhancement processing on the converted X-ray image with the calibration target mark to obtain an enhanced X-ray image with the calibration target mark; Performing binary segmentation processing on the enhanced X-ray image with the calibration target mark to obtain a segmented X-ray image with the calibration target mark; Performing region of interest extraction processing on the segmented X-ray image with the calibration target mark to obtain an extracted X-ray image with the calibration target mark; Calculating the circular confidence of the extracted X-ray image with the calibration target mark to obtain the circular confidence value of the X-ray image with the calibration target mark; Based on the circular confidence value of the X-ray image with the calibration target mark, the minimum circumscribed circle of the shape is obtained and stored, and the feature extraction processing is cyclically performed on the converted X-ray image with the calibration target mark to obtain the features of the X-ray image with the calibration target mark.

6. The coordinate transformation registration method based on CT images and X-ray images according to claim 5, characterized in that: The step of performing feature sorting processing on the X-ray image features with the calibration target marks to obtain sorted two-dimensional X-ray image features with the calibration target marks specifically includes: The X-ray image features with calibration target marks are processed into feature point sets, and a point set containing mark points of the anteroposterior lower mark plate, a point set containing mark points of the anteroposterior upper and lower mark plates, a point set containing mark points of the lateral lower mark plate, and a point set containing mark points of the lateral upper and lower mark plates are obtained; The point set containing the marking points of the lower marking plate in the correct position and the point set containing the marking points of the upper and lower marking plates in the correct position are judged by the distance between the centers of the circles and classified to obtain the point set containing the upper marking in the correct position; The point set containing the marking points of the lateral lower marking plate and the point set containing the marking points of the lateral upper and lower marking plates are judged and classified to obtain the point set containing the lateral upper marking; The point set containing the marking points of the anteroposterior lower marking plate, the point set containing the anteroposterior upper marking, the point set containing the marking points of the lateral lower marking plate and the point set containing the lateral upper marking are classified according to the center coordinates, sorted and grouped from small to large according to the y coordinate, and the sorted two-dimensional X-ray image features with calibration target markings are output.

7. The coordinate transformation registration method based on CT images and X-ray images according to claim 6, characterized in that: The step of performing ideal image establishment and point set matching on the sorted two-dimensional X-ray image features with calibration target marks to obtain a correction matrix for eliminating X-ray image distortion specifically includes: Based on the upper and lower marking plate marking points in the sorted two-dimensional X-ray image features with calibration target markings, a real point set is constructed; Extract the center point of the real point set to generate a virtual distortion-free point set; The mapping relationship between the real point set and the virtual distortion-free point set is obtained by the least square method, and the correction matrix for eliminating the distortion of the X-ray image is obtained.

8. The coordinate transformation registration method based on CT images and X-ray images according to claim 7, characterized in that: The step of obtaining a conversion matrix between a two-dimensional X-ray feature image coordinate system and a real three-dimensional space coordinate system based on the two-dimensional X-ray image coordinate feature, and completing the coordinate conversion and registration between the three-dimensional real space and the CT three-dimensional space by combining the registration method and correction matrix of the two-dimensional X-ray image and the CT image, specifically includes: Based on the coordinate features of the two-dimensional X-ray image, a one-to-one correspondence between the two-dimensional coordinates of the features in the X-ray image and the three-dimensional coordinates of the features in the real space is established, and the transformation matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system is solved by the least squares method; Determine the conversion matrix from the calibration target coordinate system to the optical positioning mark coordinate system on the side of the calibration target through mechanical dimensions; Determine the conversion matrix from the optical positioning mark coordinate system on the side of the calibration target to the optical positioning device coordinate system by the optical positioning device; Superimpose the conversion matrix of the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system, the conversion matrix of the calibration target coordinate system to the optical positioning mark coordinate system on the side of the calibration target, and the conversion matrix of the optical positioning mark coordinate system on the side of the calibration target to the optical positioning device coordinate system, and obtain the transformation matrix from the two-dimensional coordinates of the X-ray image to the optical positioning device coordinate system; Based on the transformation matrix from the two-dimensional coordinates of the X-ray image to the coordinate system of the optical positioning device, combined with the registration method and correction matrix of the CT image and X-ray image, the coordinate conversion and registration between the three-dimensional real space and the CT three-dimensional space is completed.

9. A registration system for CT images and X-ray images based on coordinate transformation, characterized in that: Includes the following modules: The first module is used for performing rendering processing based on the preoperative CT data to obtain an X-ray image with a calibration target mark, wherein the X-ray image with the calibration target mark includes an X-ray image with a calibration target mark in an anteroposterior position and an X-ray image with a calibration target mark in a lateral position; The second module is used to perform feature preprocessing based on the X-ray image with the calibration target mark, obtain the coordinate features of the two-dimensional X-ray image and obtain the correction matrix for eliminating the distortion of the X-ray image; The third module is used to obtain the transformation matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system based on the two-dimensional X-ray image coordinate features, and combine the registration method and correction matrix of the two-dimensional X-ray image and CT image to complete the coordinate transformation and registration between the three-dimensional real space and the CT three-dimensional space.

Images