A coordinate transformation registration method and system based on CT images and X-ray images

Abstract

The present invention discloses a coordinate conversion and registration method and system based on CT images and X-ray images. The method includes: performing rendering processing based on preoperative CT data to obtain an X-ray image with calibration target marks; performing feature preprocessing based on the X-ray image with calibration target marks to obtain two-dimensional X-ray image coordinate features and obtain a correction matrix for eliminating X-ray image distortion; obtaining a conversion matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system based on the two-dimensional X-ray image coordinate features, and combining the registration method of the two-dimensional X-ray image and the CT image with the correction matrix to complete the coordinate conversion and registration between the three-dimensional real space and the CT three-dimensional space. The present invention can reduce the hazards of doctors being exposed to radiation environments, simplify the conversion of image space coordinates, and thereby improve the registration accuracy of CT images and X-ray images. As a coordinate conversion and registration method and system based on CT images and X-ray images, the present invention can be widely used in the field of image registration technology.

Description

Technical Field

[0001] The present invention relates to the technical field of image registration, and in particular to a coordinate conversion registration method and system based on CT images and X-ray images. Background Art

[0002] Current surgical navigation systems are generally categorized as follows: navigation systems based on preoperative 3D CT images, navigation systems based on intraoperative 2D X-ray images, and navigation systems based on intraoperative 3D X-ray images. Navigation systems based on preoperative 3D CT images use preoperative CT data to render 3D models. However, due to the displacement and rotation of the lesion during surgery, the bone structure relationships in the area surrounding the lesion will change during surgery. The 3D models rendered from preoperative CT data reflect bone structure relationships that do not match the real-time reality. Therefore, navigation systems based on 3D CT data generally focus on helping doctors develop a comprehensive, scientific, and patient-specific surgical plan before surgery. However, the execution of the surgery still relies on the doctor's empirical judgment. Navigation systems based on intraoperative 2D X-ray images use intraoperative X-ray images to provide doctors with 2D image information of the lesion. However, these systems require extensive X-ray imaging, which prolongs surgery time and increases the doctor's radiation exposure, posing potential risks. Different from the traditional two-dimensional C-arm X-ray acquisition device, the navigation system based on the intraoperative three-dimensional C-arm realizes the acquisition of X-ray images in another way. With its unique imaging principle, the three-dimensional C-arm can reconstruct a three-dimensional image from a two-dimensional image, achieving the effect of taking both a two-dimensional image and a corresponding three-dimensional image model in one shot. It can be said that it has the advantages of both CT image-based and X-ray image navigation systems. However, the main manufacturers of three-dimensional C-arm equipment are foreign manufacturers, the cost is relatively high, and there are no large number of clinical examples to verify the effect, resulting in the final image registration being unsatisfactory. In summary, the surgical navigation system in the relevant technology has problems such as reliance on a single data, non-intuitive surgical process information, unclear impact of surgical process on surgical results, and high intraoperative radiation dose. Summary of the Invention

[0003] In order to solve the above technical problems, the purpose of the present invention is to provide a coordinate transformation and registration method and system based on CT images and X-ray images, which can reduce the hazards of doctors' exposure to radiation environment and simplify the conversion of image space coordinates, thereby improving the registration accuracy of CT images and X-ray images.

[0004] The first technical solution adopted by the present invention is: a coordinate transformation registration method based on CT images and X-ray images, comprising the following steps:

[0005] Performing rendering processing based on preoperative CT data to obtain X-ray images with calibration target marks, wherein the X-ray images with calibration target marks include an anteroposterior X-ray image with calibration target marks and a lateral X-ray image with calibration target marks;

[0006] Perform feature preprocessing based on the X-ray image with calibration target marks to obtain the two-dimensional X-ray image coordinate features and obtain the correction matrix for eliminating X-ray image distortion;

[0007] Based on the coordinate features of the two-dimensional X-ray image, the transformation matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system is obtained. Combined with the registration method and correction matrix of the two-dimensional X-ray image and CT image, the coordinate transformation and registration between the three-dimensional real space and the CT three-dimensional space is completed.

[0008] Furthermore, the step of performing rendering processing based on the preoperative CT data to obtain an X-ray image with calibration target markers specifically includes:

[0009] Turn on the C-arm X-ray machine and the optical positioning device, wherein a calibration target is installed on one side of the image intensification end of the C-arm X-ray machine, and the optical positioning mark of the optical positioning device is within a preset working range;

[0010] Import preoperative CT data, render the 3D model using the VTK open source library algorithm, connect the optical positioning device so that the optical positioning markers can be detected, import the surgical probe for calibration, and install the calibration target;

[0011] Based on the installed and fixed calibration target, the C-arm X-ray machine is used to shoot and obtain an X-ray image with calibration target marks.

[0012] Furthermore, the step of performing feature preprocessing based on the X-ray image with the calibration target marker to obtain two-dimensional X-ray image coordinate features and obtain a correction matrix for eliminating X-ray image distortion specifically includes:

[0013] Performing format conversion preprocessing on the X-ray image with the calibration target mark to obtain a converted X-ray image with the calibration target mark;

[0014] Performing feature extraction processing on the converted X-ray image with the calibration target mark to obtain features of the X-ray image with the calibration target mark;

[0015] Performing feature sorting processing on the X-ray image features with the calibration target mark to obtain sorted two-dimensional X-ray image features with the calibration target mark;

[0016] The ideal image construction and point set matching are performed on the arranged two-dimensional X-ray image features with calibration target marks to obtain a correction matrix for eliminating X-ray image distortion.

[0017] Furthermore, the step of performing format conversion preprocessing on the X-ray image with the calibration target mark to obtain the converted X-ray image with the calibration target mark specifically includes:

[0018] Based on the SimpleITK library, the X-ray image with calibration target marks is converted to obtain a SimpleITK format image with calibration target marks;

[0019] Convert the SimpleITK format image with calibration target marks into a NumPy array to obtain the two-dimensional data of the X-ray image with calibration target marks;

[0020] Normalizing the two-dimensional data of the X-ray image with the calibration target mark to obtain a normalized X-ray image with the calibration target mark;

[0021] The normalized X-ray image with the calibration target mark is subjected to pixel expansion and storage processing to obtain a converted X-ray image with the calibration target mark.

[0022] Furthermore, the step of performing feature extraction processing on the converted X-ray image with the calibration target mark to obtain the features of the X-ray image with the calibration target mark specifically includes:

[0023] performing gamma enhancement processing on the converted X-ray image with the calibration target mark to obtain an enhanced X-ray image with the calibration target mark;

[0024] performing binary segmentation processing on the enhanced X-ray image with the calibration target mark to obtain a segmented X-ray image with the calibration target mark;

[0025] Performing region of interest extraction processing on the segmented X-ray image with the calibration target mark to obtain an extracted X-ray image with the calibration target mark;

[0026] Performing circular confidence calculation on the extracted X-ray image with the calibration target mark to obtain a circular confidence value of the X-ray image with the calibration target mark;

[0027] Based on the circular confidence value of the X-ray image with the calibration target mark, the minimum circumscribed circle of the shape is obtained and stored, and the feature extraction processing is cyclically performed on the converted X-ray image with the calibration target mark to obtain the X-ray image features with the calibration target mark.

[0028] Furthermore, the step of performing feature sorting processing on the X-ray image features with the calibration target mark to obtain sorted two-dimensional X-ray image features with the calibration target mark specifically includes:

[0029] Perform feature point set grouping processing on the X-ray image features with calibration target markers to obtain a point set containing marker points of the anteroposterior lower marker plate, a point set containing marker points of the anteroposterior upper and lower marker plates, a point set containing marker points of the lateral lower marker plate, and a point set containing marker points of the lateral upper and lower marker plates;

[0030] The point set containing the mark points of the correct lower mark plate and the point set containing the mark points of the correct upper and lower mark plates are judged by the center distance and classified to obtain the point set containing the correct upper mark;

[0031] The point set containing the marking points of the lateral lower marking plate and the point set containing the marking points of the lateral upper and lower marking plates are judged by the center distance and classified to obtain the point set containing the lateral upper marking;

[0032] The point set containing the marking points of the anteroposterior lower marking plate, the point set containing the anteroposterior upper marking, the point set containing the marking points of the lateral lower marking plate, and the point set containing the lateral upper marking are classified according to the center coordinates, sorted and grouped from small to large according to the y coordinate, and the sorted two-dimensional X-ray image features with calibration target marks are output.

[0033] Furthermore, the step of performing ideal image construction and point set matching on the sorted two-dimensional X-ray image features with calibration target marks to obtain a correction matrix for eliminating X-ray image distortion specifically includes:

[0034] Based on the upper and lower marking plate marking points in the sorted two-dimensional X-ray image features with calibration target markings, a real point set is constructed;

[0035] Extract the center point of the real point set to generate a virtual distortion-free point set;

[0036] The mapping relationship between the real point set and the virtual distortion-free point set is obtained by the least squares method, and the correction matrix for eliminating the distortion of the X-ray image is obtained.

[0037] Furthermore, the step of obtaining a conversion matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system based on the two-dimensional X-ray image coordinate features, and combining the registration method and correction matrix of the two-dimensional X-ray image and the CT image to complete the coordinate conversion and registration between the three-dimensional real space and the CT three-dimensional space specifically includes:

[0038] Based on the coordinate features of the two-dimensional X-ray image, a one-to-one correspondence between the two-dimensional coordinates of the features in the X-ray image and the three-dimensional coordinates of the features in the real space is established, and the transformation matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system is solved by the least squares method;

[0039] Determine the conversion matrix from the calibration target coordinate system to the optical positioning mark coordinate system on the side of the calibration target through mechanical dimensions;

[0040] Determine the conversion matrix from the optical positioning mark coordinate system on the side of the calibration target to the optical positioning device coordinate system by the optical positioning device;

[0041] Superimpose the conversion matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system, the conversion matrix between the calibration target coordinate system and the optical positioning mark coordinate system on the side of the calibration target, and the conversion matrix between the optical positioning mark coordinate system on the side of the calibration target and the optical positioning device coordinate system to obtain the transformation matrix from the two-dimensional coordinates of the X-ray image to the optical positioning device coordinate system;

[0042] Based on the transformation matrix from the two-dimensional coordinates of the X-ray image to the coordinate system of the optical positioning device, combined with the registration method and correction matrix of the CT image and X-ray image, the coordinate conversion and registration between the three-dimensional real space and the CT three-dimensional space is completed.

[0043] The second technical solution adopted by the present invention is: a registration system for CT images and X-ray images based on coordinate transformation, comprising:

[0044] The first module is configured to perform rendering processing based on preoperative CT data to obtain an X-ray image with calibration target marks, wherein the X-ray image with calibration target marks includes an anteroposterior X-ray image with calibration target marks and a lateral X-ray image with calibration target marks;

[0045] The second module is used to perform feature preprocessing based on the X-ray image with calibration target marks to obtain two-dimensional X-ray image coordinate features and obtain a correction matrix for eliminating X-ray image distortion;

[0046] The third module is used to obtain the transformation matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system based on the two-dimensional X-ray image coordinate features, and combine the registration method and correction matrix of the two-dimensional X-ray image and CT image to complete the coordinate transformation and registration between the three-dimensional real space and the CT three-dimensional space.

[0047] The beneficial effects of the method and system of the present invention are as follows: the present invention uses preoperative CT data for rendering processing to obtain X-ray images with calibration target marks, and combines the preoperative CT three-dimensional data. The intraoperative effect can be directly presented on the three-dimensional model, which is convenient for doctors to intuitively observe the effects of the surgical process, reduce the number of times doctors take X-ray images, and reduce the hazards of doctors being exposed to radiation environments. Further, feature preprocessing is performed based on the X-ray image with calibration target marks to obtain two-dimensional X-ray feature images and three-dimensional X-ray feature images, which can more effectively obtain real-time information during the operation and judge the actual implementation effect of the operation. By obtaining the conversion matrix between the two-dimensional coordinates of the X-ray image and the three-dimensional coordinates of the world coordinate system, and combining the registration of the CT image and the X-ray image, the alignment and registration between the world coordinate system and the CT virtual coordinate system are completed, and the conversion of spatial coordinates is completed more simply. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flowchart of the steps of a coordinate transformation registration method based on CT images and X-ray images of the present invention;

[0049] Figure 2 This is a structural block diagram of a registration system for CT images and X-ray images based on coordinate transformation according to the present invention;

[0050] Figure 3 1 is a schematic diagram of a registration process framework for CT images and X-ray images provided by a specific embodiment of the present invention;

[0051] Figure 4 It is a schematic diagram of the operation flow of connecting the optical device and obtaining the surgical probe provided by a specific embodiment of the present invention;

[0052] Figure 5 is a schematic diagram of a virtual space rendering result provided by a specific embodiment of the present invention;

[0053] Figure 6 It is a schematic diagram of a process for collecting images with a calibration target provided by a specific embodiment of the present invention;

[0054] Figure 7 This is a schematic diagram of the process of converting DCM image format provided by a specific embodiment of the present invention;

[0055] Figure 8 1 is a flow chart of image feature extraction according to a specific embodiment of the present invention;

[0056] Figure 9 is a schematic diagram of extracting an image region of interest provided by a specific embodiment of the present invention;

[0057] Figure 10 is a schematic diagram of image feature extraction provided by a specific embodiment of the present invention;

[0058] Figure 11 is a schematic diagram of a process for organizing image features according to a specific embodiment of the present invention;

[0059] Figure 12 is a schematic diagram of the result of feature sorting of an orthotopic image provided by a specific embodiment of the present invention;

[0060] Figure 13 is a schematic diagram of the result of lateral image feature sorting provided by a specific embodiment of the present invention;

[0061] Figure 14 is a schematic diagram of the image correction process provided by a specific embodiment of the present invention;

[0062] Figure 15 is a schematic diagram of applying a correction matrix to a distorted image provided by a specific embodiment of the present invention;

[0063] Figure 16 It is a schematic diagram of the two-dimensional coordinate to three-dimensional coordinate mapping process provided by a specific embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are provided for ease of description only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted based on the understanding of those skilled in the art.

[0065] First, the technical terms in the embodiments of the present invention are explained:

[0066] 1) C-arm X-ray Machine: This is the current mainstream X-ray imaging device, structurally composed of an X-ray transmitter, an image intensifier, and a C-arm. During operation, X-rays are emitted from the transmitter, penetrate human tissue, and are received by the image intensifier. After computer processing, they form an image of the body's internal structures. The C-arm can flexibly rotate and move around the patient, facilitating X-ray imaging from various angles. Due to its real-time imaging and high flexibility, it is often used for intraoperative image acquisition in orthopedic clinical surgery.

[0067] 2) Calibration Target: Installed on the image intensifier side of the C-arm X-ray machine, it features upper and lower calibration plates with sequentially arranged markers. Optical positioning markers are attached to the sides of the target to capture coordinates in the optical positioning device's coordinate system. Its primary functions include calibrating X-ray images, eliminating distortion, and converting coordinates from 2D X-ray images to 3D real-world space.

[0068] 3) Optical positioning markers and optical positioning equipment: An optical positioning marker is a device that is manufactured strictly in accordance with the design. It is usually in the shape of an "X" cross, and is equipped with more than three small marking balls coated with special coatings that can reflect near-infrared light. Optical positioning markers are usually attached to equipment in the surgical scene for positioning, including robotic arms in surgical robot scenes and probe tools in surgical navigation. Optical positioning equipment is mostly used in surgical navigation. It is a device that uses optical principles to determine the position and rotation matrix of an object. It can emit high-frequency near-infrared light and capture the near-infrared light reflected by the optical positioning marker through a binocular camera. It reconstructs the three-dimensional coordinates of the marker in the coordinate system of the positioning device, and the positioning accuracy can reach the millimeter level. The optical positioning marker must be within the working range of the optical positioning device to be captured normally. Any obstruction of the marker during surgery is considered an abnormality and should be avoided.

[0069] 4) Real space coordinate system: that is, the coordinate system of the real surgical scene, which is a three-dimensional coordinate system, generally refers to the coordinate system of the optical positioning device.

[0070] 5) Virtual space coordinate system: refers to the virtual space in the computer, which is a three-dimensional coordinate system. It generally refers to the CT image rendering space coordinate system, that is, the space in which the CT image renders the three-dimensional model.

[0071] 6) X-ray image coordinate system: This is the image coordinate system after the X-ray image is taken, which is a two-dimensional coordinate system.

[0072] 7) VTK: VTK is an open source software system for data visualization and image processing. It provides a large number of algorithms and tools that can help developers efficiently process and visualize various data, such as scientific computing data, medical image data, etc. The rendering algorithm used in the present invention is the Marching Cubes rendering algorithm, which is mainly used to extract isosurfaces from three-dimensional scalar field data (such as density values ​​in medical CT scan data). The core idea is to divide the three-dimensional space into small cubes (voxels). For each small cube, the intersection of the isosurface and the cube is determined by analyzing the relationship between the scalar value of its vertex (such as density value) and the given isosurface value. According to the pre-defined lookup table, the intersection points of the isosurface and the edge of the cube can be obtained. By connecting these intersection points, the geometric shape of the isosurface can be approximately constructed.

[0073] 8) Least Squares Method: This is a mathematical optimization technique that finds the best function matching the data by minimizing the sum of squared errors. Simply put, given a set of data points, and assuming that these data points conform to a certain functional relationship (such as a linear function or a polynomial function), the least squares method can help us find the optimal parameters of this function, so that the function is as close as possible to the data points.

[0074] Reference Figure 1 The present invention provides a coordinate transformation registration method based on CT images and X-ray images, the method comprising the following steps:

[0075] S100, performing rendering processing based on preoperative CT data to obtain X-ray images with calibration target marks, wherein the X-ray images with calibration target marks include an anteroposterior X-ray image with calibration target marks and a lateral X-ray image with calibration target marks;

[0076] In this embodiment, the C-arm X-ray machine is first turned on, and a calibration target is installed on one side of the image intensification end of the C-arm X-ray machine; the optical positioning device is turned on to ensure that the optical positioning mark is within the specified working range and then the preoperative CT data is imported, and the VTK open source library algorithm is used to render the three-dimensional model; the optical positioning device is connected to ensure that the optical positioning mark is detected, the surgical probe tool is imported and relevant calibration is performed, and finally, two images with calibration target marks are taken with the C-arm X-ray machine in the anteroposterior and lateral positions, and the coordinates and rotation matrix of the optical mark attached to the calibration target under the optical positioning device in the anteroposterior (0°) and lateral (90°) positions are recorded respectively.

[0077] Furthermore, it should be noted that if Figure 4 As shown, connect the optical device, ensure the normal transmission of device data, determine whether the surgical probe and the calibration target positioning mark are within the working range, use the collection file to distinguish the two optical marks, and bind the data transmitted by the optical positioning device to the corresponding data structure respectively. Import the probe, import the probe in the virtual space, update the position of the probe by binding the data transmitted by the optical positioning device to reflect the movement of the probe in the real space, and complete the needle tip calibration by sampling around the needle tip with the probe: find the coordinate transformation from the probe tip to the probe center, reflect the direction of the probe in the real space (probe center to needle tip) in the virtual space, and render the entire probe. At this time, the probe direction and movement in the virtual space (CT data space) can reflect the events in the real space, such as Figure 5 shown.

[0078] Furthermore, it should be noted that if Figure 6 As shown, capturing images of the calibration target in the anteroposterior and lateral positions refers to capturing X-ray images of the upper and lower plates of the calibration target in the anteroposterior and lateral positions, i.e., four X-ray images need to be captured, wherein the first X-ray image is an anteroposterior image containing only the lower calibration plate, the second X-ray image is an anteroposterior image containing both the upper and lower calibration plates, the third X-ray image is a lateral image containing both the upper and lower calibration plates, and the fourth X-ray image is a lateral image containing only the lower calibration plate. The purpose of this capture is to complete the separate capture of the image marking features of the upper and lower calibration plates in the anteroposterior and lateral positions without having to disassemble the calibration plates multiple times. The capture process is as follows:

[0079] 1) Install the lower calibration plate and capture the first X-ray image;

[0080] 2) Install the upper calibration plate and capture the second X-ray image;

[0081] 3) Rotate the C-arm 90° to acquire the third X-ray image;

[0082] 4) Remove the upper calibration plate and collect the fourth X-ray image;

[0083] 5) Restore the C-arm machine and remove the lower calibration plate.

[0084] S200, performing feature preprocessing based on the X-ray image with the calibration target marker to obtain two-dimensional X-ray image coordinate features and acquire a correction matrix for eliminating X-ray image distortion;

[0085] In this embodiment, the format of the two images with calibration target marks in the anteroposterior and lateral positions are first converted, and feature extraction and feature sorting operations are performed. The feature extraction is divided into the following steps: image enhancement, threshold segmentation, region of interest extraction, and minimum circumscribed circle extraction to obtain the marking features of the upper and lower plates of the calibration target. The extracted marking features are then sorted separately, and the X-ray image distortion correction matrix is ​​obtained by polynomial fitting, which can be used to correct the X-ray image to eliminate the distortion caused by the imaging principle. Finally, the two sets of rotation matrices obtained are combined with the mechanical dimensions of the calibration target to obtain the coordinates of the marking points on the calibration target in the anteroposterior and lateral positions in the coordinate system of the optical positioning device.

[0086] S210, performing format conversion preprocessing on the X-ray image with the calibration target mark to obtain a converted X-ray image with the calibration target mark;

[0087] Specifically, the X-ray image with calibration target marks is converted based on the SimpleITK library to obtain a SimpleITK format image with calibration target marks; the SimpleITK format image with calibration target marks is converted into a NumPy array to obtain the two-dimensional data of the X-ray image with calibration target marks; the two-dimensional data of the X-ray image with calibration target marks is normalized to obtain the normalized X-ray image with calibration target marks; the normalized X-ray image with calibration target marks is pixel expanded and stored to obtain the converted X-ray image with calibration target marks.

[0088] In some embodiments, such as Figure 7As shown, determine whether the file format is .dcm and use the SimpleITK library (sitK) in Python to read the DICOM file. Read the DICOM file from the specified path and convert it into a SimpleITK image object. Convert the SimpleITK image object to a NumPy array, obtain its shape information, and then resize it to a two-dimensional array. This is usually because the DICOM file may contain multiple slices, and only the image data of one slice is required here. Therefore, convert the three-dimensional array to a two-dimensional array, obtain the maximum and minimum values ​​of the image, and create an array containing the lower and upper limits of the window. Convert the maximum and minimum values ​​to floating-point numbers to ensure accuracy in subsequent calculations. Normalize the input DICOM image data. Map the grayscale values ​​of the image data to the range of 0 to 1. Normalization is achieved by subtracting the lower limit of the window from each pixel value and then dividing it by the window range, scaling the pixel values ​​to [0, 255]. The pixel values ​​of the normalized image data are expanded to the range of 0 to 255, and then the data type is converted to uint8 to meet the image storage format requirements. The processed file is stored in the specified directory. All files in the folder are traversed and the format conversion processing steps are repeated for the X-ray image with calibration target marks to obtain the processed image.

[0089] S220, performing feature extraction processing on the converted X-ray image with the calibration target mark to obtain features of the X-ray image with the calibration target mark;

[0090] Specifically, gamma enhancement processing is performed on the converted X-ray image with calibration target marks to obtain an enhanced X-ray image with calibration target marks; binary segmentation processing is performed on the enhanced X-ray image with calibration target marks to obtain a segmented X-ray image with calibration target marks; region of interest extraction processing is performed on the segmented X-ray image with calibration target marks to obtain an extracted X-ray image with calibration target marks; circular confidence calculation is performed on the extracted X-ray image with calibration target marks to obtain a circular confidence value of the X-ray image with calibration target marks; based on the circular confidence value of the X-ray image with calibration target marks, the minimum circumscribed circle of the shape is obtained and stored, and feature extraction processing is cyclically performed on the converted X-ray image with calibration target marks to obtain features of the X-ray image with calibration target marks.

[0091] In some embodiments, such as Figure 8 As shown in the figure, firstly, the image is gamma enhanced to increase the contrast of the image, the grayscale histogram of the image is calculated and the upper and lower bounds of the threshold segmentation are determined, the image is binary segmented, and then the region of interest is extracted from the image. The extraction results are shown in the figure. Figure 9As shown, the number of points of each shape in the image is counted. If the number is greater than a certain value, the shape is considered to be a polygon (including a circle). The contour area and perimeter of the shape are calculated, and the confidence level that the shape is a circle is calculated. The expression is:

[0092]

[0093] Among them, S is the area of ​​the shape, d is the perimeter of the shape, if the confidence level is within the confidence range, the shape is considered to be a circle, and it is added to the set A. The minimum circumscribed circle of the shape outline in the set Q is calculated, and the center coordinates and radius of the circle are obtained and stored in the set In the above example, the feature extraction process of the converted X-ray image with calibration target markers is repeated for the processed image, and the point set of the storage marker circumscribed circle features is A1, A2, A3, A4, as shown in Figure 10 shown.

[0094] S230, performing feature sorting processing on the X-ray image features with the calibration target mark to obtain sorted two-dimensional X-ray image features with the calibration target mark;

[0095] Specifically, the X-ray image features with calibration target marks are grouped and processed into feature point sets to obtain a point set containing marking points of the upper and lower marking plates in the upper orthogonal position, a point set containing marking points of the lower marking plate in the lateral position, and a point set containing marking points of the upper and lower marking plates in the lateral position; the point set containing marking points of the lower marking plate in the upper orthogonal position and the point set containing marking points of the upper and lower marking plates in the upper orthogonal position are judged by the center distance and classified to obtain a point set containing the upper marking in the upper orthogonal position; the point set containing marking points of the lower marking plate in the lateral position and the point set containing marking points of the upper and lower marking plates in the lateral position are judged by the center distance and classified to obtain a point set containing the upper marking in the lateral position; the point set containing marking points of the lower marking plate in the upper orthogonal position, the point set containing marking points of the lower marking plate in the lateral position, and the point set containing marking points of the upper and lower marking plates in the lateral position are classified according to the center coordinates, sorted and grouped according to the y coordinate from small to large, and the sorted two-dimensional X-ray image features with calibration target marks are output.

[0096] In some embodiments, such as Figure 11 As shown in FIG, the point set after feature extraction is divided into two groups: group a contains the point set A1 of the marking points of the lower marker plate in the anteroposterior position, and the point set A2 of the marking points of the upper and lower marker plates in the anteroposterior position; group b contains the point set A3 of the marking points of the lower marker plate in the lateral position, and the point set A4 of the marking points of the upper and lower marker plates in the lateral position.

[0097] Determine the distance between the centers of two point sets in the same group. If the distance between the centers is less than a specific value, then this circle is the common circle of the lower marker plate in the frontal position and the upper + lower marker plates in the frontal position, that is, the marker on the lower marker plate. If the distance between the centers is less than the specific value, then this circle is considered to be the marker of the upper plate, and it is added to the new point set B1. The same operation is performed on another group of point sets to obtain the new point set B2. The following four point sets are obtained: the lower frontal marker A1, the upper frontal marker B1, the lower lateral marker A3, and the upper lateral marker B2. Classify the four point sets according to the center coordinates, specifically: first, sort them into several groups according to the x coordinate, and then sort them in ascending order of the y coordinate within each group. The final obtained markers are grouped by column, and the y coordinate within the group is in ascending order. The expression is as follows:

[0098] C i ={c1,c2,...c n}, i = 1, 2, 3, 4

[0099] where c j ={((x1,y1)),((x2,y2)),((x3,y3)),...((x m ,y m ))|y1 < y2 <... < y m ,|x o -x p |< ε}, 0 < j ≤ n, and the result is as Figure 12 and Figure 13 shown.

[0100] S240. Establish an ideal image and perform point set matching on the features of the processed two-dimensional X-ray image with calibration target markers to obtain a correction matrix for eliminating X-ray image distortion.

[0101] Specifically, based on the upper and lower marker points in the features of the processed two-dimensional X-ray image with calibration target markers, construct a real point set; extract the center points of the real point set to generate a virtual undistorted point set; obtain the mapping relationship between the real point set and the virtual undistorted point set by the least squares method, and obtain a correction matrix for eliminating X-ray image distortion.

[0102] In some embodiments, as Figure 14 shown, use the upper + lower marker points as the real point set Extract the center points to generate the arrangement of the virtual undistorted point set Solve the mapping relationship from the real point set to the virtual point set by the least squares method. The obtained relationship is the correction matrix, and its principle is shown in the following formula:

[0103]

[0104] Where (p,q) is the coordinate of the undistorted point set, S is the correction parameter, and its expression is

[0105]

[0106] That is, the overdetermined equation is:

[0107]

[0108] Solve it using the least squares method, A is what we want to solve, and finally, if Figure 15 As shown in Figure 2, apply the correction matrix to the original image and observe the effect comparison.

[0109] S300. Obtain a conversion matrix between a two-dimensional X-ray feature image coordinate system and a real three-dimensional space coordinate system based on the two-dimensional X-ray image coordinate features, and complete the coordinate conversion and registration between the three-dimensional real space and the CT three-dimensional space by combining the registration method and correction matrix of the two-dimensional X-ray image and the CT image.

[0110] In this embodiment, a one-to-one correspondence is established between the obtained two-dimensional coordinates and the obtained three-dimensional coordinates, and the conversion matrix from the two-dimensional coordinates (X-ray image) to the three-dimensional coordinates (real space) is solved by the least squares method, and the calibration plate on the calibration target is unloaded. After unloading, intraoperative X-ray photography can be performed, and the lesion site can be photographed in the frontal and lateral positions. Combined with the two-dimensional and three-dimensional registration modules of CT images and X-ray images, the preoperative planning process can be completed: point planning is performed on the model rendered by CT data, and the point planning is mapped to the real space through: CT space->X-ray image->optical positioning device coordinate system; intraoperative effect visualization: intraoperative cutting and other operations on the lesion site can be mapped to the virtual space through: X-ray image->calibration target coordinate system->optical positioning device coordinate system->CT space coordinate system; postoperative effect evaluation: the data obtained through surgery by postoperative CT and preoperative CT are compared to evaluate the execution of the surgical plan and the surgical effect.

[0111] Specifically, based on the coordinate features of the two-dimensional X-ray image, a one-to-one correspondence between the two-dimensional coordinates of the features in the X-ray image and the three-dimensional coordinates of the features in the real space is established, and the conversion matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system is solved by the least squares method; the conversion matrix from the calibration target coordinate system to the optical positioning mark coordinate system on the side of the calibration target is determined by the mechanical size; the conversion matrix from the optical positioning mark coordinate system on the side of the calibration target to the optical positioning device coordinate system is determined by the optical positioning device; the conversion matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system, the conversion matrix from the calibration target coordinate system to the optical positioning mark coordinate system on the side of the calibration target, and the conversion matrix from the optical positioning mark coordinate system on the side of the calibration target to the optical positioning device coordinate system are superimposed to obtain the transformation matrix from the two-dimensional coordinates of the X-ray image to the coordinate system of the optical positioning device; based on the transformation matrix from the two-dimensional coordinates of the X-ray image to the coordinate system of the optical positioning device, combined with the registration method and correction matrix of the CT image and the X-ray image, the coordinate conversion and registration between the three-dimensional real space and the CT three-dimensional space is completed.

[0112] In some embodiments, such as Figure 16 As shown, the rotation and translation matrix T from the calibration target coordinate system to the optical positioning mark coordinate system on the side of the calibration target is determined by the mechanical dimensions. The coordinates of the mark points in the upper plate and the lower plate in the calibration target coordinate system can be calculated by the following method:

[0113] TD i ,i=1,2

[0114] Where D1 represents the set of marked points on the upper plate, and D2 represents the set of marked points on the lower plate.

[0115] Determine the transformation matrices T1 and T2 from the calibration target mark coordinate system to the optical positioning device coordinate system, where T1 is the transformation matrix when the optical positioning device is in the upright position, and T2 is the transformation matrix when the optical positioning device is in the side position. The coordinates of the mark points on the calibration target in the optical positioning system coordinate system can be obtained by coordinate superposition, specifically:

[0116] T1TD i ,i=1,2

[0117] T2TD i ,i=1,2

[0118] Determine the one-to-one correspondence between two-dimensional and three-dimensional point sets:

[0119] When upright: C i →T1TD i ,i=1,2;

[0120] Lateral position: C j →T2TD j ,j=1,2;

[0121] The least squares method is used to calculate the mapping relationship M1 from the two-dimensional image to the upper plate in the anteroposterior position, the mapping relationship M2 from the two-dimensional image to the lower plate in the anteroposterior position, the mapping relationship N1 from the two-dimensional image to the upper plate in the lateral position, and the mapping relationship N2 from the two-dimensional image to the lower plate in the lateral position. The expressions are:

[0122]

[0123] Based on the imaging principle of the C-arm X-ray machine, the mapping of the two-dimensional image coordinates of a certain point P during surgery to the three-dimensional coordinates (under the optical positioning system) can be calculated as follows: respectively calculate the intersection points of the X-ray passing through this point P with the upper and lower plates in the anteroposterior position, as well as the spatial equation of this ray; respectively calculate the intersection points of the X-ray image irradiating this point with the upper and lower plates in the lateral position, as well as the spatial equation of this ray. Solve for the point closest to these two straight lines, which is the coordinate of point P in space. For example, if the two-dimensional X-ray image coordinates of a certain point P in the anteroposterior and lateral positions under the C-arm X-ray machine are known to be (x1, y1) and (x2, y2), the three-dimensional coordinates of this point under the optical positioning device can be calculated in the following way:

[0124] When in the correct position, the intersection of the X-ray passing through this point and the upper and lower plates of the calibration target is:

[0125] (X1,Y1,Z1)=M1(x1,y1,1) T

[0126] (X2,Y2,Z2)=M2(x1,y1,1) T

[0127] The intersection of the X-ray ray passing through this point and the upper and lower plates of the calibration target in the lateral position is:

[0128] (X3,Y3,Z3)=N1(x2,y2,1) T

[0129] (X4, Y4, Z4) = N2(x2, y2, 1) T

[0130] The equations of the X-ray lines passing through this point in the anteroposterior and lateral positions are:

[0131]

[0132] tidy:

[0133]

[0134] The point to be sought is: the point closest to the two straight lines, that is, the distance between the two closest points on the two straight lines:

[0135]

[0136] Right now:

[0137]

[0138] At this point, the conversion of the two-dimensional coordinates of the X-ray image to the three-dimensional coordinates of the real space (optical navigation device) has been completed. Combined with the alignment of the CT image and the X-ray image, the alignment between the real space coordinate system and the virtual space coordinate system (CT space coordinate system) can be completed.

[0139] In summary, if Figure 3 As shown, the embodiments of the present invention leverage the advantages of CT image and X-ray image navigation systems by designing a calibration target mounted on a C-arm X-ray machine to implement a navigation system based on CT images and X-ray images. This invention achieves X-ray image correction and unifies the X-ray image coordinate system, the CT image coordinate system, and the optical positioning system coordinate system (real-space coordinate system). Preoperative registration enables surgical planning, intuitive reflection of intraoperative information, and analysis of the impact of surgical procedures on outcomes.

[0140] Therefore, compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0141] 1) Compared to traditional X-ray image-based navigation systems, this system incorporates preoperative CT 3D data, allowing the intraoperative results to be directly displayed on a 3D model. This allows doctors to intuitively observe the effects of the surgical process and analyze which steps in the procedure affect the outcome. This reduces the number of X-rays doctors need to take, minimizing their exposure to radiation.

[0142] 1) Compared with traditional navigation systems based on CT images, this invention combines intraoperative X-ray images to more effectively obtain real-time information during surgery, judge the actual effect of the surgery, and compare it with the preoperative plan to evaluate the real-time effect of the surgery. At the same time, it reduces the operating difficulty of the doctor and streamlines the surgical process.

[0143] 2) The present invention performs abnormality detection and judgment in the data acquisition stage after connecting the optical positioning device, which can ensure that abnormal data of the optical positioning device is eliminated without affecting the stability of the system.

[0144] 3) This invention distinguishes different optical markers through unique geometry files for each, and can update these files continuously, ensuring rapid and effective differentiation after data acquisition by the optical positioning device. During probe import, rendering, and real-time tracking, this invention integrates the probe tool's tip calibration function, which can find the coordinate transformation from the probe tip to the probe center and can be manually updated multiple times after import, ensuring real-time and accurate probe tracking before each surgery.

[0145] 4) The design of the calibration target accomplished two tasks: 1) conversion between the 2D coordinate system of intraoperative X-ray images and the 3D real-world coordinate system (optical positioning coordinate system); and 2) intraoperative X-ray image calibration to reduce surgical precision issues caused by image distortion due to the imaging principle of the C-arm X-ray machine. The calibration target underwent multiple design and field testing to ensure its compatibility with the C-arm X-ray machine structure. The dual-layer structure and the arrangement of markings on each layer were designed, and a corresponding algorithm was developed based on this design.

[0146] 5) The extraction of intraoperative information is completed by designing two layers of markers on the upper and lower layers of the calibration target and designing a corresponding automatic marker extraction algorithm. The automatic marker extraction algorithm includes steps such as image format conversion, image enhancement, binary segmentation, region of interest extraction, shape determination, and marker point organization. Corresponding response methods are provided for some abnormal situations that may occur, and the algorithm as a whole can accurately complete the information extraction requirements.

[0147] 6) The proposed method of converting 2D X-ray image coordinates to 3D real-world coordinates using a double-layer calibration target facilitates this conversion, bridging the gap between 2D and 3D space and providing a foundational technical foundation for surgical navigation. This method also significantly reduces the number of intraoperative X-ray exposures, lowering radiation exposure and potential surgical risks for on-site surgeons. This also reduces surgical duration and physician workload.

[0148] Reference Figure 2 , a registration system for CT images and X-ray images based on coordinate transformation, comprising:

[0149] The first module 201 is configured to perform rendering processing based on preoperative CT data to obtain an X-ray image with calibration target marks, wherein the X-ray image with calibration target marks includes an anteroposterior X-ray image with calibration target marks and a lateral X-ray image with calibration target marks;

[0150] The second module 202 is used to perform feature preprocessing based on the X-ray image with the calibration target mark to obtain the two-dimensional X-ray image coordinate features and obtain a correction matrix for eliminating the X-ray image distortion;

[0151] The third module 203 is used to obtain the transformation matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system based on the two-dimensional X-ray image coordinate features, and combine the registration method and correction matrix of the two-dimensional X-ray image and CT image to complete the coordinate transformation and registration between the three-dimensional real space and the CT three-dimensional space.

[0152] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0153] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A coordinate transformation registration method based on CT images and X-ray images, characterized in that: The following steps are involved: Performing rendering processing based on preoperative CT data to obtain X-ray images with calibration target marks, wherein the X-ray images with calibration target marks include an anteroposterior X-ray image with calibration target marks and a lateral X-ray image with calibration target marks; Performing format conversion preprocessing on the X-ray image with the calibration target mark to obtain a converted X-ray image with the calibration target mark; Performing feature extraction processing on the converted X-ray image with the calibration target mark to obtain features of the X-ray image with the calibration target mark; Performing feature sorting processing on the X-ray image features with the calibration target mark to obtain sorted two-dimensional X-ray image features with the calibration target mark; Based on the upper and lower marking plate marking points in the sorted two-dimensional X-ray image features with calibration target markings, a real point set is constructed; Extract the center point of the real point set to generate a virtual distortion-free point set; The mapping relationship between the real point set and the virtual distortion-free point set is obtained by the least square method, and the correction matrix for eliminating the distortion of the X-ray image is obtained; Based on the coordinate features of the two-dimensional X-ray image, a one-to-one correspondence between the two-dimensional coordinates of the features in the X-ray image and the three-dimensional coordinates of the features in the real space is established, and the transformation matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system is solved by the least squares method; Determine the conversion matrix from the calibration target coordinate system to the optical positioning mark coordinate system on the side of the calibration target through mechanical dimensions; Determine the conversion matrix from the optical positioning mark coordinate system on the side of the calibration target to the optical positioning device coordinate system by the optical positioning device; Superimpose the conversion matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system, the conversion matrix between the calibration target coordinate system and the optical positioning mark coordinate system on the side of the calibration target, and the conversion matrix between the optical positioning mark coordinate system on the side of the calibration target and the optical positioning device coordinate system to obtain the transformation matrix from the two-dimensional coordinates of the X-ray image to the optical positioning device coordinate system; Based on the transformation matrix from the two-dimensional coordinates of the X-ray image to the coordinate system of the optical positioning device, combined with the registration method and correction matrix of the CT image and X-ray image, the coordinate conversion and registration between the three-dimensional real space and the CT three-dimensional space is completed.

2. The coordinate transformation registration method based on CT images and X-ray images according to claim 1, characterized in that: The step of performing rendering processing based on the preoperative CT data to obtain an X-ray image with calibration target markers specifically includes: Turn on the C-arm X-ray machine and the optical positioning device, wherein a calibration target is installed on one side of the image intensification end of the C-arm X-ray machine, and the optical positioning mark of the optical positioning device is within a preset working range; Import preoperative CT data, render the 3D model using the VTK open source library algorithm, connect the optical positioning device so that the optical positioning markers can be detected, import the surgical probe for calibration, and install the calibration target; Based on the installed and fixed calibration target, the C-arm X-ray machine is used to shoot and obtain an X-ray image with calibration target marks.

3. The coordinate transformation registration method based on CT images and X-ray images according to claim 2, characterized in that: The step of performing format conversion preprocessing on the X-ray image with the calibration target mark to obtain the converted X-ray image with the calibration target mark specifically includes: Based on the SimpleITK library, the X-ray image with calibration target marks is converted to obtain a SimpleITK format image with calibration target marks; Convert the SimpleITK format image with calibration target marks into a NumPy array to obtain the two-dimensional data of the X-ray image with calibration target marks; Normalizing the two-dimensional data of the X-ray image with the calibration target mark to obtain a normalized X-ray image with the calibration target mark; The normalized X-ray image with the calibration target mark is subjected to pixel expansion and storage processing to obtain a converted X-ray image with the calibration target mark.

4. The coordinate transformation registration method based on CT images and X-ray images according to claim 3, characterized in that: The step of performing feature extraction processing on the converted X-ray image with the calibration target mark to obtain the features of the X-ray image with the calibration target mark specifically includes: performing gamma enhancement processing on the converted X-ray image with the calibration target mark to obtain an enhanced X-ray image with the calibration target mark; performing binary segmentation processing on the enhanced X-ray image with the calibration target mark to obtain a segmented X-ray image with the calibration target mark; Performing region of interest extraction processing on the segmented X-ray image with the calibration target mark to obtain an extracted X-ray image with the calibration target mark; Performing circular confidence calculation on the extracted X-ray image with the calibration target mark to obtain a circular confidence value of the X-ray image with the calibration target mark; Based on the circular confidence value of the X-ray image with the calibration target mark, the minimum circumscribed circle of the shape is obtained and stored, and the feature extraction processing is cyclically performed on the converted X-ray image with the calibration target mark to obtain the X-ray image features with the calibration target mark.

5. The coordinate transformation registration method based on CT images and X-ray images according to claim 4, characterized in that: The step of performing feature sorting processing on the X-ray image features with the calibration target marks to obtain sorted two-dimensional X-ray image features with the calibration target marks specifically includes: Perform feature point set grouping processing on the X-ray image features with calibration target markers to obtain a point set containing marker points of the anteroposterior lower marker plate, a point set containing marker points of the anteroposterior upper and lower marker plates, a point set containing marker points of the lateral lower marker plate, and a point set containing marker points of the lateral upper and lower marker plates; The point set containing the mark points of the correct lower mark plate and the point set containing the mark points of the correct upper and lower mark plates are judged by the center distance and classified to obtain the point set containing the correct upper mark; The point set containing the marking points of the lateral lower marking plate and the point set containing the marking points of the lateral upper and lower marking plates are judged by the center distance and classified to obtain the point set containing the lateral upper marking; The point set containing the marking points of the anteroposterior lower marking plate, the point set containing the anteroposterior upper marking, the point set containing the marking points of the lateral lower marking plate, and the point set containing the lateral upper marking are classified according to the center coordinates, sorted and grouped from small to large according to the y coordinate, and the sorted two-dimensional X-ray image features with calibration target marks are output.

6. A registration system for CT images and X-ray images based on coordinate transformation, characterized in that: Includes the following modules: The first module is configured to perform rendering processing based on preoperative CT data to obtain an X-ray image with calibration target marks, wherein the X-ray image with calibration target marks includes an anteroposterior X-ray image with calibration target marks and a lateral X-ray image with calibration target marks; The second module is used to perform format conversion preprocessing on the X-ray image with the calibration target mark to obtain a converted X-ray image with the calibration target mark; Performing feature extraction processing on the converted X-ray image with the calibration target mark to obtain features of the X-ray image with the calibration target mark; Performing feature sorting processing on the X-ray image features with the calibration target mark to obtain sorted two-dimensional X-ray image features with the calibration target mark; Based on the upper and lower marking plate marking points in the sorted two-dimensional X-ray image features with calibration target markings, a real point set is constructed; Extract the center point of the real point set to generate a virtual distortion-free point set; The mapping relationship between the real point set and the virtual distortion-free point set is obtained by the least square method, and the correction matrix for eliminating the distortion of the X-ray image is obtained; The third module is used to establish a one-to-one correspondence between the two-dimensional coordinates of the features in the X-ray image and the three-dimensional coordinates of the features in the real space based on the two-dimensional X-ray image coordinate features, and solve the transformation matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system through the least squares method; Determine the conversion matrix from the calibration target coordinate system to the optical positioning mark coordinate system on the side of the calibration target through mechanical dimensions; Determine the conversion matrix from the optical positioning mark coordinate system on the side of the calibration target to the optical positioning device coordinate system by the optical positioning device; Superimpose the conversion matrix between the two-dimensional X-ray feature image coordinate system and the real three-dimensional space coordinate system, the conversion matrix between the calibration target coordinate system and the optical positioning mark coordinate system on the side of the calibration target, and the conversion matrix between the optical positioning mark coordinate system on the side of the calibration target and the optical positioning device coordinate system to obtain the transformation matrix from the two-dimensional coordinates of the X-ray image to the optical positioning device coordinate system; Based on the transformation matrix from the two-dimensional coordinates of the X-ray image to the coordinate system of the optical positioning device, combined with the registration method and correction matrix of the CT image and X-ray image, the coordinate conversion and registration between the three-dimensional real space and the CT three-dimensional space is completed.

Images