Method and navigation system for registering a two-dimensional image dataset with a three-dimensional image dataset of a region of interest - Patents.com

Abstract

A method for registering a two-dimensional image dataset and a three-dimensional image dataset of a region of interest is provided. [Solution] The method includes a step of defining a first vector of a registered virtual camera in the coordinate system of a three-dimensional image dataset; a step of transforming the first vector of the registered virtual camera by at least one transformation matrix to obtain a first transformation vector of an unregistered virtual camera in the coordinate system of the three-dimensional image dataset; a step of defining a focal point of the unregistered virtual camera in the coordinate system of the three-dimensional image dataset to be located at a reference point of the two-dimensional image dataset; and a step of generating an updated reconstructed image after repositioning the unregistered virtual camera by repositioning the unregistered virtual camera based on the first transformation vector and the focal point of the unregistered virtual camera.

Description

[Technical Field]

[0001] This application was filed as a PCT international patent application on November 18, 2022, and claims priority to and the benefit of U.S. Provisional Application No. 63 / 264250, filed November 18, 2021, and U.S. Non-Provisional Application Nos. 17 / 856158, 17 / 856289, 17 / 856297, and 17 / 856301, filed July 1, 2022, the disclosures of which are incorporated herein in their entireties. The present application relates to a method and a navigation system for establishing a relationship between two-dimensional image data and three-dimensional image data corresponding to a region of interest, and in particular to a method for establishing a relationship between a two-dimensional image dataset and a three-dimensional image dataset of a region of interest. Alignment The present invention relates to a method and a navigation system. [Background technology]

[0002] Currently, the incidence of spine-related diseases is increasing day by day, seriously threatening people's health. Spinal surgery is the main method of treating spinal diseases, and how to reduce surgical scars is increasingly important, thereby reducing the probability of infection and enabling patients to recover quickly without needing hospitalization. To achieve this goal, navigation systems with image-navigated surgical procedures have been introduced and used in spinal surgery. In image-navigated surgical procedures, an image of the patient's region of interest is displayed on the navigation system's display. At the same time, the navigation system can track surgical instruments and / or implants and import their simulated images into the image of the patient's region of interest. The advantages of the image-navigated surgical procedure and navigation system described above allow the surgeon to know the position of the surgical instruments and / or implants relative to the surgical target, without the need to frequently use C-arm fluoroscopy to know the relative positions throughout the surgical procedure, and the above technology is mostly disclosed in U.S. Patent No. "US6,470,207," entitled "Navigational Guidance Via Computer-Assisted Fluoroscopic Imaging," published on October 22, 2002. The entirety of the above-listed patent documents is incorporated herein by reference.

[0003] However, the spinal navigation system typically requires the use of X-ray detection to obtain the internal structure of the patient's body, which exposes the patient to the risk of radiation. This shows that the conventional method has obvious inconveniences and deficiencies and needs improvement. To solve the above problem, the related field has been struggling to find a solution, but no suitable solution has been developed for a long time.

[0004] Two-dimensional (2D) and three-dimensional (3D) Alignment There are many methods that can realize this, for example, the well-known two-dimensional image and three-dimensional image AlignmentTwo-dimensional and three-dimensional images are Alignment The method is a contour algorithm, point Alignment (point registration) algorithm, surface Alignment Surface registration algorithms, density comparison algorithms and pattern intensity Alignment (pattern intensity registration) algorithm. Alignment Since both methods require a large amount of calculation, they are usually used in two-dimensional and three-dimensional Alignment It takes several minutes to complete. Alignment The time required for this is 20 minutes to 1 hour. Alignment The procedure is as follows: Alignment may be inaccurate.

[0005] As described above, the above conventional methods have obvious inconveniences and deficiencies and need to be improved. Alignment It is also necessary to develop methods and devices to complete the two-dimensional and three-dimensional Alignment The flow should also be improved.

[0006] Furthermore, two-dimensional images (e.g., X-ray images) usually cover multiple layers of vertebrae. Matching between two-dimensional images and digitally reconstructed radiographs (DRRs) generated by computed tomography (CT) data is not only inefficient but also prone to interference between regions of interest. Alignment It is prone to failure, errors and inaccuracies. The interferences include interferences from pedicle screws, vertebral cages, ribs and hip bones. The specialized algorithm focuses on comparison between multiple partitions rather than overall matching, thus contributing to comparative analysis on anatomical images. 2D and 3D Alignment When executing, two-dimensional and three-dimensional AlignmentThe failure rate is still more than 20%. Therefore, rather than rerunning the entire process, we used some of the results to calculate the 2D and 3D. Alignment Efficiently completing this is a goal that those skilled in the art are working towards. Summary of the Invention [Problem to be solved by the invention]

[0007] The present disclosure is intended to provide a simplified summary of the present disclosure so that readers can have a basic understanding of the present disclosure. This disclosure is not an exhaustive overview of the present disclosure, and is not intended to point out key / core elements of embodiments of the present invention or to define the scope of the present invention. [Means for solving the problem]

[0008] One aspect of the present invention is to provide a two-dimensional image dataset of a region of interest and a three-dimensional image dataset of the region of interest. Alignment generating a first reconstructed image from the three-dimensional image dataset based on a first spatial parameter; calculating a reference similarity number based on the first reconstructed image and the two-dimensional image dataset; generating a second reconstructed image from the three-dimensional image dataset based on a second spatial parameter; calculating a comparative similarity number based on the second reconstructed image and the two-dimensional image dataset; comparing the comparative similarity number with the reference similarity number; and associating the two-dimensional image dataset with the three-dimensional image dataset if the comparative similarity number is less than or equal to the reference similarity number. Alignment death, Alignment and utilizing the two-dimensional image dataset and the three-dimensional image dataset for computer-assisted surgical navigation. Alignment Regarding how to do it.

[0009] Another aspect of the present invention is to provide a method for generating a two-dimensional image dataset of a region of interest and a three-dimensional image dataset of the region of interest. Alignment a memory for storing a plurality of instructions; and retrieving the plurality of instructions by the memory, the memory comprising the steps of: generating a first reconstructed image from the three-dimensional image dataset based on a first spatial parameter; calculating a reference similarity number based on the first reconstructed image and the two-dimensional image dataset; generating a second reconstructed image from the three-dimensional image dataset based on a second spatial parameter; calculating a comparative similarity number based on the second reconstructed image and the two-dimensional image dataset; comparing the comparative similarity number with the reference similarity number; and associating the two-dimensional image dataset with the three-dimensional image dataset if the comparative similarity number is less than or equal to the reference similarity number. Alignment death, Alignment and utilizing the generated two-dimensional image dataset and the three-dimensional image dataset for computer-assisted surgical navigation.

[0010] Another aspect of the present invention is to combine a two-dimensional image dataset and a three-dimensional image dataset of a region of interest. Alignmenta first virtual camera based on a distance parameter calculated based on the two-dimensional image dataset and the region of interest; a second vector calculated from a first spatial mark in the three-dimensional image dataset and a second vector calculated from a first planar mark in the two-dimensional image dataset; and a second vector calculated from a first planar mark in the two-dimensional image dataset. The first virtual camera is rotated based on an angle, the angle corresponding to an angle difference between a reconstructed image having a maximum similarity parameter and the two-dimensional image dataset, the reconstructed image having the maximum similarity parameter having a maximum similarity parameter among a plurality of similarity parameters calculated for a plurality of reconstructed images in the three-dimensional image dataset, the plurality of reconstructed images including a reconstructed image generated by the first virtual camera and another reconstructed image generated by another virtual camera, the other reconstructed image and the reconstructed image generated by the first virtual camera having different angles or pixels. After the adjustment and rotation, the method is used in a navigation system to display the two-dimensional image dataset and the three-dimensional image dataset. Alignment and

[0011] Another aspect of the present invention is to combine a two-dimensional image dataset and a three-dimensional image dataset of a region of interest. Alignmenta navigation system for navigating a three-dimensional image dataset, the navigation system comprising: a memory for storing a plurality of instructions; and retrieving the plurality of instructions by the memory, the navigation system comprising the steps of: adjusting a first virtual camera based on distance parameters calculated based on the two-dimensional image dataset and the region of interest; rotating the first virtual camera based on an angle difference between a first vector calculated from two spatial marks in the three-dimensional image dataset and a second vector calculated from two first planar marks in the two-dimensional image dataset; and rotating the first virtual camera based on an angle, the angle corresponding to an angle difference between a reconstructed image having a maximum similarity parameter and the two-dimensional image dataset, the reconstructed image having the maximum similarity parameter having a maximum similarity parameter among a plurality of similarity parameters calculated for a plurality of reconstructed images in the three-dimensional image dataset, the plurality of reconstructed images including a reconstructed image generated by the first virtual camera and another reconstructed image generated by another virtual camera, the other reconstructed image and the reconstructed image generated by the first virtual camera having different angles or pixels; and Alignment and a processor for performing the steps of:

[0012] One aspect of the present invention is to provide a method for obtaining a two-dimensional image data set and a three-dimensional image data set of a region of interest. Alignment 1. A method of imaging a three-dimensional image dataset, comprising: Alignment defining a first vector of the virtual camera by at least one transformation matrix; Alignment Transform the first vector of the virtual camera into the coordinate system of the three-dimensional image dataset: Not aligned obtaining a first transformation vector of the virtual camera, in a coordinate system of the three-dimensional image dataset; Not aligned defining a focal point of a virtual camera to be located at a reference point of the two-dimensional image data set; Not aligned Based on the focus of the virtual camera, Not aligned By repositioning the virtual camera, Not aligned and generating an updated reconstructed image after repositioning the virtual camera.

[0013] One aspect of the present invention is to provide a method for obtaining a two-dimensional image data set and a three-dimensional image data set of a region of interest. Alignment 1. A navigation system for performing a navigation process, comprising: a memory for storing a plurality of instructions; and retrieving the plurality of instructions by the memory to perform, in a coordinate system of a three-dimensional image data set: Alignment defining a first vector of the virtual camera by at least one transformation matrix; Alignment Transform the first vector of the virtual camera into the coordinate system of the three-dimensional image dataset: Not aligned obtaining a first transformation vector of the virtual camera, in a coordinate system of the three-dimensional image dataset; Not aligned defining a focal point of a virtual camera to be located at a reference point of the two-dimensional image data set; Not aligned Based on the focus of the virtual camera, Not aligned By repositioning the virtual camera, Not aligned and generating an updated reconstructed image after repositioning the virtual camera.

[0014] One aspect of the present invention is to provide a method for generating a two-dimensional image dataset and a three-dimensional image dataset of a region of interest. Alignment A method for manufacturing a semiconductor device, comprising: Alignment obtaining first spatial parameters of the virtual camera, Alignment a preset virtual camera position corresponding to a first two-dimensional image of the two-dimensional image data set; Alignment The first spatial parameter of the virtual camera is Not aligned A second spatial parameter of the virtual camera is adjusted to Not aligned generating an updated reconstructed image by repositioning the virtual camera, Not aligneda setting position of a virtual camera does not correspond to the first two-dimensional image of the two-dimensional image dataset.

[0015] One aspect of the present invention is to provide a method for generating a two-dimensional image dataset and a three-dimensional image dataset of a region of interest. Alignment a memory for storing a plurality of instructions; and a first navigation system for acquiring the plurality of instructions by the memory. Alignment obtaining first spatial parameters of the virtual camera, Alignment a preset virtual camera position corresponding to a first two-dimensional image of the two-dimensional image data set; Alignment The first spatial parameter of the virtual camera is Not aligned A second spatial parameter of the virtual camera is adjusted to Not aligned generating an updated reconstructed image by repositioning the virtual camera, Not aligned and a processor for performing the steps of: a virtual camera setting position not corresponding to the first two-dimensional image of the two-dimensional image dataset. [Effects of the Invention]

[0016] Therefore, based on the technical contents of the present application, the method and navigation system shown in the embodiments of the present application can be used for both ordinary two-dimensional and three-dimensional Alignment If the procedure fails, a re-registration procedure is offered. The re-registration procedure is a continuation of the obtained results, so that both 2D and 3D Alignment The procedure is based on the existing results above. Alignment The method and navigation system of the present application allows the user to continue completing the procedure rather than re-running the entire flow. Alignment The procedure can be completed more quickly.

[0017] After referring to the following embodiments, those skilled in the art can easily understand the basic spirit and other inventive objects of the present application, as well as the technical means and embodiments adopted by the present application. [Brief explanation of the drawings]

[0018] To make the above and other objects, features, advantages and embodiments of the present application more clearly and comprehensibly, reference is made to the accompanying drawings as follows: [Figure 1] 1 is a schematic diagram of a navigation system according to an embodiment of the present application; [Figure 2] 2 is a schematic diagram of a computing device, a database, and an optical tracking device of the navigation system shown in FIG. 1 according to one embodiment of the present application. [Figure 3] 1 is a flowchart of a method for registering a two-dimensional image dataset and a three-dimensional image dataset of a region of interest according to an embodiment of the present application; [Figure 4] 1 is a schematic diagram of the contents of a portion of a region of interest according to an embodiment of the present application; [Figure 5] 1 is a schematic diagram of the contents of a portion of a region of interest according to an embodiment of the present application; [Figure 6] 1 is a schematic diagram of the contents of a portion of a region of interest according to an embodiment of the present application; [Figure 7] 1 is a schematic diagram of the contents of a portion of a region of interest according to an embodiment of the present application; [Figure 8] 2 is a schematic diagram illustrating the operation of the C-arm device shown in FIG. 1 according to one embodiment of the present application. [Figure 9] 2 is a schematic diagram illustrating the operation of the C-arm device shown in FIG. 1 according to one embodiment of the present application. [Figure 10] 1 is a flowchart of a method for registering a two-dimensional image dataset and a three-dimensional image dataset of a region of interest according to an embodiment of the present application; [Figure 11] 1 is a flowchart of a method for registering a two-dimensional image dataset and a three-dimensional image dataset of a region of interest according to an embodiment of the present application; [Figure 12] 1 is a flowchart of a method for registering a two-dimensional image dataset and a three-dimensional image dataset of a region of interest according to an embodiment of the present application; [Figure 13]1 is a flowchart of a method for registering a two-dimensional image dataset and a three-dimensional image dataset of a region of interest according to an embodiment of the present application; [Figure 14] 2 is a schematic diagram of a virtual camera simulated by a computing device of the navigation system shown in FIG. 1 according to one embodiment of the present application; [Figure 15] 1 is a flowchart of a method for registering a two-dimensional image dataset and a three-dimensional image dataset of a region of interest according to an embodiment of the present application; [Figure 16] 1 is a flowchart of a method for registering a two-dimensional image dataset and a three-dimensional image dataset of a region of interest according to one embodiment of the present application. According to common practice, the various features and elements in the drawings are not drawn to scale, and the drawing style is used to best present the specific features and elements relevant to the present application. Furthermore, the same or similar element symbols are used in different drawings to refer to similar elements / components. DETAILED DESCRIPTION OF THE INVENTION

[0019] To provide a more detailed and complete description of the present disclosure, the following provides illustrative descriptions of embodiments and specific examples of the present disclosure. However, these are not the only ways to implement or operate the specific examples of the present disclosure. The embodiments include features of multiple specific examples and method steps and sequences for constructing and operating these specific examples. However, other specific examples may be used to achieve the same or equivalent functions and sequence of steps.

[0020] Unless otherwise defined herein, scientific and technical terms used herein have the same meaning as understood and commonly used by those skilled in the art. Also, unless conflicting with the context, singular nouns used herein cover the plural of that noun. The use of plural nouns also covers the singular of that noun.

[0021] 1 is a schematic diagram of a navigation system 1000 having a computing device 1100, a database 1200, and an optical tracking device 1400 according to one embodiment of the present application. The navigation system 1000 can be used to navigate an instrument 9500 during a surgical operation of a target 9000, but the present application is not limited thereto. In some embodiments, the navigation system 1000 can navigate any kind of instrument during any type of operation according to actual needs. For example, the navigation system 1000 can be used to navigate screws, graspers, clamps, surgical scissors, cutters, needle drivers, retractors, distractors, dilators, suction tips and tubes, sealing devices, irrigation and injection needles, powered devices, scopes and probes, carriers and appliers, ultrasonic tissue disruptors, cryotomes and cutting laser guides, and measurement devices. The navigation system 1000 can similarly be used to navigate any type of implant, including orthopedic implants, spinal implants, cardiovascular implants, neurovascular implants, soft tissue implants, or other implants that can be implanted within a target 9000 (e.g., a patient).

[0022] The computing device 1100 may be, but is not limited to, a computer, a desktop computer, a laptop computer, a tablet computer, or other device capable of performing computing functions. The database 1200 may be, but is not limited to, a data storage device, a computer, a server, a cloud storage device, or other device capable of storing data. The database 1200 is used to store a pre-acquired three-dimensional (3D) image dataset of the target 9000. The 3D image dataset may be acquired by imaging the target 9000 using computed tomography (CT).

[0023] The imaging device 1300 may be established and executed by the navigation system 1000 and may be, but is not limited to, a C-arm mobile fluoroscopy machine or a mobile X-ray image intensifier. The imaging device 1300 includes an X-ray emitter 1310 and an X-ray receiver 1320. The former emits X-rays that pass through the torso of the target 9000, and the latter receives the X-rays and converts them into a digital two-dimensional (2D) image that is displayed on the display of the imaging device 1300. Based on the plane along which the image is acquired along the imaging device 1300, the two-dimensional image may be an anterior / posterior (AP) two-dimensional image and a lateral (LA) two-dimensional image of the target 9000. The target 9000 may be, but is not limited to, a patient.

[0024] During the image acquisition process, a calibrator 1510 having calibration marks is used to calibrate the imaging device 1300 and track its position. The calibrator 1510 is located within or on the X-ray receiving device 1320, and dynamic reference frames 1520A, 1520B are placed at or adjacent to the regions of interest of the X-ray receiving device 1320 and the target 9000. The calibrator 1510 is located on the path from the X-ray emitting device 1310 to the X-ray receiving device 1320 and is not transparent or semi-transparent to X-rays. Therefore, after X-rays enter the target 9000 and the calibrator 1510, they are partially absorbed by specific tissues and the calibration marks in the calibrator 1510 before being received by the X-ray receiving device 1320. In this manner, the two-dimensional image can display the region of interest of the target 9000 and the calibration marks of the calibrator 1510. The computing device 1100 calculates the relationship between the X-ray emitting device 1310 and the X-ray receiving device 1320 based on the pattern of the calibration marks displayed in the two-dimensional image. Both the AP two-dimensional image and the LA two-dimensional image of the vertebrae of interest of the target 9000 are included in the two-dimensional image dataset shown in the embodiment of the present application. Note that in other embodiments, only the AP two-dimensional image or the LA two-dimensional image may be included in the two-dimensional image dataset. The two-dimensional image dataset can also be transferred to the computing device 1100 as an electronic file for subsequent use.

[0025] The three-dimensional image dataset in the database 1200, the two-dimensional image dataset generated by the imaging device 1300, the optical tracking device 1400, the calibrator 1510, the dynamic reference coordinates 1520A and 1520B, and the target 9000 each have coordinates in a different coordinate system. The navigation system 1000 of the present application can establish a relationship between these coordinate systems so that the surgeon can use the navigation system 1000 to navigate the instrument 9500 during a surgical procedure. The relationship between the above coordinates will be described below. The calibrator 1510 and the dynamic reference coordinates 1520A are mounted on the X-ray receiving device 1320, and the positions of these three are considered to be substantially the same, so a relationship between the calibrator coordinate system and the imaging device coordinate system is established. The optical tracker 1400 then tracks the dynamic reference coordinate 1520A including the reflector to introduce the calibrator 1510, the dynamic reference coordinate 1520A, and the X-ray receiving device 1320 into the tracker coordinate system, thereby obtaining the position of the X-ray receiving device 1320 in the tracker coordinate system. In some embodiments, the optical tracker 1400 can track the calibrator 1530 including the reflector to introduce the calibrator 1530 into the tracker coordinate system, thereby obtaining the position of the calibrator 1530 in the tracker coordinate system. The relationship between the X-ray emitting device 1310 and the X-ray receiving device 1320 can be obtained as described above, and the position of the X-ray emitting device 1310 in the tracker coordinate system can be obtained. It should be noted that the present application is not limited to the embodiment shown in FIG. 1 , which is used only to illustratively illustrate one implementation of the present application.

[0026] It should be noted that the navigation system 1000 Alignment The procedure includes an initialization and calibration procedure, two-dimensional and three-dimensional Alignment The program is roughly composed of three procedures: an initialization calibration procedure, a 2D and 3D calibration procedure, and a recalibration procedure. In some embodiments, the program installed in the navigation system 1000 executes the above three procedures in detail, as will be described later. Note that in other embodiments, the program executes the initialization calibration procedure, the 2D and 3D calibration procedure, and the recalibration procedure depending on the current situation or the required functions. Alignment Procedure, again Alignment The procedures can be executed individually or in combination of the above procedures. Other details will be described later.

[0027] Initial alignment

[0028] Before using the navigation system 1000, the initialization alignment procedure is executed in the navigation system 1000 to further improve efficiency and enhance the accuracy of the navigation system 1000. Other details will be described later.

[0029] FIG. 2 is a schematic diagram of the computing device 1100, the database 1200, and the optical tracking device 1400 of the navigation system 1000 shown in FIG. 1 according to an embodiment of the present application. As shown in the figure, the computing device 1100 is electrically connected to the database 1200 and the imaging device 1300.

[0030] The computing device 1100 includes a memory 1110, a processor 1120, and an I / O interface 1130. The processor 1120 is electrically connected to the memory 1110 and the I / O interface 1130. The memory 1110 is used to store a plurality of instructions, and the processor 1120 executes the steps of the method 2000 by acquiring the plurality of instructions to obtain the two-dimensional image data set and the three-dimensional image data set of the region of interest shown in FIG. 3. Alignment to perform.

[0031] In some embodiments, memory 1110 may include, but is not limited to, at least one of a flash memory, a hard disk drive (HDD), a solid-state drive (SSD), a dynamic random access memory (DRAM), and a static random access memory (SRAM), or a combination of these elements. In some embodiments, memory 1110 may be used as a non-transitory computer-readable storage medium to store computer-readable instructions executable by processor 1120.

[0032] In some embodiments, processor 1120 may include, but is not limited to, a single processor or a multi-microprocessor integrated device, such as a central processing unit (CPU), a graphics processing unit (GPU), or an application-specific integrated circuit (ASIC).

[0033] See also Figures 2 and 3. In operation, processor 1120 executes step 2100 to adjust a first virtual camera based on the two-dimensional image dataset and distance parameters calculated corresponding to the region of interest.

[0034] For example, the processor 1120 of the computing device 1100 may acquire X-ray images directly using the imaging device 1300, or may acquire X-ray images previously stored in the database 1200. As described above, the two-dimensional data set includes at least an AP two-dimensional image and an LA two-dimensional image, both of which include the calibrator 1510 of FIG. 1 . The calibrator 1510 can be used to identify the position of the imaging device 1300 in the above-described calculation relationship. Once the position of the imaging device 1300 is obtained, the processor 1120 can estimate the distance between the X-ray emitter 1310 and the region of interest of the target vertebral segment in the target 9000. The estimated distance is then converted into a distance parameter by the processor 1120 of the computing device 1100.

[0035] As described above, the initialization alignment 2000 of the present embodiment has been outlined. Briefly, the initialization alignment 2000 may include three steps 2100, 2200, and 2300.

[0036] In the first step, the surgeon acquires a first or AP image and a second or LA image using the imaging device 1300. These images can be used to aid in system refinement and navigation, as well as to initialize the orientation. Because the imaging device 1300 includes a calibrator 1510, the position of the X-ray emitter 1310 (e.g., the patient's AP direction and the patient's LA direction) is known during the image acquisition procedure relative to the patient coordinate system. This information is combined with information about how the patient 9000 will be positioned during the three-dimensional acquisition, as configured by the user. After the orientation is configured, a digitally reconstructed radiograph (DRR) can be created from the three-dimensional data set acquired by computed tomography (CT). Because the digitally reconstructed radiograph (DRR) is also a digital image, it can be used to interpret how and where the virtual camera generated these digital images. In other words, the AP and LA digitally reconstructed radiographs (DRR) are expected to be generated by simulating the capture of images by two virtual cameras, for example, by placing the two virtual cameras at specific points along the AP and LA planes of the patient 9000, respectively, in a direction toward the vertebrae of interest of the patient 9000. The digitally reconstructed radiographs (DRR) correspond to actual AP and LA radiographs. The digitally reconstructed radiographs (DRR) and the actual radiographs are displayed to the surgeon, who identifies the same points in all images. This process provides the same points or positions in the CT image data and the dynamic reference coordinates 1520A or patient space coordinates. Once the position, orientation, and dynamic reference coordinates of the patient 9000 in the three-dimensional dataset are known, the method continues with the subsequent refinement process. After the refinement process, all related coordinate systems can be connected and calculated.

[0037] During the initialization process, the surgeon is required to obtain AP and LA radiographs, or radiographs along any two planes with the imaging device 1300. In spine surgery, the surgeon is instructed to point to the center of the vertebral body of interest. Combining all of the above information, a good orientation and position assessment is obtained, and this assessment is used to generate a Digitally Reconstructed Radiograph (DRR). Such a Digitally Reconstructed Radiograph (DRR) has a high degree of correspondence with the actual radiograph and can be used to visualize the radiograph in both two and three dimensions. Alignment The digitally reconstructed radiograph (DRR) was calculated by combining three-dimensional data acquired by computed tomography (CT) and data acquired by a local target acquired by a C-arm.

[0038] The processor 1120 of the computing device 1100 can access the 3D image dataset stored in the database 1200, and an AP virtual camera or an LA virtual camera corresponding to the 3D image dataset can be created by the processor 1120. Each virtual camera has its own spatial parameters that mark the location and orientation of the feature of interest in its corresponding 3D image. Like a real camera, the position and orientation of the camera relative to the target object determine the image content captured. Therefore, changing the spatial parameters of the virtual camera generates different reconstructed images in the 3D image dataset. In this embodiment, the spatial parameters include at least one distance parameter indicating the distance from the target feature of interest in the target 9000 to the virtual camera.

[0039] initial Alignment

[0040] initial Alignment In the process, the software includes the initial 2D and 3D Alignment The software design includes a matching algorithm for reprocessing the digital radiograph (DRR) and the actual radiograph by adjusting the initialization position and orientation. AlignmentUsed to improve the overall flow. Alignment In this process, a similarity or cost measurement method can be adopted to identify the degree of image matching. The initial inversion algorithm maximizes the similarity between the digitally reconstructed radiograph (DRR) and the actual radiograph by simultaneously adjusting the initial position and orientation parameters. The calculation of the similarity or cost can be selected from known similarity and cost calculation methods, such as normalized mutual information, mutual information, gradient difference algorithms, surface contour algorithms, pattern intensity algorithms, sum of squared differences, normalized cross-correlation, local normalized correlation, and correlation ratio. By performing this procedure, two-dimensional and three-dimensional images can be obtained. Alignment The whole can be made efficient and accurate.

[0041] The processor 1120 of the computing device 1100 can then adjust the AP virtual camera or the LA virtual camera based on the distance parameter.

[0042] In some embodiments, the distance parameter is calculated based on the actual distance between two planar marks in an image of the two-dimensional image dataset. For example, a surgeon can calculate the distance parameter by manually marking two markers on a single AP or LA two-dimensional image of interest.

[0043] 4, the target 9000 may be a patient with a spinal injury, and a surgeon may manually mark two marks V0 and V1 at the centers of two target vertebrae of the target 9000 through the I / O interface 1130 of FIG. 2, but the present application is not limited thereto. In this manner, the distance between the marks V0 and V1 may be calculated by the processor 1120 of the computing device 1100 using image recognition techniques. In some embodiments, the I / O interface 1130 may be electrically connected to any type of input device, such as a mouse, a keyboard, or a touch panel.

[0044] 5, this is an AP digitally reconstructed radiographic image (DRR) generated by the processor 1120 using input data. The input data includes position or orientation information of the imaging device 1300 relative to the target 9000 when capturing the X-ray image.

[0045] In some embodiments, the virtual camera can be used in two-dimensional and three-dimensional Alignment Modular coding in the program, which is used for parameter-driven position and parameter-driven orientation or perspective, causes the processor 1120 to compute a known digital reconstruction radiography (DRR) algorithm, thereby generating that two-dimensional image for a particular three-dimensional volume.

[0046] Using this orientation estimation method, a digitally reconstructed radiograph (DRR) generated based on the computed tomography (CT) is established ( FIG. 5 ), which substantially corresponds to the actual AP radiograph of FIG. 4 . Using patient orientation information from the three-dimensional image data and the patient orientation information at the position of the imaging device 1300, the information is combined with a well-known digitally reconstructed radiograph (DRR) algorithm to generate the digitally reconstructed radiograph (DRR). In other words, since the content of the radiograph is known, and the left-right and AP directions or any other direction of the computed tomography (CT) data are both known, an accurate digitally reconstructed radiograph (DRR) can be obtained by looking into the three-dimensional volume data in the direction of the radiograph. The digitally reconstructed radiograph (DRR) is essentially a two-dimensional image generated for the three-dimensional volume, and its content is the image result of looking inside the three-dimensional volume at a precise orientation or angle, which allows virtual lines to be generated for each volumetric pixel, and each line is used to generate a new volumetric pixel value to generate the digitally reconstructed radiograph (DRR).

[0047] However, early digitally reconstructed radiographs (DRRs) do not always accurately match AP and LA X-ray images. This is because 2D and 3D Alignment This is the reason why the alignment procedure is performed in the following steps.

[0048] The surgeon can manually mark two other marks V0' and V1' on the two target vertebral bodies of the AP digitally reconstructed radiographic image (DRR) through the I / O interface 1130 of FIG. 2. In this manner, the distance between the two marks V0' and V1' on the AP digitally reconstructed radiographic image (DRR) of FIG. 5 can be calculated by the processor 1120. Thereafter, if the distance between the two marks V0 and V1 in the AP two-dimensional image of FIG. 4 is 100 pixels, the processor 1120 can adjust the distance parameter of the AP virtual camera to regenerate an adjusted AP digitally reconstructed radiographic image (DRR), in which the distance between the two marks V0' and V1' is also 100 pixels. Therefore, the size of the target vertebral bodies in the AP digitally reconstructed radiographic image (DRR) of FIG. 5 is adjusted to be the same size as the target vertebral bodies in the AP two-dimensional image of FIG. 4. As mentioned above, in the present application, this alignment procedure can be performed very quickly because it is only necessary to focus on the corresponding distances of the marks in the AP two-dimensional image of FIG. 4 and the AP digitally reconstructed radiographic image (DRR) of FIG. 5, and then adjust the distances to match and perform alignment.

[0049] 6, the surgeon can manually mark marks V0 and V1 on two target vertebrae of the target 9000 through the I / O interface 1130 of FIG. 2. In this manner, the distance between marks V0 and V1 can be calculated by the processor 1120 of the computing device 1100 through image recognition techniques.

[0050] As shown in FIG. 7 , this is an LA digitally reconstructed radiographic image (DRR) generated by the processor 1120, which is generated in response to information about the position or defined position of the target 9000 in the X-ray imaging procedure of the imaging device 1300. The surgeon can manually mark two marks V0′ and V1′ on the two target vertebrae of the target through the I / O interface 1130 of FIG. 2 . In this manner, the distance between the marks V0′ and V1′ can be calculated by the processor 1120 of the computing device 1100 using image recognition technology. Thereafter, if the distance between the two marks V0 and V1 in the LA two-dimensional image of FIG. 6 is 100 pixels, the processor 1120 can adjust the distance parameter of the LA virtual camera to regenerate an adjusted LA digitally reconstructed radiographic image (DRR), in which the distance between the two marks V0′ and V1′ is also 100 pixels. Therefore, the size of the target vertebrae in FIG. 7 is adjusted to be the same as the size of the target vertebrae in the LA two-dimensional image of FIG. 6 . As described above, the present application focuses only on the correlation distance of the marks in the LA two-dimensional image of FIG. 6 and the LA digitally reconstructed radiographic image (DRR) of FIG. 7, and then adjusts the distances to match, so that the present alignment procedure can be performed very quickly.

[0051] In some embodiments, the distance parameter is calculated based on the distance between the estimated position of the emitter and the estimated position of the region of interest. For example, as described above, the computing device 1100 can calculate the estimated position of the X-ray emitter 1310 using the calibrator 1510 as a reference in the two-dimensional X-ray image generated by the imaging device 1300 of FIG. 1 . Note that the estimated position of the target 9000 can be calculated by the computing device 1100 based on the displayed X-ray image of the calibrator 1510 and the estimated position of the X-ray emitter 1310. Thereafter, by obtaining the estimated positions of the X-ray emitter 1310 and the target 9000, the distance between the estimated positions of the X-ray emitter 1310 and the target 9000 can be similarly calculated by the computing device 1100.

[0052] In some embodiments, the estimated position of the emitter can be calculated based on the two-dimensional image data set, and the estimated position of the region of interest is located at a position where the first virtual line and the second virtual line are closest to each other. For example, the computing device 1100 in FIG. 2 can acquire an X-ray image using the imaging device 1300. As described above, the X-ray image of the imaging device 1300 may include the calibrator 1510 in FIG. 1. Therefore, the computing device 1100 can calculate the estimated position of the X-ray emitter 1310 based on the X-ray image including the calibrator 1510. See FIG. 8. In the figure, a first virtual line VL1 is generated between the estimated position of the X-ray emitter 1310 and the calibrator 1510 in the X-ray receiving device 1320. See FIG. 9. In the figure, a second virtual line VL2 is generated between the estimated position of the X-ray emitter 1310 and the calibrator 1510 in the X-ray receiving device 1320. When the first virtual line VL1 in Figure 8 and the second virtual line VL2 in Figure 9 are located in the same coordinate system, the estimated position of the target 9000 can be determined to be located at the position closest to the first virtual line VL1 and the second virtual line VL2.

[0053] In some embodiments, the two-dimensional image dataset includes a first two-dimensional image and a second two-dimensional image, and a first virtual line is generated between an estimated position of the launcher and a reflective center point of the at least two reflectors when the first two-dimensional image dataset is captured, and a second virtual line is generated between the estimated position of the launcher and a reflective center point of the at least two reflectors when the second two-dimensional image dataset is captured.

[0054] For example, the X-ray image of the imaging device 1300 in FIG. 1 includes an AP two-dimensional image and an LA two-dimensional image of the target 9000. See FIG. 8. A first virtual line VL1 is generated between the estimated position of the X-ray emitting device 1310 and the center point 1521 of the reflectors 1523, 1525, and 1527, which corresponds to the dynamic reference coordinate 1520A in FIG. 1 and can be acquired by the optical tracking device 1400. See FIG. 9. A second virtual line VL2 is generated between the estimated position of the X-ray emitting device 1310 and the center point 1521 of the reflectors 1523, 1525, and 1527, which corresponds to the dynamic reference coordinate 1520A in FIG. 1 and can be acquired by the optical tracking device 1400. It should be noted that the center point 1521 may be the origin of the calibrator coordinates established by the reflectors 1523, 1525, 1527, but the location of said center point 1521 is not limited to the physical center of the reflectors 1523, 1525, 1527.

[0055] See Figures 2 and 3. In operation, processor 1120 executes step 2200 to rotate a first virtual camera based on an angular difference between a first vector and a second vector, the first vector calculated from two spatial marks in the three-dimensional image dataset, and the second vector calculated from two first planar marks in the two-dimensional image dataset.

[0056] For example, the first vector is calculated by the AP virtual camera based on an appropriate DDR algorithm from two spatial marks in the three-dimensional image (see FIG. 4 ). The second vector is calculated from two planar marks V0 and V1 in the AP two-dimensional image. After obtaining the first and second vectors, the processor 1120 can calculate the angular difference between the first and second vectors. The processor 1120 can then rotate the AP virtual camera based on the angular difference. Therefore, the orientation or perspective orientation of the AP virtual camera is adjusted to resemble or be similar to the direction or perspective orientation of the X-ray emitter 1310 of the imager 1300 along the AP plane in FIG. 4 . The above steps can be performed by the computing device 1100 to adjust or modify the orientation or perspective orientation parameters of the digitally reconstructed radiograph (DRR) algorithm to generate an adjusted or modified digitally reconstructed radiograph (DRR) that is more similar to the AP two-dimensional X-ray image. As described above, the vertebral bodies in the AP two-dimensional image and the AP digitally reconstructed radiograph (DRR) can be adjusted similarly, and the frontal viewing angles associated with the LA two-dimensional image and the LA digitally reconstructed radiograph (DRR) can be adjusted similarly.

[0057] Also, the first vector is calculated by the A virtual camera from two spatial marks in the three-dimensional image based on an appropriate DDR algorithm (see FIG. 6 ). The second vector is calculated from two planar marks V0 and V1 in the LA two-dimensional image. After obtaining the first and second vectors, the processor 1120 can calculate the angular difference between the first and second vectors. Then, the processor 1120 can rotate the LA virtual camera based on the angular difference. Therefore, the orientation or perspective orientation of the LA virtual camera is adjusted to be similar to or similar to the orientation or perspective orientation of the X-ray emitter 1310 of the imaging device 1300 along the LA plane in FIG. 6 . The above steps can be performed by the computing device 1100 to adjust or modify the orientation or perspective orientation parameters of the digitally reconstructed radiograph (DRR) algorithm to generate an adjusted or modified digitally reconstructed radiograph (DRR) that is more similar to or similar to the two-dimensional X-ray image. As described above, the vertebral bodies in the LA two-dimensional image and the LA digitally reconstructed radiograph (DRR) can be adjusted similarly, and the frontal viewing angles associated with the AP two-dimensional image and the AP digitally reconstructed radiograph (DRR) can be adjusted similarly.

[0058] In some embodiments, the two-dimensional image dataset includes a first two-dimensional image, the first vector is calculated from two spatial marks in the three-dimensional image dataset by a first virtual camera, and the second vector is calculated from two first planar marks in the first two-dimensional image data. For example, the two-dimensional image dataset includes the AP two-dimensional image in FIG. 4. The first vector is calculated from two spatial marks in the three-dimensional image dataset by the AP virtual camera. The second vector is calculated from two planar marks V0 and V1 in the AP two-dimensional image in FIG. 4.

[0059] In some embodiments, the two-dimensional image dataset includes a second two-dimensional image, and the second vector is calculated from two second planar marks in the second two-dimensional image data. For example, the two-dimensional image dataset includes the LA two-dimensional image in FIG. 6. The first vector is generated by calculating from two spatial marks in the three-dimensional image dataset using the LA virtual camera. The second vector is calculated from two planar marks V0 and V1 in the LA two-dimensional image in FIG. 6.

[0060] 2 and 3. In operation, the processor 1120 executes step 2300 to rotate a first virtual camera based on an angle, the angle corresponding to an angle difference between a reconstructed image having a maximum similarity parameter and the two-dimensional image dataset, the reconstructed image having the maximum similarity parameter being the one having the maximum similarity parameter among a plurality of similarity parameters calculated for a plurality of reconstructed images in the three-dimensional image dataset, the plurality of reconstructed images including a reconstructed image generated by the first virtual camera and other reconstructed images generated by other virtual cameras, the other reconstructed images and the reconstructed image generated by the first virtual camera having different angles or pixels, and after the adjustment and rotation, the two-dimensional image dataset and the three-dimensional image dataset of the navigation system are adjusted to match the angle difference between the two-dimensional image dataset and the three-dimensional image dataset. Alignment Perform the operation.

[0061] For example, first, several LA digitally reconstructed radiographs (DRRs) are generated in the coordinate system of the three-dimensional image dataset by adjusting a preset position and / or orientation of the LA virtual camera, which may be determined by a surgeon based on a corresponding volume of interest in the three-dimensional dataset.

[0062] The LA virtual camera is centered at a preset position and / or orientation, and then rotated by a roll angle range of -20 degrees to 20 degrees and / or moved by a position range of -15 pixels to 15 pixels to generate multiple digitally reconstructed radiographs (DRRs) at different positions. Because different LA digitally reconstructed radiographs (DRRs) have different contents, multiple similarity values ​​can be calculated based on the multiple LA digitally reconstructed radiographs (DRRs) and a preset LA two-dimensional image. The preset LA two-dimensional image can be selected by the surgeon based on the region of interest in the corresponding volume of interest. The maximum similarity value can then be obtained from the multiple similarity values. To ensure good registration between the two-dimensional image and the digitally reconstructed radiograph (DRR), the current position and / or orientation of the LA virtual camera is adjusted based on the pixel and the roll angle corresponding to the maximum similarity value. In other words, the vertebrae in the LA digitally reconstructed radiograph (DRR) are adjusted to resemble the vertebrae in the LA two-dimensional image. The AP virtual camera for each vertebral body of the target 9000 can be adjusted based on the LA virtual camera corresponding to the vertebral body of the target 9000.

[0063] In addition, in the coordinate system of the three-dimensional image dataset, several AP digitally reconstructed radiographic images (DRRs) are generated by a preset position and / or orientation of the AP virtual camera, which may be determined by the surgeon based on a corresponding volume of interest in the three-dimensional dataset.

[0064] The AP virtual camera is centered at a preset position and / or orientation, and then rotated by an azimuth angle range of -15 degrees to 15 degrees and / or moved by a position range of -15 pixels to 15 pixels to generate multiple digitally reconstructed radiographs (DRRs) at different positions. Because different AP digitally reconstructed radiographs (DRRs) have different contents, multiple similarity values ​​can be calculated based on the multiple AP digitally reconstructed radiographs (DRRs) and the preset AP two-dimensional image. The preset AP two-dimensional image can be selected by the surgeon based on the region of interest in the corresponding volume of interest. The maximum similarity value can then be obtained from the multiple similarity values. To ensure good registration between the two-dimensional image and the digitally reconstructed radiograph (DRR), the current position and / or orientation of the AP virtual camera is adjusted based on the pixel and the azimuth angle corresponding to the maximum similarity value. In other words, the vertebrae in the AP digitally reconstructed radiograph (DRR) are adjusted to resemble the vertebrae in the AP two-dimensional image. The LA virtual camera for each vertebra of the target 9000 can be adjusted based on the AP virtual camera corresponding to the vertebra of the target 9000.

[0065] In some embodiments, the adjusted first virtual camera is generated after rotating the first virtual camera, where the processor 1120 is configured to perform the step of rotating the adjusted first virtual camera based on an angle, the angle corresponding to an angular difference between the reconstructed image having the adjusted maximum similarity parameter and the two-dimensional image dataset, the reconstructed image having the adjusted maximum similarity parameter being the reconstructed image having the maximum similarity parameter among the plurality of similarity parameters calculated for the adjusted plurality of reconstructed images, the adjusted plurality of reconstructed images including the reconstructed image generated by the adjusted first virtual camera and other reconstructed images generated by other virtual cameras, and the other reconstructed images and the reconstructed image generated by the adjusted first virtual camera having different angles.

[0066] For example, an adjusted AP virtual camera or an adjusted LA virtual camera is generated after rotating the AP virtual camera or the LA virtual camera. The adjusted LA virtual camera is generated by rotating the adjusted LA virtual camera through a roll angle range of -20 degrees to 20 degrees and / or moving the adjusted LA virtual camera through a position range of -15 pixels to 15 pixels, thereby generating multiple digitally reconstructed radiographs (DRRs) at different positions. Because different LA digitally reconstructed radiographs (DRRs) have different contents, multiple similarity values ​​can be calculated based on the multiple adjusted LA digitally reconstructed radiographs (DRRs) and a preset LA two-dimensional image. The preset LA two-dimensional image can be selected by the surgeon based on the region of interest in the corresponding volume of interest. Then, a maximum similarity value can be obtained from the multiple similarity values. To achieve good registration between the two-dimensional image and the digitally reconstructed radiograph (DRR), the current position and / or orientation of the LA virtual camera is adjusted based on the maximum similarity value corresponding to the pixel and roll angle. In other words, the vertebrae in the LA digitally reconstructed radiograph (DRR) are adjusted to resemble the vertebrae in the LA two-dimensional image.

[0067] The adjusted AP virtual camera is rotated by an azimuth angle range of -15 degrees to 15 degrees and / or moved by a position range of -15 pixels to 15 pixels to generate multiple DRRs at different positions. Since different AP DRRs have different contents, multiple similarity values ​​can be calculated based on the multiple adjusted AP DRRs and the AP two-dimensional image. The AP two-dimensional image calculation can be selected by the surgeon based on the region of interest in the corresponding volume of interest. Then, the maximum similarity value can be obtained from the multiple similarity values. To achieve good registration between the two-dimensional image and the DRR, the current position and / or orientation of the AP virtual camera is adjusted based on the pixel and tilt angle corresponding to the maximum similarity value. In other words, the vertebrae in the AP DRR are adjusted to resemble the vertebrae in the AP two-dimensional image.

[0068] It should be clarified that the present application is not limited to the structure and operation of the embodiment shown in FIGS. 2-9, which are only used to exemplify one implementation of the present application.

[0069] In some embodiments, the processor 1120 is configured to perform a process of calibrating a first virtual camera of a plurality of virtual cameras based on a matrix corresponding to the two-dimensional image dataset, where the plurality of virtual cameras corresponds to the three-dimensional image dataset. For example, both the AP two-dimensional image and the LA two-dimensional image of the imaging device 1300 in FIG. 1 include the calibrator 1510. The processor 1120 can calculate a matrix of a coordinate system for transforming the coordinate system of the AP two-dimensional image into the LA two-dimensional image using known parameters of the calibrator 1510. Thus, a matrix corresponding to the relationship between the AP two-dimensional image and the LA two-dimensional image can be obtained. The processor 1120 can then calibrate the AP virtual camera or the LA virtual camera based on the matrix, so that the relationship between the AP virtual camera and the LA virtual camera is similar to the relationship between the AP two-dimensional image and the LA two-dimensional image.

[0070] 2D and 3D Alignment

[0071] After the initial registration in the first procedure, the navigation system 1000 continues to establish a relationship between the three-dimensional image dataset and the X-ray two-dimensional image of the two-dimensional image dataset, the three-dimensional image dataset is stored in the database 1200, and the X-ray two-dimensional image of the two-dimensional image dataset is displayed on the display of the imaging device 1300 or stored in the database 1200. The method of establishing the relationship will be described below.

[0072] FIG. 10 illustrates a two-dimensional image of a two-dimensional image dataset of a region of interest and a three-dimensional image dataset of the same region of interest according to one embodiment of the present application. Alignment 3 is a flowchart of a method for generating a first reconstructed image from the three-dimensional image dataset based on a first spatial parameter. See also Figures 2 and 10. In operation, the processor 1120 executes step 3100 to generate a first reconstructed image from the three-dimensional image dataset based on a first spatial parameter.

[0073] For example, the processor 1120 of the computing device 1100 may access the database 1200 and simulate a virtual camera corresponding to the three-dimensional image dataset based on the first spatial parameters, and generate a first digitally reconstructed radiograph (DRR). In this example, the virtual camera may be used to reconstruct the two-dimensional and three-dimensional images (through initial registration and / or other adjustments). Alignment Although the alignment may be pre-aligned (to facilitate alignment), in other embodiments of the present application, alignment may not be performed via initial alignment and / or other alignment techniques.

[0074] See also Figures 2 and 10. In operation, the processor 1120 performs step 3200 to calculate a reference similarity value based on the first reconstructed image and at least one two-dimensional image data set. For example, the processor 1120 of the computing device 1100 can calculate the reference similarity value based on the first digitally reconstructed radiographic image (DRR) and the two-dimensional image data.

[0075] See also Figures 2 and 10. In operation, processor 1120 performs step 3300 to generate a second reconstructed image from the three-dimensional image dataset based on the second spatial parameters. For example, processor 1120 of computing device 1100 can access database 1200 and simulate a virtual camera corresponding to the three-dimensional image dataset based on the second spatial parameters, and further generate a second digitally reconstructed radiograph (DRR).

[0076] See also Figures 2 and 10. In operation, processor 1120 performs step 3400 to calculate a comparative similarity value based on the second reconstructed image and at least one two-dimensional image data set. For example, processor 1120 of computing device 1100 can calculate a comparative similarity value based on the second digitally reconstructed radiographic image (DRR) and the two-dimensional image data.

[0077] See also Figures 2 and 10. In operation, processor 1120 performs step 3500 to compare the comparison similarity number to a reference similarity number. For example, processor 1120 of computing device 1100 may compare the comparison similarity number to a reference similarity number.

[0078] See also Figures 2 and 10. In operation, the processor 1120 executes step 3600 to combine the two-dimensional image data set and the three-dimensional image data set if the comparison similarity score is less than or equal to the reference similarity score. Alignment death, Alignment The two-dimensional image data set and the three-dimensional image data set are used for computer-assisted surgical navigation. For example, if the comparison similarity value is equal to or less than the reference similarity value, it indicates that the registration between the second digitally reconstructed radiograph (DRR) and the two-dimensional image data set is complete. Therefore, the processor 1120 of the computing device 1100 compares the two-dimensional image data set of the imaging device 1300 with the three-dimensional image data set stored in the database 1200. AlignmentThereafter, the two-dimensional image data set from the imaging device 1300 and the three-dimensional image data set stored in the database 1200 can be Alignment Therefore, the method 3000 in FIG. 10 of the present application Alignment The resulting two-dimensional and three-dimensional image datasets can be used for computer-assisted surgical navigation.

[0079] It should be noted that the present application is not limited to the operation of the embodiment shown in Figure 10, which is used only to exemplify one implementation of the present application. Note that the method 3000 in Figure 10 may be executed by the processor 1120 to generate two-dimensional image datasets and three-dimensional image datasets. Alignment The two-dimensional image dataset may include an AP two-dimensional image and an LA two-dimensional image, and the AP digitally reconstructed radiographic image (DRR) and the LA digitally reconstructed radiographic image (DRR) are from the three-dimensional image dataset.

[0080] In some embodiments, generating the first or second reconstructed image from the three-dimensional image dataset based on the first or second reconstructed image includes positioning a virtual camera based on a three-dimensional target, the three-dimensional target being constructed based on the first or second reconstructed image, and capturing the first or second reconstructed image of the three-dimensional target with the virtual camera.

[0081] For example, the three-dimensional image dataset may be volume pixel values ​​acquired after computed tomography (CT) of the target 9000, which includes the region of interest. The volume pixel values ​​may be represented by volume rendering, after which the three-dimensional target (also referred to as a three-dimensional model) of the region of interest may be simultaneously displayed on a display and stored in the database 1200. The processor 1120 may access the three-dimensional model in the database 1200 and determine the position and orientation of a virtual camera for the corresponding three-dimensional model based on the spatial parameters of the given virtual camera. Once the virtual camera is set, the processor 1100 may acquire one or more two-dimensional images as a first digitally reconstructed radiographic image (DRR) or a second digitally reconstructed radiographic image (DRR) using a digitally reconstructed radiographic image (DRR) algorithm known in the art.

[0082] In some embodiments, the virtual camera is a modularized functional instruction and includes one of an algorithm and an equation. In some embodiments, the virtual camera can be simulated but is not limited to radiographic volume rendering.

[0083] In the initial alignment, the virtual camera is adjusted to a preset initial spatial parameter having the position and / or orientation of the virtual camera, which further improves the efficiency and accuracy of the navigation system 1000. However, since the efficiency is still lacking for medical purposes, after the initial alignment, the adjusted virtual camera is used for two-dimensional and three-dimensional AlignmentFurther optimization of the procedure is needed to achieve greater accuracy. In some embodiments, each of the first and second spatial parameters is used to define a position and / or orientation of a virtual camera, where the position and / or orientation is defined to view the three-dimensional volume relative to the virtual camera, or the virtual camera is defined to view the three-dimensional volume. As described above, the adjusted virtual camera needs to be further optimized, and the optimization method for the adjusted virtual camera is to adjust the spatial parameters with the position and / or orientation of the adjusted virtual camera. For example, each of the first and second spatial parameters is used to define a position and / or orientation of a virtual camera, where the position and / or orientation is defined to view the three-dimensional volume relative to the virtual camera, or the virtual camera is defined to view the three-dimensional volume.

[0084] In some embodiments, each of the first spatial parameter and the second spatial parameter includes one of a position, an orientation, and a parameter including a position and an orientation. For example, the first spatial parameter may be a parameter including a position, an orientation, or a parameter including a position and an orientation, and the second spatial parameter may be a parameter including a position, an orientation, or a parameter including a position and an orientation.

[0085] In some embodiments, each comparison similarity measure and reference similarity measure is calculated by local normalized correlation (LNC), sum of squared differences (SSD), normalized cross-correlation (NCC), or correlation ratio (CR).

[0086] In some embodiments, the three-dimensional image dataset (e.g., the three-dimensional image dataset stored in database 1200 in FIG. 1 ) is generated by one of a magnetic resonance imaging (MRI) device, an isocentric C-arm fluoroscopic imaging device, an O-arm device, a bi-plane fluoroscopy device, a computed tomography (CT) device, a multi-slice computed tomography (MSCT) device, a high frequency ultrasound (HIFU) device, an optical coherence tomography (OCT) device, an intra-vascular ultrasound (IVUS) device, a 3D or 4D ultrasound device, and an intraoperative computed tomography (CT) device.

[0087] In some embodiments, the two-dimensional image dataset is generated by one of a C-arm fluoroscopic imaging device, a magnetic resonance imaging (MRI) device, an iso-centric C-arm fluoroscopic imaging device, an O-arm device, a bi-plane fluoroscopy device, a computed tomography (CT) device, a multi-slice computed tomography (MSCT) device, a high frequency ultrasound (HIFU) device, an optical coherence tomography (OCT) device, an intra-vascular ultrasound (IVUS) device, a two-dimensional, three-dimensional, or four-dimensional ultrasound device, and an intraoperative computed tomography (CT) device.

[0088] In some embodiments, the first spatial parameters are generated based on a spatial relationship between at least one marker in the two-dimensional image capture device and the region of interest. As described above, the virtual camera is adjusted to the initial spatial parameters in a preset initial alignment, which further improves the efficiency and accuracy of the navigation system 1000, and the adjusted spatial parameters after the initial alignment are used for subsequent two-dimensional and three-dimensional image capture. Alignment For example, the first spatial parameter is generated from the spatial relationship between the calibrator 1510 and the target 9000 in the X-ray receiving device 1320 in FIG.

[0089] In some embodiments, the first spatial parameter is the spatial parameter with the greatest similarity among the multiple similarities generated in the previous comparison step. For example, the comparison step may be performed multiple times, and each comparison step may generate a similarity. The first spatial parameter is obtained by finding the parameter with the greatest similarity among the multiple similarities.

[0090] In some embodiments, two-dimensional and three-dimensional Alignment At the start of the process, the adjusted virtual camera has first spatial parameters, which have a first distance and / or a first orientation. Alignment After initiation of the process, the adjusted virtual camera moves from the first spatial parameter to the second spatial parameter, where the second spatial parameter has a second distance and / or a second orientation. Thus, the second spatial parameter differs from the first spatial parameter in the distance and / or orientation of the corresponding virtual camera relative to the three-dimensional image dataset (e.g., the three-dimensional image dataset is stored in database 1200 in FIG. 1 ). More specifically, the adjusted virtual camera in this embodiment is a modularized functional instruction, including an algorithm, an equation, and one or more parameters, so that processor 1100 can use the parameters and then simulate the adjusted virtual camera at a specific position in the coordinate system of the three-dimensional image dataset.

[0091] In some embodiments, the processor 1120 further performs the steps of: generating a third reconstructed image from the three-dimensional image dataset based on the third spatial parameter if the comparative similarity number is greater than the reference similarity number; calculating a second comparative similarity number based on the third reconstructed image and the at least one two-dimensional image dataset; comparing the second comparative similarity number with the reference similarity number; and reconstructing the two-dimensional image dataset and the three-dimensional image dataset if the second comparative similarity number is less than or equal to the reference similarity number. Alignment and

[0092] For example, if the comparison similarity value is greater than the reference similarity value in the previous comparison, the processor 1120 of the computing device 1100 can access the database 1200 and simulate an adjusted virtual camera corresponding to the three-dimensional image dataset in a third position, and further generate a third digitally reconstructed radiographic image (DRR).

[0093] Additionally, the processor 1120 of the computing device 1100 may calculate a second comparison similarity value based on the third AP and / or LA digitally reconstructed radiographic image (DRR) and the corresponding AP and / or LA two-dimensional image of the two-dimensional image dataset.

[0094] Additionally, processor 1120 of computing device 1100 may compare the second comparison similarity value with a reference similarity value.

[0095] If the second comparison similarity value is equal to or less than the reference similarity value, it indicates that the third AP and / or LA digitally reconstructed radiographic image (DRR) has been registered with the corresponding AP and / or LA two-dimensional image of the two-dimensional image dataset. Thereafter, the two-dimensional image dataset acquired in real time by the imaging device 1300 is compared with the previously acquired three-dimensional image dataset pre-stored in the database 1200. Alignment can be considered completed.

[0096] FIG. 11 illustrates a two-dimensional image dataset of a region of interest and a three-dimensional image dataset of a region of interest according to one embodiment of the present application. Alignment2 and 11 . In operation, processor 1120 executes step 4100 to simulate moving the adjusted virtual camera from an origin or a previous spatial position in the coordinate system of the three-dimensional image dataset to a first spatial position, and processor 1120 executes step 4150 to generate a first digitally reconstructed radiographic image (DRR) (also referred to as a first reconstructed image) of the first spatial position corresponding to the first virtual camera. Processor 1120 then executes step 4200 to calculate a first similarity measure based on the first digitally reconstructed radiographic image (DRR) and the two-dimensional image, where the two-dimensional image generated by imaging device 1300 may be an AP or LA two-dimensional image, and processor 1120 executes step 4250 to move the adjusted virtual camera back to the initial spatial position.

[0097] Thereafter, processor 1120 executes step 4300 to determine whether the adjusted virtual camera has moved from the initial spatial position to all spatial positions. For example, the adjusted virtual camera may move from the initial spatial position to a first spatial position (e.g., from (0,0) to (1,0) in the Cartesian coordinate system), and then move from the initial spatial position to a second spatial position, etc. (e.g., from (0,0) to (-1,0) in the Cartesian coordinate system). The amount of movement of the adjusted virtual camera may be preset according to actual needs. If it is determined that the adjusted virtual camera has not moved to all spatial positions, method 4000 returns to execute step 4100 to further move the adjusted virtual camera to another spatial position (e.g., the second spatial position). Thereafter, processor 1120 performs steps 4150 and 4200 to generate another Digitally Reconstructed Radiograph (DRR) (e.g., a second Digitally Reconstructed Radiograph (DRR)) and calculate another similarity measure (e.g., a second similarity measure). Thereafter, processor 1120 performs step 4250 to move the adjusted virtual camera back to the initial spatial position.

[0098] If it is determined that the adjusted virtual camera has been moved to all spatial positions, method 4000 continues with step 4350. Processor 1120 then executes step 4350 to determine whether the similarity value corresponding to the moved virtual camera is high. If the similarity value corresponding to the moved virtual camera is determined to be high, this indicates that the digitally reconstructed radiographic image (DRR) generated by the adjusted virtual camera at the next position is more similar to the two-dimensional image than the digitally reconstructed radiographic image (DRR) generated by the adjusted virtual camera at the initial or previous spatial position. Therefore, processor 1120 then executes step 4400 to adjust the adjusted virtual camera from the initial or previous spatial position to the next spatial position. In this embodiment, the initial spatial position represents the position to which processor 1120 initially positions the adjusted virtual camera, and the previous spatial position represents the position to which the adjusted virtual camera is repositioned after adjustment, and the adjustment may be performed after the aforementioned alignment (e.g., initial alignment) or Alignment It is produced by one or a combination of the following processes:

[0099] After performing step 4400, method 4000 returns to perform step 4100. Steps 4100, 4150, 4200, 4250, 4300, and 4350 are then performed by processor 1120. See step 4350. If it is determined that the similarity value corresponding to the adjusted virtual camera at the next position is less than or equal to the similarity value of any previous position, method 4000 continues with execution of step 4450.

[0100] After performing step 4450, processor 1120 performs step 4500 to decrease the adjustment amount. For example, if the previous adjustment amount was to move the adjusted virtual camera by 1 mm from its previous position in the coordinate system of the displayed three-dimensional target in the three-dimensional image dataset, processor 1120 performs step 4500 to decrease the adjustment amount to move by 0.5 mm. Thereafter, processor 1120 performs step 4550 to determine whether the adjustment amount is less than a preset spatial value, for example, 0.75 mm. If it is determined that the adjustment amount is greater than or equal to the preset spatial value, method 4000 returns to perform step 4100.

[0101] If the adjustment amount is determined to be smaller than the preset spatial value, it indicates that the registration between the digitally reconstructed radiograph (DRR) and the two-dimensional image is almost complete. Therefore, the two-dimensional image data set of the imaging device 1300 and the three-dimensional image data set pre-stored in the database 1200 are registered. Alignment can be defined as completed.

[0102] In some embodiments, if the difference between the first spatial parameter and the second spatial parameter is less than or equal to a predetermined spatial value, the two-dimensional image dataset and the three-dimensional image dataset are combined. Alignment For example, in step 4550 in FIG. 11 , the difference between the first spatial position and the second spatial position is the adjustment amount in step 4550. If the adjustment amount between the first spatial position and the second spatial position is equal to or less than a preset spatial value, it indicates that the registration between the digitally reconstructed radiographic image (DRR) and the two-dimensional image data is complete. For example, the current spatial value may be 0.01 mm. If the adjustment amount is equal to or less than 0.01 mm, it indicates that the registration between the digitally reconstructed radiographic image (DRR) and the two-dimensional image data is complete. Therefore, the processor 1120 of the computing device 1100 compares the two-dimensional image data set of the imaging device 1300 with the three-dimensional image data set stored in the database 1200. Alignment It is possible.

[0103] It should be noted that the present application is not limited to the operation of the embodiment shown in FIG. 11, which is only used to exemplify one implementation of the present application.

[0104] FIG. 12 illustrates a two-dimensional image dataset and a three-dimensional image dataset of a region of interest according to one embodiment of the present application. Alignment 5 is a flowchart of a method 5000. See also Figures 2 and 12. In operation, the processor 1120 executes step 5100 to perform an initial alignment procedure for the navigation platform 1000 and further refine the accuracy of the navigation platform 1000.

[0105] The processor 1120 executes step 5200 to perform an XY alignment procedure to pre-set the XY position of the adjusted virtual camera, which corresponds to the three-dimensional image dataset pre-stored in the database 1200 of the navigation platform 1000.

[0106] The processor 1120 executes step 5300 to perform an initial thumbnail alignment for the calibrated virtual camera, which corresponds to a three-dimensional image dataset pre-stored in the database 1200 of the navigation platform 1000.

[0107] The processor 1120 executes step 5400 to perform original view alignment with the adjusted virtual camera, which corresponds to the three-dimensional image dataset pre-stored in the database 1200 of the navigation platform 1000.

[0108] It should be noted that the flowcharts of XY coordinate alignment in step 5200, thumbnail alignment in step 5300, and initial alignment in step 5400 are all shown in Fig. 11. The difference is that XY coordinate alignment focuses on aligning images in X and Y coordinates, thumbnail alignment achieves the purpose of quick alignment using preview images or reduced images, and initial alignment further improves the alignment situation after thumbnail alignment (e.g., preview alignment).

[0109] Processor 1120 executes step 5500 to determine whether realignment is necessary. If an error or alignment failure is determined by processor 1120 or the surgeon, a realignment procedure must be performed. For example, if after multiple calculation cycles the difference is much larger than a preset value, method 5000 continues executing step 5600 to perform realignment. On the other hand, if the alignment is nearly complete and no realignment procedure needs to be performed, method 5000 continues executing step 5700. In this situation, the entire alignment procedure in FIG. 12 has already been performed.

[0110] In some embodiments, before the adjusted virtual camera generates the first and second reconstructed images, the navigation system 1000 performs an initial registration based on the two-dimensional images using a low-resolution image. The use of a low-resolution format is to reduce interference or noise between two similar images, thus contributing to the initial registration. In step 5300 of method 5000, the digitally reconstructed radiograph (DRR) used for thumbnail registration is a low-resolution image.

[0111] Realign

[0112] It should be noted that the present application is not limited to the operation of the embodiment shown in FIG. 12, which is only used to exemplify one implementation of the present application.

[0113] The re-registration flow of step 5600 in method 5000 will be described in detail in the following Fig. 13. Fig. 13 is a flowchart of a method 6000 for re-registration according to one embodiment of the present application. In clinical practice, the re-registration of two-dimensional images and three-dimensional images is performed. Alignment The procedure may fail during the registration procedure, so that the AP or LA two-dimensional image may not be successfully registered with the digitally reconstructed radiographic image (DRR), which is generated based on the three-dimensional image dataset. However, a registration failure occurs when only one of the two-dimensional images is not registered with the corresponding digitally reconstructed radiographic image (DRR). When any kind of failure occurs, known navigation systems Alignment The impact on the skilled person or surgeon is so great that they have to go back to the first step and Alignment You need to re-run the procedure once. Alignment This significantly slows down the entire procedure. In this embodiment, the navigation system 1000 can solve this problem by performing a realignment flow.

[0114] See Figures 2 and 13. If the registration between the LA digitally reconstructed radiographic image (DRR) (also called the LA three-dimensional reconstructed image) and the LA C-arm X-ray image (also called the two-dimensional image) is not completed, re-registration is required, and the processor 1120 can obtain the spatial parameters of the AP virtual camera, which can be obtained by the processor 1120 after the AP C-arm X-ray image has been successfully registered in the previous step. Alignment As mentioned above, the AP virtual camera corresponds to the AP digitally reconstructed radiograph (DRR) in the following text. Alignment The LA Virtual Camera is considered to be a virtual camera that has been installed, and the ... Not aligned However, the present application is not limited to the above embodiment, and the AP virtual camera can be configured according to actual needs. Not aligned Virtual camera and LA virtual camera AlignmentIt can be set as a virtual camera.

[0115] To facilitate understanding of the method 6000 shown in Fig. 13, please refer to Fig. 14, which is a schematic diagram of a virtual camera VC according to one embodiment of the present application. It should be noted that the virtual camera VC in Fig. 14 is used to explain the concept of the present application. The spatial parameters of the virtual camera VC may include, but are not limited to, multiple position and / or orientation information, such as vectors. Alignment Virtual camera and Not aligned Any virtual camera can be interpreted as shown in Figure 14. For example, Alignment Virtual camera and Not aligned The spatial parameters of each virtual camera may include Z and Y axis vectors and a focal point FP, which may define a coordinate system of the three-dimensional target generated from the three-dimensional image dataset. Note that the focal point FP may not be included in the spatial parameters, but may be obtained from the database 1200 by the processor 1120. In the recognition procedure, Alignment The virtual camera is subsequently marked VC1, and its axis direction and focus are marked Z1, Y1, and FP1. Not aligned The virtual camera is subsequently marked VC2, and its axis direction and focus are marked Z2, Y2 and FP2.

[0116] See Figures 2, 13 and 14. In operation, the processor 1120 performs step 6100 to calculate, in the coordinate system of the three-dimensional image dataset: Alignment For example, the processor 1120 may obtain a first vector from the spatial parameters of the virtual camera in the coordinate system of the three-dimensional image dataset pre-stored in the database 1200, as follows: Alignment The Z1 axis can be obtained from the spatial parameters of the virtual camera VC1.

[0117] The processor 1120 executes step 6200 to convert the image data into a vector by at least one transformation matrix. AlignmentTransform the first vector of the virtual camera into the coordinate system of the three-dimensional image dataset: Not aligned For example, the processor 1120 may obtain a first transformation vector of the virtual camera by using a preset or pre-stored transformation matrix. Alignment Transform the Z1 axis of the virtual camera VC1 into the coordinate system of the 3D image dataset: Not aligned The Z2 axis of the virtual camera VC2 can be obtained, and a set or pre-stored transformation matrix is ​​established by the processor 1120 based on the spatial relationships of particular elements in the navigation system 1000.

[0118] The processor executes step 6300 to calculate, in the coordinate system of the three-dimensional image dataset: Not aligned Get the focus of the virtual camera, Not aligned The virtual camera is located on a reference point of the two-dimensional image dataset, Not aligned The two-dimensional image is Not aligned For example, the processor 1120 may perform a previously unsuccessful Alignment Based on the center point of the calibrator of the two-dimensional image in the coordinate system of the three-dimensional image dataset, Not aligned The focal point FP2 of the virtual camera VC2 can be obtained.

[0119] The processor executes step 6400 to obtain a first transformation vector and Not aligned Based on the focus of the virtual camera Not aligned Reposition the virtual camera Not aligned Generate an updated reconstructed image based on the repositioning of the virtual camera, e.g. Not aligned The virtual camera VC2 can be simulated to move from its initial position to a calculated position calculated by the processor 1120, said calculated position being: Not aligned Z2 axis of virtual camera VC2 and Not aligned Calculated based on the focal point FP2 of virtual camera VC2.

[0120] In some embodiments, Alignment The first vector of the virtual camera is Alignment From the virtual camera position Alignment For example, as shown in Figure 14, Alignment The Z1 axis of the virtual camera VC1 is Alignment From the position of the virtual camera VC1 Alignment The focus FP1 of the virtual camera VC1 is already captured.

[0121] In some embodiments, the first vector is a vector in the coordinate system of the three-dimensional image dataset. Alignment A virtual camera (e.g., in Fig. 14) Alignment 14. The Z axis of the virtual camera VC1 is defined as the Z axis (for example, the Z1 axis in FIG. 14).

[0122] In some embodiments, the processor 1120 retrieves instructions from the memory 1110 to perform, in the coordinate system of the three-dimensional image dataset: Alignment obtaining a second vector from the spatial parameters of the virtual camera in the coordinate system of the three-dimensional image dataset; Alignment Based on the second vector of the virtual camera Not aligned and obtaining a second transformation vector from the spatial parameters of the virtual camera.

[0123] For example, the processor 1120 may calculate, in the coordinate system of the three-dimensional image dataset, Alignment The processor 1120 obtains the Y axis from the spatial parameters of the virtual camera VC1 in the coordinate system of the three-dimensional image dataset. Alignment Based on the Y1 axis of the spatial parameters of the virtual camera VC1 Not aligned Gets the Y2 axis of virtual camera VC2.

[0124] In some embodiments, the second vector is from a center point (e.g., point FP in FIG. 14) to the top of the reconstructed image (e.g., the top in FIG. 14), and the reconstructed image is parallel to the reconstructed image. Alignment A virtual camera (e.g., in Fig. 14) AlignmentThe image is acquired by a virtual camera VC1.

[0125] In some embodiments, in the coordinate system of the three-dimensional image dataset, the second vector is: Alignment A virtual camera (e.g., in Fig. 14) Alignment 14. The Y axis of the spatial parameters of the virtual camera VC1 is defined as the Y axis (for example, the Y1 axis in FIG. 14).

[0126] It should be clarified that the present application is not limited to the structure and operation of the embodiment shown in FIGS. 13 and 14, which are used only to exemplify one implementation of the present application.

[0127] In some embodiments, the two-dimensional image dataset includes first and second two-dimensional images, and the at least one transformation matrix includes a first matrix for transforming a coordinate system of the first two-dimensional image into a coordinate system of the second two-dimensional image, for example, the two-dimensional image dataset includes an AP two-dimensional image and an LA two-dimensional image, and the transformation matrix includes a first matrix of a coordinate system for transforming a coordinate system of the AP two-dimensional image into a coordinate system of the LA two-dimensional image.

[0128] In some embodiments, in the coordinate system of the three-dimensional image dataset, the reference point is located at the center point of the calibrator module in the two-dimensional image dataset. For example, the reference point is located at the center point (also called the origin) of the calibrator module, and the calibrator module: Not aligned It appears on two-dimensional AP or LA X-ray images. Not aligned The two-dimensional X-ray image of the AP or LA is included in the two-dimensional image data set, and the reference points are simulated to be positioned in position and orientation in the coordinate system of the three-dimensional image data set for further provision to the program of the navigation system 1000.

[0129] In some embodiments, the at least one transformation matrix includes a second matrix and a third matrix, where the second matrix is ​​used to transform the coordinate system of the fiducial marks into the coordinate system of the three-dimensional image dataset, and the third matrix is ​​used to transform the coordinate system of the fiducial marks into the coordinate system of the tracker module. For example, the at least one transformation matrix includes a second matrix and a third matrix. The second matrix is ​​used to transform the coordinate systems of the calibrators 1510, 1530 and the dynamic reference coordinates 1520A in FIG. 1 into the coordinate system of the three-dimensional image dataset stored in the database 1200 in FIG. 1. The third matrix is ​​used to transform the coordinate systems of the calibrators 1510, 1530 and the dynamic reference coordinates 1520A in FIG. 1 into the coordinate system of the tracker module 1400 in FIG. 1. Transformation matrices and coordinate system conversion are well-known techniques. For more details, see "L. Dorst, et al., Geometric Algebra For Computer Science, published by Morgan Kaufmann Publishers and M. N. Oosterom, et al., Navigation of a robot-integrated fluorescence laparoscope in preoperative SPECT / CT and intraoperative freehand SPECT imaging data: A phantom study, August 2016, Journal of Biomedical Optics 21(8):086008," the entire contents of which are incorporated herein by reference.

[0130] The realignment step 5600 of the method 5000 in Fig. 12 is described in the following Fig. 15. Fig. 15 is a flowchart of a realignment method 7000 according to one embodiment of the present application. Please refer to Fig. 2 and Fig. 15. Not aligned If the virtual camera needs to be realigned, the processor 1120 selects a vertebra corresponding to vertebra V0 or V2 in FIG. 5 or FIG. 7. Alignment You can find virtual cameras that have already been installed.

[0131] See Figures 2 and 15. In operation, the processor 1120 executes step 7100 to generate a first Alignment Get the first spatial parameters of the virtual camera and Alignment The set position of the virtual camera is positioned corresponding to the first two-dimensional image of the two-dimensional image dataset. For example, the processor 1120 may position the first two-dimensional image of the vertebra V0 or V2 in FIG. 5 or FIG. 7. Alignment The first spatial parameter corresponding to the virtual camera can be obtained. Alignment The set position of the virtual camera is positioned corresponding to the two-dimensional image of the vertebral body V0 or V2 in FIG. 4 or FIG.

[0132] See Figures 2 and 15. In operation, the processor 1120 executes step 7200 to generate a first Alignment The first spatial parameter of the virtual camera is Not aligned Adjust the second spatial parameters of the virtual camera and Not aligned By repositioning the virtual camera, an updated reconstructed image is generated. Not aligned The set position of the virtual camera may not correspond to the first two-dimensional image of the two-dimensional image dataset. For example, the processor 1120 may: Alignment The first spatial parameter of the virtual camera is Not aligned A second spatial parameter of the virtual camera can be adjusted. Alignment The first spatial parameter of the virtual camera is the vertebra V0 or V2 shown in FIG. 5 or FIG. 7, Not aligned The second spatial parameter of the virtual camera corresponds to the vertebral body V1 shown in FIG. 5 or FIG. Not aligned The setting position of the virtual camera cannot correspond to the two-dimensional image of the vertebral body V1 shown in FIG. 4 or FIG.

[0133] In some embodiments, a first portion of the region of interest is included in the first two-dimensional image, a second portion of the region of interest is included in the second two-dimensional image, and the first and second portions of the region of interest are adjacent to each other. For example, target 9000 is a patient with a spinal disorder and surgery needs to be performed to stabilize three vertebral segments. All three vertebral segments are considered regions of interest and are depicted in FIGS. 4 and 6. Based on the success or failure of the registration of the three-dimensional image datasets, the vertebral segments can be classified into two portions.

[0134] The first portion includes vertebral body V0 or V2, and the second portion includes vertebral body V1. More specifically, the first portion of the area of ​​interest in FIG. 1 is included in a first two-dimensional image corresponding to vertebral body V0 or V2. The first two-dimensional image in FIG. 4 is an AP X-ray image, and the first two-dimensional image in FIG. 6 is an LA X-ray image. The second portion of the area of ​​interest in FIG. 1 is included in the first two-dimensional image corresponding to vertebral body V1. It is the first two-dimensional image depicted in FIGS. 4 and 6. As shown in FIGS. 4 and 6, vertebral body V0 or V2 included in the first portion of the target 9000 is adjacent to vertebral body V1 included in the second portion of the target 9000 corresponding to vertebral body V1.

[0135] In some embodiments, a first portion of the region of interest is defined based on a first marker in a first two-dimensional image of the two-dimensional image dataset, and a second portion of the region of interest is defined based on a second marker in the first two-dimensional image of the two-dimensional image dataset. For example, the first portion of the region of interest of the target 9000 in Figure 1 can be defined by the program of the navigation system 1000 to automatically recognize the image edges of the vertebral body, which is marked with V0 or V2 in the two-dimensional images of Figures 4 and 6. Similar to the first portion, the second portion of the region of interest of the target 9000 in Figure 1 can be defined by the program of the navigation system 1000 to automatically recognize the image edges of the vertebral body, which is marked with V1 in the two-dimensional images of Figures 4 and 6.

[0136] In some embodiments, the first AlignmentThe first spatial parameters of the virtual camera include position and / or orientation data, and the position and / or orientation are used to visualize the three-dimensional volume. Alignment The first virtual camera is defined to look at the Alignment A virtual camera is defined to view a 3D volume. For example, Alignment The first spatial parameters of the virtual camera include position and / or orientation data, the position and / or orientation being used to visualize the three-dimensional volume. Alignment defined to look relative to a pre-defined virtual camera, or Alignment The virtual camera is defined relative to the 3D volume. Alignment The first spatial parameter of the obtained virtual camera corresponds to the vertebral body V0 or V2 in FIG. 5 or FIG. 7, and a three-dimensional volume is generated from the three-dimensional image dataset.

[0137] In some embodiments, the first Not aligned The second spatial parameters of the virtual camera include position and / or orientation data, the position and / or orientation being used to visualize the three-dimensional volume. Not aligned For a virtual camera, the first Not aligned A virtual camera is defined to view a 3D volume. For example, Not aligned The second spatial parameters of the virtual camera include position and / or orientation data, the position and / or orientation being used to visualize the three-dimensional volume. Not aligned defined as viewed relative to a virtual camera, or Not aligned A virtual camera is defined to view the three-dimensional volume. Not aligned The second spatial parameter of the virtual camera corresponds to the vertebral body V1 in FIG. 5 or FIG. 7, and the three-dimensional volume is generated from the three-dimensional image dataset.

[0138] In some embodiments, a first similarity score is calculated based on the comparison between the plurality of similarity scores. AlignedA position of the virtual camera is set, and a comparison result between the plurality of similarity values ​​is calculated based on the plurality of different reconstructed images and a first two-dimensional image of the two-dimensional image data, and the plurality of different reconstructed images are obtained from the three-dimensional image data set. For example, based on the comparison result between the plurality of similarity values Alignment The position of the virtual camera is set, and the comparison results between the multiple similarity values ​​are calculated based on different digitally reconstructed radiographs (DRRs) and two-dimensional images, where the different digitally reconstructed radiographs (DRRs) are obtained from the three-dimensional image dataset, and the two-dimensional images are associated with the vertebral bodies V0 or V2 in Figure 4 or Figure 6.

[0139] In some embodiments, each similarity measure is calculated by local normalized correlation (LNC), sum of squared differences (SSD), normalized cross-correlation (NCC), or correlation ratio (CR).

[0140] In some embodiments, the processor 1120 further comprises a first Not aligned defining a second spatial parameter of the virtual camera as an N spatial parameter; Alignment determining whether the set position of the virtual camera corresponds to a first two-dimensional image of the two-dimensional image data set, Alignment The virtual camera has N,N,space parameters, N and M are integers, and M is less than N. Alignment If the set position of the virtual camera corresponds to the first 2D image of the 2D image dataset, the NM spatial parameters are set to the first Alignment and defining the first spatial parameter of the virtual camera as the first spatial parameter of the virtual camera.

[0141] For example, the processor 1120 of the computing device 1100 may: Not alignedThe second spatial parameter of the virtual camera can be defined as the N spatial parameter. Alignment It can be determined whether the setting position of the virtual camera corresponds to the two-dimensional image, and the two-dimensional image is related to the vertebra V0 or V2 in FIG. 4 or FIG. 6, Alignment The calculated virtual camera has N M spatial parameters, where N and M are integers and M is less than N. Alignment If the set position of the virtual camera corresponds to the two-dimensional image, the processor 1120 of the computing device 1100 calculates the NM spatial parameters as Alignment The N spatial parameters are defined as the first spatial parameters of the virtual camera, and the two-dimensional image is related to the vertebra V0 or V2 in FIG. 4 or FIG. 6. In particular, the N spatial parameters are Not aligned The navigation system 1000 corresponds to a virtual camera. Alignment The NM space parameters corresponding to the virtual camera can be found, and the NM space parameters are Not aligned It is used by the virtual camera and contributes to subsequent steps.

[0142] In some embodiments, the processor 1120 further comprises a first Not aligned defining a second spatial parameter of the virtual camera as an N spatial parameter; Alignment determining whether the set position of the virtual camera corresponds to a first two-dimensional image of the two-dimensional image data set, Alignment The virtual camera has N+M spatial parameters, where N and M are integers and M is less than N, and the first Aligned If the virtual camera position corresponds to the first 2D image in the 2D image dataset, then the N+M spatial parameters are Aligned and defining the first spatial parameter of the virtual camera.

[0143] For example, the processor 1120 of the computing device 1100 may: Not aligned The second spatial parameter of the virtual camera can be defined as the N spatial parameter. AlignmentIt is determined whether the set position of the virtual camera corresponds to the two-dimensional image, and the two-dimensional image is related to the vertebra V0 or V2 in FIG. 4 or FIG. 6, Alignment The calculated virtual camera has N+M spatial parameters, where N and M are integers and M is less than N. Aligned When the virtual camera position corresponds to a two-dimensional image, the processor 1120 of the computing device 1100 calculates N+M spatial parameters as Aligned It can be defined as the first spatial parameter of the virtual camera. The two-dimensional image relates to the vertebral body V0 or V2 in FIG.

[0144] It should be noted that the present application is not limited to the operation of the embodiment shown in FIG. 15, which is only used to exemplify one implementation of the present application.

[0145] As mentioned above, the realignment of step 5600 of method 5000 in Figure 12 includes, but is not limited to, the two realignment procedures in Figures 13 and 15. To facilitate understanding of the realignment procedures in Figures 13 and 15, please refer to Figure 16, which is a flowchart of a method 8000 for realignment according to one embodiment of the present application.

[0146] See also Figures 2 and 16. In operation, the processor 1120 executes step 8200 to generate a digitally reconstructed radiographic image (DRR) acquired by the AP virtual camera and a digitally reconstructed radiographic image (DRR) acquired by the LA virtual camera. Alignment Determine whether one of the digitally reconstructed radiographs (DRR) acquired by the AP virtual camera or the LA virtual camera has been completed. Alignment If it is determined that the AP virtual camera has not been Alignment If not, the processor 1120 executes step 8300 to Alignment Based on the LA virtual camera Not aligned13 is similar to step 8300 in method 8000. The digitally reconstructed radiograph (DRR) acquired by the AP virtual camera or the LA virtual camera is Alignment If not, method 6000 Alignment Find and use a pre-installed virtual camera, Not aligned The virtual camera is reset, and then the processor 1120 executes step 8700 to perform original view registration using the reset virtual camera. Note that the processor 1120 executes step 8400 to adjust the ROI region to avoid interference.

[0147] After performing step 8200, the digitally reconstructed radiographic images (DRR) acquired by the AP virtual camera and the LA virtual camera are both Alignment If it is determined that the AP virtual camera and the LA virtual camera are not, the method 8000 continues with step 8500. In particular, Alignment If not, AP and LA digitally reconstructed radiographs (DRR) corresponding to the first vertebral body Alignment The processor 1120 executes step 8500 to find another AP virtual camera and another LA virtual camera corresponding to the second vertebra, and the digitally reconstructed radiographs (DRRs) of the virtual cameras are compared with the three-dimensional image dataset. Alignment Thereafter, the processor 1120 executes the Alignment AP virtual camera and Alignment Based on the spatial parameters or data of the LA virtual camera, Not aligned AP Virtual Camera and Not aligned The LA virtual camera can be repositioned. Method 7000 in FIG. 15 is similar to step 8500 in method 8000. The digitally reconstructed radiographs (DRRs) acquired by the AP virtual camera and the LA virtual camera corresponding to the first vertebra can be repositioned. Alignment If the first vertebra fails, the method 7000 selects the second vertebra corresponding to the first vertebra. Alignment AP virtual camera and AlignmentFind and use the spatial parameters or data of the previously acquired LA virtual camera to find the location of the first vertebra. Not aligned AP Virtual Camera and Not aligned The LA virtual camera is reset, and then the processor 1120 executes step 8700 to perform original view registration using the reset virtual camera. Note that the processor 1120 executes step 8600 to adjust the region of interest to avoid interference.

[0148] As can be seen from the above-described embodiments of the present application, the present application has the following advantages: Alignment The method and navigation system of the present application can pre-store a three-dimensional image dataset of the region of interest in a database, and then only need to take two X-ray images (two-dimensional images) of the patient (region of interest) during surgery to establish the relationship between the two-dimensional image dataset and the three-dimensional image dataset. The method and navigation system of the present application can then provide accurate navigation during surgery using the pre-stored three-dimensional image dataset. Because the method and navigation system of the present application only need to take two X-ray images (two-dimensional images) of the region of interest, the radiation exposure of the patient (region of interest) can be reduced by more than 98%. As described above, the method and navigation system of the present application can perform two-dimensional and three-dimensional imaging in a more accurate and efficient manner. Alignment can be executed.

[0149] In the above embodiments, specific examples of the present application are disclosed, but are not intended to limit the present application. Those skilled in the art can make various changes and modifications without departing from the principles and spirit of the present application. Therefore, the scope of protection of the present application should be based on that defined in the attached patent application scope.

Claims

1. 1. A method for registering a two-dimensional image dataset and a three-dimensional image dataset of a region of interest, comprising: When a three-dimensional reconstruction image generated by the three-dimensional image dataset is not aligned with the two-dimensional image dataset, acquiring spatial parameters of an aligned virtual camera and defining a first vector based on the spatial parameters of the aligned virtual camera in a coordinate system of the three-dimensional image dataset, wherein the aligned virtual camera acquires a first two-dimensional image dataset of the region of interest at a first viewpoint, and the spatial parameters of the aligned virtual camera correspond to a first three-dimensional reconstruction image dataset aligned with the first two-dimensional image dataset; transforming the first vector of the registered virtual camera by at least one transformation matrix to obtain a first transformation vector of an unregistered virtual camera in the coordinate system of the three-dimensional image dataset, wherein the unregistered virtual camera acquires the two-dimensional image dataset of the area of ​​interest at a second perspective different from the first two-dimensional image dataset; defining, in the coordinate system of the three-dimensional image dataset, a focal point of the unaligned virtual camera to be located at a reference point of the two-dimensional image dataset, the reference point being located at a center point of a calibrator module within the two-dimensional image dataset in the coordinate system of the three-dimensional image dataset; repositioning the unaligned virtual camera with first spatial parameters corresponding to the coordinate system of the three-dimensional image dataset based on the first transformation vector and the focal point of the unaligned virtual camera to align the two-dimensional image dataset with the three-dimensional image data to generate an updated reconstructed image based on the repositioning of the unaligned virtual camera; A method comprising:

2. The method of claim 1 , wherein the first vector is from the position of the aligned virtual camera to the focal point of the aligned virtual camera.

3. The method described in claim 1, wherein the spatial parameters of the aligned virtual camera include Z-axis and Y-axis vectors and a focus FP, and the first vector is defined as the Z-axis of the aligned virtual camera in the coordinate system of the three-dimensional image dataset.

4. The method comprises: defining a second vector of the aligned virtual camera in the coordinate system of the three-dimensional image dataset, the second vector being defined as a Y-axis of the aligned virtual camera in the coordinate system of the three-dimensional image dataset; obtaining a second transformation vector of the unaligned virtual camera based on the second vector of the aligned virtual camera in the coordinate system of the three-dimensional image dataset; 4. The method of claim 3, comprising:

5. The method of claim 4 , wherein the second vector is directed from a center point to a top edge of the reconstructed image, the reconstructed image being acquired with the aligned virtual camera parallel to the reconstructed image.

6. 2. The method of claim 1, wherein the two-dimensional image dataset comprises a first two-dimensional image and a second two-dimensional image, and the at least one transformation matrix comprises a first matrix, the first matrix being used to transform the coordinate system of the first two-dimensional image to the coordinate system of the second two-dimensional image. generating a first reconstructed image with the unaligned virtual camera repositioned with first spatial parameters corresponding to the coordinate system of the three-dimensional image dataset; calculating a reference similarity measure based on the first reconstructed image and the two-dimensional image data set; generating a second reconstructed image with the unregistered virtual camera repositioned from the first spatial parameters to second spatial parameters corresponding to the coordinate system of the three-dimensional image data set; calculating a comparative similarity measure based on the second reconstructed image and the two-dimensional image data set; comparing the comparison similarity value with the reference similarity value; The method of claim 1 , further comprising: if the comparison similarity score is less than or equal to the reference similarity score, aligning the two-dimensional image data set and the three-dimensional image data set.

8. 1. A navigation system for registering a two-dimensional image dataset and a three-dimensional image dataset of a region of interest, comprising: a memory for storing a plurality of instructions; retrieving the plurality of instructions by the memory; When a three-dimensional reconstruction image generated by the three-dimensional image dataset is not aligned with the two-dimensional image dataset, acquiring spatial parameters of an aligned virtual camera and defining a first vector based on the spatial parameters of the aligned virtual camera in a coordinate system of the three-dimensional image dataset, wherein the aligned virtual camera acquires a first two-dimensional image dataset of the region of interest at a first viewpoint, and the spatial parameters of the aligned virtual camera correspond to a first three-dimensional reconstruction image dataset aligned with the first two-dimensional image dataset; transforming the first vector of the registered virtual camera by at least one transformation matrix to obtain a first transformation vector of an unregistered virtual camera in the coordinate system of the three-dimensional image dataset, wherein the unregistered virtual camera acquires the two-dimensional image dataset of the area of ​​interest at a second perspective different from the first two-dimensional image dataset; defining, in the coordinate system of the three-dimensional image dataset, a focal point of the unaligned virtual camera to be located at a reference point of the two-dimensional image dataset, the reference point being located at a center point of a calibrator module within the two-dimensional image dataset in the coordinate system of the three-dimensional image dataset; and repositioning the unaligned virtual camera with first spatial parameters corresponding to the coordinate system of the three-dimensional image dataset based on the first transformation vector and the focal point of the unaligned virtual camera, to align the two-dimensional image dataset and the three-dimensional image data, and to generate an updated reconstructed image based on the repositioning of the unaligned virtual camera.

9. The navigation system of claim 8 , wherein the first vector is from the position of the aligned virtual camera to the focal point of the aligned virtual camera.

10. The navigation system described in Claim 8, wherein the spatial parameters of the aligned virtual camera include Z-axis and Y-axis vectors and a focus FP, and the first vector is defined as the Z-axis of the aligned virtual camera in the coordinate system of the three-dimensional image dataset.

11. The processor retrieves the instructions from the memory, defining a second vector of the aligned virtual camera in the coordinate system of the three-dimensional image dataset, the second vector being defined as a Y-axis of the aligned virtual camera in the coordinate system of the three-dimensional image dataset; and obtaining a second transformation vector of the unaligned virtual camera based on the second vector of the aligned virtual camera in the coordinate system of the three-dimensional image dataset.

12. The navigation system of claim 11 , wherein the second vector is directed from a center point to an upper edge of the reconstructed image, the reconstructed image being acquired with the aligned virtual camera parallel to the reconstructed image.

13. 9. The navigation system of claim 8, wherein the two-dimensional image data set includes a first two-dimensional image and a second two-dimensional image, and the at least one transformation matrix includes a first matrix, the first matrix being used to transform the coordinate system of the first two-dimensional image to the coordinate system of the second two-dimensional image.

14. The processor retrieves the instructions from the memory, generating a first reconstructed image with the unregistered virtual camera repositioned with first spatial parameters corresponding to the coordinate system of the three-dimensional image data set; calculating a reference similarity measure based on the first reconstructed image and the two-dimensional image data set; generating a second reconstructed image with the unregistered virtual camera repositioned from the first spatial parameters to second spatial parameters corresponding to the coordinate system of the three-dimensional image data set; calculating a comparative similarity measure based on the second reconstructed image and the two-dimensional image data set; comparing the comparison similarity value with the reference similarity value; and if the comparison similarity score is less than or equal to the reference similarity score, registering the two-dimensional image data set and the three-dimensional image data set.

Images