2D / 3D image alignment method and surgical robot system for performing the same
The image alignment method addresses the challenge of aligning 3D and 2D surgical images by calculating rotation angles and transformation matrices, ensuring precise surgical navigation and planning despite patient movements.
Patent Information
- Application Number
- JP2023186147
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2022-11-02
- Filing Date
- 2023-10-31
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2043-10-31
AI Technical Summary
Current image registration techniques in surgical navigation face challenges in accurately aligning pre-operative 3D images with intra-operative 2D images due to patient movements, such as joint rotations, leading to inconsistent image alignment and prolonged processing times.
An image alignment method that includes obtaining 3D and 2D images, determining rotation angles using geometric spatial relationships, and calculating transformation matrices to compensate for motion caused by joint rotations, utilizing a surgical robot system for real-time alignment and navigation.
The method provides accurate and rapid image alignment, enabling precise surgical planning and navigation by compensating for patient movements, thereby improving surgical accuracy and efficiency.
Smart Images

Figure 0007577378000031 
Figure 0007577378000032 
Figure 0007577378000033
Abstract
Description
[Technical field]
[0001] The present invention relates to image registration, and more particularly to an image registration method and surgical robotic system that can display surgical planning and navigation information in a desired image through registration between pre-operative 3D images and intra-operative 2D images. [Background technology]
[0002] Image registration-based surgical navigation techniques have been used to assist surgeons in surgery.
[0003] Before surgery, the surgeon establishes a surgical plan based on 3D CT images of the surgical site to determine the optimal implant product, surgical position of the implant, trajectory of the surgical tools, etc. During surgery, the surgeon operates the surgical tools while checking the real-time positions of the surgical tools and implants that correspond to the current state of the surgery and comparing them with the surgical plan to check whether the surgery is going well according to the surgical plan.
[0004] While surgical planning is established based on 3D CT images, which are primarily used before surgery, the current state of surgery is provided based on 2D images from the C-arm, which is the imaging device primarily used during surgery, because the surgical tools and the C-arm during surgery are aligned to the same coordinate system.
[0005] Therefore, in order to provide integrated information for surgical planning and the current state of surgery, 3D / 2D images should be aligned, and the accuracy of image alignment must be improved and the processing time of image alignment must be shortened for successful surgery.
[0006] However, depending on the patient's movements, spinal rotation, asymmetry between the left and right pelvis, etc. occur, and the images obtained are not consistent each time they are acquired. Therefore, even if the patient does not move, even if there is no movement in position, minute changes in pose that occur due to joint rotation, etc., should be quickly processed for image matching, but in the conventional navigation technology field, image matching technology that reflects such changes in pose due to joint rotation has not been utilized. [Prior art documents] [Patent documents]
[0007] [Patent Document 1] Korean Patent No. 2203544 [Patent Document 2] Korean Patent No. 2394901 Summary of the Invention [Problem to be solved by the invention]
[0008] Therefore, an object of the present invention is to provide an image alignment method capable of rapid image alignment processing and compensation for movement caused by joint rotation, as well as a surgical robot system, an image alignment device, and a computer program medium for carrying out the same. [Means for solving the problem]
[0009] In one aspect of the present invention, an image matching method includes the steps of: obtaining a 3D image of a surgical site of a patient from a 3D image device before surgery; extracting DRR images in an AP direction and an LL direction from the 3D image; obtaining 2D images of an AP image and an LL image of the surgical site of the patient from a 2D image device during surgery; and calculating, from a geometric space relationship of a source and a detector to the DRR image, a first rotation angle between a reference position of the surgical site of the patient corresponding to a first reference position of the AP image or the LL image and the first reference position, the first rotation angle being parallel to a cross product vector of a first normal vector to a plane of the AP image and the LL image and passing through a first predetermined origin. determining a second rotation angle between a second reference position of the AP image or the LL image corresponding to the reference position and the reference position based on a second rotation axis that is parallel to a cross product vector of a second normal vector to the plane of the AP image and the LL image and passes through the predetermined second origin from the geometric spatial relationship of the source and the detector with respect to the 2D image; and determining a transformation relationship between the 2D image and the DRR image based on the first rotation angle and the second rotation angle from the geometric spatial relationship of the DRR image and the 2D image with respect to the source and the detector, respectively.
[0010] Here, the first reference position and the second reference position may be the center of the AP image or the LL image for the 2D image and the DRR image, respectively, or a line or a plane including the center.
[0011] The image matching method further includes a step of performing a surgical plan based on the 3D image, and the first origin for the DRR image can be determined through a relative relationship of an implant mounting position or a surgical tool trajectory for mounting the implant applied to the surgical plan, and the reference position for the DRR image or the 2D image can be determined by a user input.
[0012] In the image matching method, the geometric spatial relationship of the source and the detector to the DRR image may be an orthogonal projection relationship, and the geometric spatial relationship of the source and the detector to the 2D image may be a perspective projection relationship.
[0013] The image alignment method may further include determining a first volume of interest where a plane intersects with the movement of the AP image plane and the LL image plane in the direction of the first normal vector from the geometric spatial relationship of the source and the detector with respect to the DRR image, and determining a second volume of interest where a plane intersects with the movement of the AP image plane and the LL image plane in the direction of the second normal vector within a perspective projection range, where the geometric spatial relationship of the source and the detector with respect to the 2D image is a perspective projection relationship, wherein the first origin may be a center point of the first volume of interest, and the second origin may be a center point of the second volume of interest.
[0014] The image matching method may further include a step of determining a first region of interest for each of the AP image and the LL image of the DRR image, and a step of determining a second region of interest corresponding to the first region of interest for each of the AP image and the LL image of the 2D image, where the first reference position is located within the first region of interest and the second reference position is located within the second region of interest. The method may further include a step of determining a first volume of interest in which a plane intersects with a movement of the region of interest of the AP image and the region of interest of the LL image in the direction of the first normal vector from the geometric spatial relationship of the source and the detector with respect to the DRR image, and a step of determining a second volume of interest in which a plane intersects with a movement of the region of interest of the AP image and the region of interest of the LL image in the direction of the second normal vector within a perspective projection range, where the geometric spatial relationship of the source and the detector with respect to the 2D image is a perspective projection relationship, where the first origin may be a center point of the first volume of interest, and the second origin may be a center point of the second volume of interest.
[0015] In the image alignment method, the first origin may be a midpoint between target positions of the patient's spinal pedicle screws, and the first rotation angle may be an angle formed by a line segment connecting the midpoint between the pedicle screw entry points and the first origin, with the first origin as the center, and the first normal vector passing through the center point of the volume of interest.
[0016] The image matching method includes a first step of calculating a first intersection point between an epipolar line on the LL image for each vertex of the region of interest of the AP image and a midline connecting the epipolar line on the LL image and a midpoint of the periphery or side of the region of interest of the LL image for the DRR image, a second step of orthogonally projecting the first intersection point onto the normal vector from each vertex of the region of interest of the AP image to obtain four restoration points, a third step of calculating a second intersection point between the epipolar line on the AP image for each vertex of the region of interest of the LL image and a midline connecting the epipolar line on the AP image and a midpoint of the periphery or side of the region of interest of the AP image for the LL image, a fourth step of orthogonally projecting the second intersection point onto the normal vector from each vertex of the region of interest on the LL image to obtain four restoration points, and a fifth step of calculating a first hexahedral volume of interest formed based on the eight restoration points obtained through the first to fourth steps.
[0017] The image alignment method includes a first step of calculating a first intersection point of an epipolar line on the LL image for each vertex of a region of interest in the AP image and a midline connecting the epipolar line on the LL image with a midpoint of a perimeter or a side of the region of interest in the LL image, for the 2D image; a second step of perspectively projecting the first intersection point onto a perspective projection vector from each vertex of the region of interest in the AP image toward the source to obtain four restoration points; a third step of calculating a second intersection point of an epipolar line on the AP image for each vertex of the region of interest in the LL image and a midline connecting the epipolar line on the AP image with a midpoint of a perimeter or a side of the region of interest in the AP image, for each vertex of the region of interest in the LL image; a fourth step of perspectively projecting the second intersection point onto a perspective projection vector from each vertex of the region of interest in the LL image toward the source to obtain four restoration points; and a fifth step of determining a hexahedral second volume of interest based on the eight restoration points obtained in the first to fourth steps.
[0018] Further, the object of the present invention is to provide an image matching method according to another aspect of the present invention, comprising the steps of: obtaining a 3D image of a surgical site of a patient from a 3D image device before surgery; extracting DRR images in AP and LL directions from the 3D image; obtaining 2D images of an AP image and an LL image of the surgical site of the patient from a 2D image device during surgery; determining a first region of interest for each of the AP image and the LL image of the DRR image; determining a second region of interest corresponding to the first region of interest for each of the AP image and the LL image of the 2D image; determining a first volume of interest formed by intersecting planes when translating the region of interest of the AP image and the region of interest of the LL image in the direction of a first normal vector to the planes of the AP image and the LL image from the geometric spatial relationship of a source and a detector with respect to the DRR image; This can be achieved by an image matching method comprising the steps of: determining a second volume of interest formed by the intersection of a plane when moving a region of interest of the AP image and a region of interest of the LL image in the direction of a line vector; determining a first displacement between a first reference position in the first volume of interest corresponding to a first predetermined reference position in the first region of interest and a predetermined reference position corresponding to the first reference position; determining a second displacement between a second reference position in the second volume of interest and the reference position relative to a second predetermined reference position in the second region of interest corresponding to the reference position; and determining a transformation relationship between the 2D image and the DRR image from a geometric spatial relationship of the DRR image and the 2D image to the source and detector, respectively, based on the first displacement and the second displacement, wherein the transformation relationship is determined such that a Euclidean distance between a vertex of the first region of interest and a vertex of the second region of interest is minimized according to the transformation relationship.
[0019] Here, the step of determining the first displacement includes a step of determining an angle between the reference position and the first reference position to be a first rotation angle based on a first rotation axis that is parallel to a cross product vector of the first normal vector with respect to the plane of the AP image and the LL image and passes through a predetermined first origin, and the step of determining the second displacement includes a step of determining an angle between the reference position and the second reference position to be a second rotation angle based on a second rotation axis that is parallel to a cross product vector of the second normal vector with respect to the plane of the AP image and the LL image and passes through a predetermined second origin.
[0020] And, the steps of determining the first and second volumes of interest may each include projecting epipolar lines of vertices of the first and second regions of interest onto the first and second normal vectors to form a polyhedron.
[0021] Furthermore, the above object can be achieved by an image matching device according to another aspect of the present invention, which includes a processor for executing the image matching method according to any one of claims 1 to 17.
[0022] Furthermore, the above object can be achieved by a surgical robot system according to another aspect of the present invention, comprising a 2D imaging device for obtaining 2D images of a surgical site of a patient during surgery, a robot arm including an end effector to which a surgical tool can be attached, a position sensor for sensing the real-time position of the surgical tool or the end effector, a controller for controlling the robot arm in accordance with a predetermined surgical plan, a display, and a navigation system for performing the image alignment method described in any one of claims 1 to 17, and displaying, via the display, the planning information for a surgical tool or an implant in a 2D image obtained during surgery, and displaying the real-time position of the surgical tool or implant in the 2D image or a 3D image obtained before surgery.
[0023] Furthermore, the above object can be achieved by a computer program medium storing software for performing the image matching method according to any one of claims 1 to 17 according to an aspect of the present invention. Effect of the Invention
[0024] The present invention provides an accurate and fast image registration process that allows for motion compensation during rotation relative to a patient's body part. [Brief description of the drawings]
[0025] [Figure 1] FIG. 1 is a block diagram of an image alignment apparatus according to an embodiment of the present invention. [Diagram 2] 3 is a flowchart of an image alignment method according to an embodiment of the present invention. [Figure 3a] 1 is an exemplary diagram for explaining a process of establishing a surgical plan in a CT image and extracting a volume of interest therefrom. [Figure 3b] 1 is an exemplary diagram for explaining a process of establishing a surgical plan in a CT image and extracting a volume of interest therefrom. [Figure 4] FIG. 13 is an exemplary diagram for explaining generation of a DRR image based on orthogonal projection from a CT image. [Diagram 5] FIG. 13 is an exemplary diagram illustrating a process of automatically generating a region of interest after labeling through machine learning in an AP image. [Figure 6a] FIG. 13 is a diagram showing an example of a result in which a first region of interest is generated in an AP image and an LL image as a DRR image. [Figure 6b] FIG. 13 is a diagram showing an example of a result in which a first region of interest is generated in an AP image and an LL image as a DRR image. [Figure 7a] FIG. 13 is a diagram showing an example of the results when a second region of interest is generated in an AP image and an LL image as a C-arm image. [Figure 7b]FIG. 13 is a diagram showing an example of the results when a second region of interest is generated in an AP image and an LL image as a C-arm image. [Figure 8] FIG. 13 is a diagram showing an example of a user interface that enables size adjustment, position movement, rotational movement, SP setting, etc. of a region of interest. [Figure 9a] 11A to 11C are diagrams for explaining a process in which vertices of a first region of interest are restored within a CT volume. [Figure 9b] 11A to 11C are diagrams for explaining a process in which vertices of a first region of interest are restored within a CT volume. [Figure 9c] 11A to 11C are diagrams for explaining a process in which vertices of a first region of interest are restored within a CT volume. [Figure 9d] 11A to 11C are diagrams for explaining a process in which vertices of a first region of interest are restored within a CT volume. [Figure 10] 13 is an exemplary diagram for explaining a result in which eight vertices of a first region of interest are restored within a CT volume. FIG. [Figure 11] 13A to 13C are diagrams illustrating a process in which vertices of a second region of interest are restored within a C-arm volume. [Figure 12] FIG. 13 is a diagram for explaining the result of eight vertices of the second region of interest being restored within a C-arm volume. [Figure 13a] 13A and 13B are diagrams for explaining a case where the axial direction of the first volume of interest coincides with the tip line of the spinous process and a case where the axial direction does not coincide with the tip line of the spinous process and is rotated. [Figure 13b] 13A and 13B are diagrams for explaining a case where the axial direction of the first volume of interest coincides with the tip line of the spinous process and a case where the axial direction does not coincide with the tip line of the spinous process and is rotated. [Figure 14] FIG. 13 is an exemplary diagram for explaining a method for determining a first rotation angle based on a position of a planning object for which a surgical plan is calculated before a surgical operation. [Figure 15] 13 is an exemplary diagram for explaining a user interface for a user to set the tip line of a spinous process in a C-arm AP image. FIG. [Figure 16] FIG. 13 is an exemplary diagram illustrating a process of calculating a second rotation angle based on a C-arm AP image. [Figure 17] FIG. 13 is an exemplary diagram illustrating a process of calculating a second rotation angle based on a C-arm AP image. [Figure 18] 1 is a diagram for explaining a local coordinate system V whose origin is the center of a first volume of interest. FIG. [Figure 19] 1 is a schematic diagram of a general configuration of a surgical robot system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0026] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.
[0027] FIG. 1 is a block diagram showing a schematic configuration of an image alignment device 10 for performing image alignment according to an embodiment of the present invention. The image alignment device 10 is an electronic device capable of performing image alignment, and refers to computing devices such as computers, notebook computers, laptop computers, tablet terminals, smartphones, mobile phones, PMPs (Personal Media Players), and PDAs (Personal Digital Assistants). The image alignment method according to an embodiment of the present invention is applicable to medical image processing software, including medical navigation software. The image alignment device 10 according to an embodiment of the present invention can execute image processing software, such as surgery planning and planning, in addition to medical navigation software.
[0028] Referring to FIG. 1, an image registration device 10 according to one embodiment of the present invention includes a memory 11 in which data and program code are stored, and a processor 13 for executing the same.
[0029] The memory 11 is a computing device readable recording medium and can store at least one computer program code to be executed by the processor 13. Such computer program code can be loaded into the memory 11 from a separate floppy drive, disk, tape, DVD / CD-ROM drive, memory card, etc. The memory 11 can store software for image registration, patient medical images and data, etc.
[0030] The processor 13 is for executing and processing instructions of a computer program by performing basic logic, calculations, operations, etc., and the computer program code stored in the memory 11 is loaded into and executed by the processor 13. The processor 13 executes algorithms stored in the memory 11 to perform a series of 2D / 3D image matching.
[0031] 1, an image registration device 10 according to an embodiment of the present invention may further include a display 15 for displaying data and images, a user interface 17 for receiving user input, and a communication interface 19 for linking with the outside. The user interface 17 may include input devices such as a microphone, a keyboard, a mouse, or a foot pad.
[0032] An image matching method performed by image matching device 10 (or processor 13) according to one embodiment of the present invention will be described below.
[0033] FIG. 2 is a flow chart of an image matching method according to one embodiment of the present invention, and FIGS. 3 to 14 are schematic diagrams illustrating steps of the flow chart of FIG.
[0034] Referring to FIG. 2, an image registration method according to an embodiment of the present invention includes a pre-operative planning step.
[0035] First, a CT image of the surgical site of a patient before surgery is taken through an imaging device to obtain a 3D image (S1). Here, imaging devices other than CT that obtain 3D images, such as MRI, can also be used, and the present invention is not limited to a specific imaging device.
[0036] The surgeon uses surgical planning software to establish a surgical plan for the 3D image of the patient (S2). For example, in the case of a surgery in which a screw is inserted into a pedicle for fixation during spinal surgery, the surgeon can select a screw product based on the diameter, length, material, etc. of the screw, set the screw's entry point into the pedicle, and set the target of the screw's end fixation position, and display them on the 3D image. Many companies provide such planning software in various surgical fields.
[0037] Next, the image registration device 10 extracts a volume of interest for the surgical site for which the surgical planning has been established (S3). Such extraction can be performed automatically via an automated algorithm based on the planning objects displayed on the 3D image, e.g., the positions of the screws, or the volume boundary can be adjusted and extracted by its size directly provided by the physician.
[0038] Referring to Figure 3a, the doctor can set or adjust the volume of interest by controlling the white box provided on the 3D image on the left, and referring to Figure 3b, compared to Figure 3a, the size of the left and right margins (a, b) can be given as a default based on, for example, the screw, which is the planning object, so that they are automatically extracted.
[0039] From the volume of interest extracted in this way, an AP image (Anterior-Posterior image) and an LL image (Lateral-Lateral image) are generated as DRR (Digitally Reconstructed Radiograph) images (S4). That is, as shown in Fig. 4, the AP image and the LL image are virtual C-arm images based on the CT coordinate system provided by the CT device, and the AP image is generated from the AP source and detector by orthogonal projection, and the LL image is generated from the LL source and detector.
[0040] Next, AP and LL images of the surgical site are obtained through the C-arm device during surgery (S5), and the C-arm device is aligned to a spatial coordinate system based on a marker attached to a part of the patient's body (hereinafter referred to as a "PM marker") or other marker that can be referenced in the surgical space (S6). The applicant's Korean Patent No. 2203544 discloses a technique for aligning a C-arm device in 3D space or aligning a 2D image in 3D space, and this application incorporates the entirety of the '544' patent by reference. Numerous techniques for spatially aligning a C-arm device are known other than the referenced '544' patent, and the present invention is not limited to a specific spatial alignment technique.
[0041] The image alignment device 10 determines a first region of interest ROI and a second region of interest ROI for the DRR image and the C-arm image, respectively (S7, S8). In the case of pedicle screw fixation surgery, each region of interest can be set in units of vertebrae, and as shown in FIG. 5, the regions of interest can be extracted and labeled in units of vertebrae through machine learning, and the regions of interest can be defined as rectangles including each vertebra.
[0042] 6a and 6b show the results of a first region of interest being extracted in the AP and LL images of the labeled DRR image, and FIGS. 7a and 7b show the results of a second region of interest being extracted in the labeled C-arm AP and LL images.
[0043] It should be noted here that the first region of interest for the DRR image and the second region of interest for the C-arm image are extracted equally to each other. At least, the vertices of the rectangles of the regions of interest are selected as points that correspond to each other and have equivalence. Equivalence does not mean perfect matching points, but means that, for example, the first region of interest and the second region of interest are extracted or selected so that the relationship between the image features of the vertebrae and the four vertices of the selected region of interest is maintained constant.
[0044] In this embodiment, for example, a first region of interest and a second region of interest are selected for a vertebra such that the tip of the spinous process is centered in the region of interest for the AP image, and the outer margins of the vertebra are selected to be uniformly applied to the first and second regions of interest in each of the AP / LL images.
[0045] Referring to FIG. 8, the physician can size, position, rotate, etc. the extracted region of interest via the provided user interface, and can also adjust the region of interest to generate a line pointing to a particular body part, e.g., the tip of a spinous process, or to coincide with a center line.
[0046] Next, the image alignment device 10 restores the first region of interest displayed in the AP image and the LL image to the DRR image in space, i.e., restores it in a volume of interest (S9). Intuitively, the first volume of interest is defined as the space where the two planes intersect when the AP image is assumed to be at a virtual detector position and translated along the normal vector, and the LL image is assumed to be at a virtual detector position and translated along the normal vector of the LL image, i.e., the space where the first region of interest on the two planes intersect with each other.
[0047] The process of calculating the first volume of interest based on a coordinate system will be described with reference to FIGS.
[0048] 9, for a first region of interest for a vertebra labeled L3, a first vertex A1 of the first region of interest on the AP image and a virtual detector plane on which the AP image is located on the CT coordinate system are defined. A virtual detector plane on which the LL image is located is also defined, and the first region of interest on the LL image is also displayed. Intuitively, when a volume formed by the intersection of the planes of the AP image and the LL image in space is called a CT volume, the CT volume is a hexahedron having six boundary surfaces and a three-axis CT coordinate system.
[0049] Since the DRR images are orthogonal or orthogonal projections, such a hexahedral CT volume is formed, and the normal vector passing through A1 crosses the upper and lower boundary surfaces of the CT volume to reach A'1 at an arbitrary height.
[0050] Therefore, the intersection points I1 and I2 between A'1 and the upper and lower boundary surfaces can be obtained as follows.
[0051]
number
[0052]
number
[0053]
number
[0054] where λ1 is an arbitrary number and N AP is the normal vector of the AP plane, and f inter is represented by two points and one plane as input variables, and represents a function that finds the intersection point between the line segment connecting the two points and the plane.
[0055] Therefore, I1 can be calculated as follows:
[0056]
number
[0057] Here, P Top is π Top where I2 is calculated in the same manner as I1.
[0058] Referring to FIG. 9b, when arbitrary points on arbitrary normal lines perpendicularly extending from I1 and I2 down to the LL image plane are called I'1 and I'2, respectively, I'1 and I'2 can be expressed as follows.
[0059]
number
[0060]
number
[0061] where λ2 is an arbitrary number and N LL is the normal vector of the LL plane.
[0062] Referring to the following Figure 9c, the epipolar line of vertex A1 can be obtained by connecting the line segments from I1 and I2 to I'1 and I'2 and the intersection point with the LL image plane. The intersection point of this epipolar line with any position of the region of interest on the LL image can be obtained. In Figure 9c, the intersection point C3 with the line segment connecting the midpoints C1 and C2 of the left and right sides of the first region of interest is obtained.
[0063] If the intersection C3 between the epipolar line and the region of interest thus obtained is projected again onto the normal A1-A'1 from the vertex A1, P1 can be obtained as shown in Fig. 9d. If the position of the region of interest intersecting with the epipolar line is taken as the upper side, P1 is located at a position parallel to the upper side of the first region of interest in the LL image. Note that P1, which is the point in the CT volume corresponding to A1, can be selected at any position on the normal from A1 within the positions parallel to the upper and lower sides of the region of interest on the LL image.
[0064] Applying A1 to the remaining three vertices, we can see that the plane of the first region of interest on the AP image is transferred to the CT volume and restored, and similarly, the vertices on the LL image are transferred to the CT volume and restored.
[0065] FIG. 10 shows that the eight vertices A1 to A8 that define the first region of interest are eight points ( CT P1, CT P2,.... CT A restored example is shown on page 8.
[0066] FIG. 11 shows the relationship in which one vertex A1 defining the second region of interest in the C-arm image is restored to the C-arm volume.
[0067] Compared to Figures 9a to 9d, since the C-arm images are obtained according to perspective projection, instead of being defined using unit vectors on the mutually orthogonal CT coordinate system, the vector between the source and detector is defined as a direction vector based on the PM marker-based PM coordinate system.
[0068] Therefore, in FIG. 11, I1, I2, I'1, I'2, C3, and P1 can be calculated as follows.
[0069]
number
[0070]
number
[0071]
number
[0072]
number
[0073]
number
[0074]
number
[0075] Here, S LL represents the source location of the LL image, λ2 is an arbitrary number,
number
[0076] When the same procedure is repeated for the eight vertices A1 to A8, the eight points ( PM P1, PM P2,.... PM P8) is restored to the C-arm volume.
[0077] 10 and 12, the eight points restored in the CT volume and the eight points restored in the C-arm volume appear to be arranged such that the regions of interest on the AP image and the LL image intersect at their centers. It can be seen that these eight points form the midline of a hexahedron, forming one volume of interest.
[0078] Referring again to the flowchart of FIG. 2, the image alignment device 10 restores the eight points or the first and second volumes of interest in the CT volume and C-arm volume, respectively, and then calculates the rotation angle between the reference position of the patient's surgical site and its corresponding reference position around a predetermined axis (S10).
[0079] In this embodiment, the reference position is the tip of the spinous process, and the corresponding first reference position corresponds to the normal vector of the AXIAL image, which is defined in the same direction as the vector of the cross product of the normal vector of the AP image plane and the normal vector of the LL image plane. However, the first origin that determines the position of the rotation axis should ideally reflect the center of rotation of the vertebrae, but since it is difficult to define this, one of the two is selected, as described later.
[0080] First, looking at FIG. 13a, the tip of the spinous process is designated SP, and referring to FIG. CT P5 and CT The line segment between P6 differs only in height, with no deviation to the left or right, so in this case we can see that there is no rotation.
[0081] On the other hand, in Fig. 13b, the tip of the spinous process indicated by SP and CT P5 and CT It can be seen that the line segment between P1 and P6 has a deviation to the left and right, and the angle θ with the axial axis passing through the center of the first volume of interest is distorted.
[0082] In this way, in order to obtain the distorted rotation angle θ (hereinafter referred to as the “first rotation angle”), a planning object made by a doctor before surgery can be used as shown in FIG. 14. Referring to FIG. 14, an approach path E in which a pair of screws enter the left and right pedicles based on the axial axis can be used. r , E l The midpoint E between c and the final position where the end of the screw is attached, T L , T r (Hereinafter referred to as "target") midpoint T c Line E connecting c -T c and a vertical line segment passing through the center of the axial plane of the first volume of interest, i.e., PT P6- PT The angle between the line segment P and P8 is the first rotation angle θ DRR and calculate it.
[0083] In this case, the first origin that the first rotation axis passes through is the left and right targets T1 and T r Midpoint T c It can be seen that the height difference with the center of the first volume of interest is d. This makes it easy to calculate the rotation value according to the doctor's surgical planning, and helps to perform quick alignment processing. As explained in the rotation of C-arm images, the first rotation origin can be the center of the first volume of interest instead of the target weight.
[0084] On the other hand, as shown in Fig. 15, since true AP and LL images of a C-arm image can be obtained, the center of the second region of interest in the AP image is often aligned with the tip of the spinous process, but a user interface can be provided so that the doctor can move the center line of the second region of interest on the AP image to match the tip of the spinous process in order to directly set the tip of the spinous process in the current step. This can be similarly applied to the DRR image.
[0085] As shown in Figure 16, the tip of the spinous process set by the user is P 12 If the second rotation angle of the C-arm image is parallel to P5-P6, the line P9-P8 of the tip of the spinous process of the input end projected at the height of P6 is the center of the intersection line of the plane formed by P1-P4 and the plane formed by P5-P8. 10 The angle between the P5-P6 line segment and the P5-P6 line segment can be calculated using the second rotation angle.
[0086]
number
[0087]
number
[0088]
number
[0089]
number
[0090] Here, P1 to P8 respectively indicate the points where the eight vertices of the second region of interest are restored, and P9 to P 10 is the user-specified spinous process tip line.
[0091] It should be noted that FIG. 16 illustrates a case where the horizontal plane formed by P1 to P8 and the vertical plane formed by P5 to P8 do not intersect with each other except at the intersection line.
[0092] Figure 17 shows the eight points reconstructed in the CT volume. CT P1~ CT P8 is the first rotation angle θ calculated in FIG. DRR Rotated only CT P'1~ CT The relationship between the points P'8 is shown. This shows the position of the point reflecting the rotation of the spine based on the tip of the spinous process after restoring the first region of interest in the CT coordinate system to the CT volume. Similarly, the second region of interest in the C-arm image is calculated in the same way to reflect the patient's spinal rotation during surgery ( PM P'1~ PM P'8) can be obtained.
[0093] The transformation relationship between the PM coordinate system and the CT coordinate system is expressed as the rotation point, i.e. PM P'1~ PM P'8 and CT P'1~ CT When applied to P'8, their Euclidean distance should ideally be zero. The optimal solution is said to be that the optimal matching is achieved when the sum or average of the Euclidean distances between the eight corresponding points is the smallest, and the objective of the initial matching is to find a transformation matrix that satisfies this (S11).
[0094]
number
[0095]
number
[0096] 18, the conversion relationship from the CT coordinate system to the V local coordinate system will be described. Here, the origin of the V local coordinate system is defined as the midpoint of the intersection of the horizontal plane and the vertical plane as follows.
[0097]
number
[0098]
number
[0099] Therefore, the three axes of the V local coordinate system are V X, V Y, V Z is defined as follows:
[0100]
number
[0101]
number
[0102]
number
[0103]
number
[0104] P in V local coordinate system i is located at V P i is defined as follows:
[0105]
number
[0106]
number
[0107] Then, the rotated point in the V local coordinate system is found as follows:
[0108]
number
[0109] Here, the Rodrigues function is defined as a function that rotates an object by an input rotation angle around an input rotation axis.
[0110] This allows the rotation point of the CT coordinate system CT P'1 is defined as follows:
[0111]
number
[0112] Therefore, we carry out the above procedure for the eight points and obtain the PM coordinate system PM P1 rotated PM By calculating P'1, it can be input into the following formula.
[0113]
number
[0114] After the initial matching is completed by searching for the optimal transformation matrix, the image matching device 10 performs a matching optimization process by deriving the optimal transformation matrix while adjusting the search range of the DRR image (S12). The optimization process is a known technique based on a global search, and therefore a detailed description thereof will be omitted.
[0115] The image matching method as described above has the advantage that the accuracy of image matching can be improved in response to body rotation, and such image matching processing can be rapidly performed.
[0116] The present invention can be embodied in a computer program recording medium having a computer program recorded thereon for executing the image matching method on a computer.
[0117] Furthermore, the present invention can also be embodied by a surgical robot system based on the above-mentioned image matching method.
[0118] Referring to FIG. 19, a surgical robot system 1 according to one embodiment of the present invention includes a C-arm imaging device 100, a surgical robot 200, a position sensor 300, and a navigation system 400, where the surgical robot 200 includes a main body 201, a robot arm 203 with an end effector 203a, and a robot controller 205.
[0119] The C-arm imaging device 100 is used to obtain AP and LL images of a surgical site on a patient during surgery.
[0120] The robot arm 203 is fixedly installed on the robot body 201, and an end effector 203a to which a surgical tool can be attached and detached is provided at the end. The position sensor 300 is embodied as an OTS that tracks the real-time position of the surgical tool or end effector 203a through recognition of a marker. The controller 205 is provided on the robot body 201 and controls the robot arm 203 according to predetermined surgical planning and control software.
[0121] The navigation system 400 executes the above-mentioned image matching method and displays planning information for a surgical tool or an implant on a C-arm image obtained during surgery, or displays a real-time position of a surgical tool or an implant on a C-arm image or a 3D image obtained before surgery, to support a surgeon in performing a surgical procedure. For this purpose, a display that enables a surgeon to visually compare the real-time position of a surgical tool as a surgical phenomenon and a surgical plan during surgery may be further included and connected to the navigation system 400. It can be easily understood by a person skilled in the art that the other components, except for the navigation system 400 in FIG. 19, are commonly used in surgical robot systems.
[0122] Although the embodiments of the present invention have been described above, it will be understood by those having ordinary skill in the art that the embodiments of the present invention can be modified or replaced in whole or in part without departing from the technical spirit of the present invention.
[0123] Therefore, the above-described embodiments are merely illustrative of the present invention, and the scope of protection of the present invention should be considered to cover the invention as defined in the appended claims and their equivalents. The inventions described in the claims at the time of filing of this application are set forth below. [1] 1. A method for image alignment, wherein each step is performed by an image alignment device including a processor, obtaining a 3D image of a surgical site on a patient from a pre-operative 3D imaging device; Extracting DRR images in the AP direction and the LL direction from the 3D image; obtaining 2D images of AP and LL views of the surgical site on the patient from an intraoperative 2D imaging device; determining a first rotation angle between a reference position of the surgical site of the patient corresponding to a first reference position of the AP image or the LL image based on a first rotation axis that is parallel to a cross product vector of a first normal vector to the plane of the AP image and the LL image and passes through a first predetermined origin, based on the geometric spatial relationship of the source and the detector with respect to the DRR image; determining a second rotation angle between a second reference position of the AP image or the LL image corresponding to the reference position and the reference position based on a second rotation axis that is parallel to a cross product vector of a second normal vector to the plane of the AP image and the LL image and passes through the second predetermined origin, based on a geometric spatial relationship of a source and a detector with respect to the 2D image; determining a transformation relationship between the 2D image and the DRR image based on the first rotation angle and the second rotation angle from a geometric spatial relationship of the DRR image and the 2D image to the source and detector, respectively. [2] The image matching method described in [1], characterized in that the first reference position and the second reference position are the center of the AP image or the LL image for the 2D image and the DRR image, respectively, or a line or plane including the center. [3] The image alignment device further includes a step of performing surgical planning based on the 3D image; The image matching method described in [1], characterized in that the first origin for the DRR image is determined via the relative relationship of the implant mounting position applied to surgical planning or the trajectory of the surgical tool for mounting the implant. [4] The image matching method according to [1], characterized in that the reference position for the DRR image or the 2D image is determined by user input. [5] a geometric spatial relationship of the source and the detector to the DRR image is an orthogonal projection relationship; The image matching method described in [1], wherein the geometric spatial relationship of the source and the detector to the 2D image is a perspective projection relationship. [6] determining a first volume of interest from the geometric spatial relationship of the source and the detector to the DRR image that intersects with the AP image plane and the LL image plane as the image alignment device moves in the direction of the first normal vector; The image matching method described in [1], further characterized in that the image matching device determines a second volume of interest in which the geometric spatial relationship of the source and the detector to the 2D image is a perspective projection relationship and in which planes intersect as the AP image plane and the LL image plane are moved in the direction of the second normal vector within the perspective projection range. [7] The image matching method according to [6], characterized in that the first origin is a center point of the first volume of interest, and the second origin is a center point of the second volume of interest. [8] The image alignment device determines a first region of interest for each of the AP image and the LL image of the DRR image; and the image alignment device determines a second region of interest corresponding to the first region of interest for each of the AP image and the LL image of the 2D image; the first reference location is located within the first region of interest; The image matching method according to [1], characterized in that the second reference position is located within the second region of interest. [9] determining a first volume of interest from the geometric spatial relationship of the source and the detector to the DRR image, the first volume of interest being intersected by a plane as the image alignment device moves through the region of interest of the AP image and the region of interest of the LL image in the direction of the first normal vector; The image matching method described in [8], further characterized in that the image matching device determines a second volume of interest in which a plane intersects as the geometric spatial relationship of the source and the detector to the 2D image is a perspective projection relationship and the region of interest of the AP image and the region of interest of the LL image are moved in the direction of the second normal vector within the perspective projection range.
[10] The image matching method according to [9], characterized in that the first origin is a center point of the first volume of interest, and the second origin is a center point of the second volume of interest.
[11] The image alignment method described in [9], wherein the first origin is a midpoint between target positions of the patient's spinal pedicle screws.
[12] The first rotation angle is a line segment that connects the midpoint between the pedicle screw entry points and the first origin, with the first origin as a center; The image matching method according to
[11] , characterized in that the angle is an angle formed by the first normal vector passing through the center point of the first volume of interest.
[13] each of the first regions of interest for the AP image and the LL image of the DRR image is rectangular; Regarding the DRR image, a first step in which the image alignment device calculates a first intersection point between an epipolar line on the LL image for each vertex of a region of interest in the AP image and a midline connecting an outer perimeter or a midpoint of a side of the region of interest in the LL image; a second step in which the image alignment device orthogonally projects the first intersection point onto the normal vectors from each vertex of the region of interest of the AP image to obtain four restoration points; a third step in which the image alignment device determines a second intersection point between an epipolar line on the AP image for each vertex of the region of interest of the LL image and a midline connecting an outer perimeter or a side midpoint of the region of interest of the AP image; a fourth step in which the image alignment device orthogonally projects the second intersection point onto the normal vectors from each vertex of the region of interest on the LL image to obtain four restoration points; The image matching method described in [8], characterized in that the image matching device includes a fifth step of calculating a first hexahedral volume of interest formed based on the eight restoration points obtained through the first to fourth steps.
[14] The step of determining the second volume of interest comprises the steps of: a first step in which the image alignment device calculates a first intersection point between an epipolar line on the LL image for each vertex of a region of interest in the AP image and a midline connecting an outer perimeter or a midpoint of a side of the region of interest in the LL image; a second step in which the image matching device perspectively projects the first intersection point onto a perspective projection vector from each vertex of the region of interest of the AP image toward the source to obtain four restoration points; a third step in which the image alignment device determines a second intersection point between an epipolar line on the AP image for each vertex of the region of interest of the LL image and a midline connecting an outer perimeter or a midpoint of a side of the region of interest of the AP image; a fourth step in which the image matching device perspectively projects the second intersection point onto a perspective projection vector from each vertex of the region of interest on the LL image toward a source to obtain four restoration points; The image matching method described in [8], characterized in that the image matching device includes a fifth step of determining a hexahedral second volume of interest based on the eight restoration points obtained in the first to fourth steps.
[15] 1. A method for image alignment, wherein each step is performed by an image alignment device including a processor, obtaining a 3D image of a surgical site on a patient from a pre-operative 3D imaging device; Extracting DRR images in the AP direction and the LL direction from the 3D image; obtaining 2D images of AP and LL views of the surgical site on the patient from an intraoperative 2D imaging device; determining a first region of interest for each of the AP image and the LL image of the DRR image; determining a second region of interest corresponding to the first region of interest for each of the AP image and the LL image of the 2D image; determining a first volume of interest from a geometric spatial relationship of a source and a detector with respect to the DRR image, the volume of interest being formed by intersecting a plane when translating a region of interest in the AP image and a region of interest in the LL image in a direction of a first normal vector with respect to a plane of the AP image and the LL image; a step of determining a second volume of interest formed by intersecting a plane when moving a region of interest of the AP image and a region of interest of the LL image in a direction of a second normal vector of the AP image and the LL image of the 2D image within the perspective projection range, the geometric spatial relationship of the source and the detector to the 2D image being a perspective projection relationship; determining a first fiducial position within the first volume of interest corresponding to a first predetermined fiducial position within the first region of interest and a first displacement of a predetermined reference position corresponding to the first fiducial position; determining a second displacement between a second reference position in the second volume of interest and the reference position relative to a second predetermined reference position in the second volume of interest that corresponds to the reference position; determining a transformation relationship between the 2D image and the DRR image from a geometric spatial relationship of the DRR image and the 2D image to the source and detector, respectively, based on the first displacement and the second displacement, wherein the transformation relationship is determined such that a Euclidean distance between vertices of the first region of interest and vertices of the second region of interest is minimized according to the transformation relationship.
[16] The step of determining the first displacement includes a step of determining an angle between the reference position and the first reference position as a first rotation angle based on a first rotation axis that is parallel to a cross product vector of the first normal vector with respect to a plane of the AP image and the LL image and passes through a first predetermined origin; The image alignment method described in
[15] , wherein the step of determining the second displacement includes a step of determining an angle between the reference position and a second reference position as a second rotation angle based on a second rotation axis that is parallel to the cross product vector of the second normal vector with respect to the plane of the AP image and the LL image and passes through a predetermined second origin.
[17] The image matching method described in
[15] , wherein the steps of determining the first and second volumes of interest respectively include steps of projecting epipolar lines of vertices of the first and second regions of interest onto the first and second normal vectors to form a polyhedron.
[18] In an image alignment apparatus, An image matching device comprising a processor for executing the image matching method according to any one of [1] to
[17] .
[19] In a surgical robot system, a 2D imaging device for obtaining a 2D image of a surgical site on a patient during surgery; a robot arm including an end effector to which a surgical tool can be attached; a position sensor for sensing a real-time position of the surgical tool or end effector; a controller that controls the robot arm in accordance with a predetermined surgical plan; A display and 18. A surgical robot system comprising: a navigation system that performs the image matching method according to any one of claims 1 to 17, and displays the planning information for a surgical tool or implant on a 2D image obtained during surgery via the display, and displays the real-time position of the surgical tool or implant on the 2D image or a 3D image obtained before surgery.
[20] A computer program medium storing software for performing the image matching method according to any one of [1] to
[17] . [Explanation of symbols]
[0124] 10: Image alignment device 11: Memory 13: Processor 15: Display 17: User Interface 19: Communication interface
Claims
1. 1. A method for image alignment, wherein each step is performed by an image alignment device including a processor, obtaining a 3D image of a surgical site on a patient from a pre-operative 3D imaging device; Extracting DRR images in the AP direction and the LL direction from the 3D image; obtaining 2D images of AP and LL views of the surgical site on the patient from an intra-operative 2D imaging device; determining a first rotation angle between a reference position of the surgical site of the patient corresponding to a first reference position of the AP image or the LL image based on a first rotation axis that is parallel to a cross product vector of a first normal vector to the plane of the AP image and the LL image and passes through a first predetermined origin, based on the geometric spatial relationship of the source and the detector with respect to the DRR image; determining a second rotation angle between a second reference position of the AP image or the LL image corresponding to the reference position and the reference position based on a second rotation axis that is parallel to a cross product vector of a second normal vector to the plane of the AP image and the LL image and passes through a second predetermined origin, based on a geometric spatial relationship of a source and a detector with respect to the 2D image; determining a transformation relationship between the 2D image and the DRR image based on the first rotation angle and the second rotation angle from a geometric spatial relationship of the DRR image and the 2D image to the source and detector, respectively.
2. The image matching method according to claim 1 , wherein the first reference position and the second reference position are the center or a line or a plane including the center of the AP image or the LL image for the 2D image and the DRR image, respectively.
3. The image alignment device further includes a step of performing surgical planning based on the 3D image; The image matching method of claim 1 , wherein the first origin for the DRR image is determined based on a relative relationship between an implant mounting position applied to a surgical plan or a trajectory of a surgical tool for mounting the implant.
4. The method of claim 1 , wherein the reference position for the DRR image or the 2D image is determined by a user input.
5. a geometric spatial relationship of the source and the detector to the DRR image is an orthogonal projection relationship; 2. The method of claim 1, wherein the geometric spatial relationship of the source and the detector to the 2D image is a perspective projection relationship.
6. determining a first volume of interest from the geometric spatial relationship of the source and the detector to the DRR image that intersects a plane as the image alignment device moves through the AP image plane and the LL image plane in the direction of the first normal vector; The image alignment method of claim 1, further comprising: determining a second volume of interest in which the geometric spatial relationship of the source and the detector to the 2D image is a perspective projection relationship and planes intersect as the image alignment device moves the AP image plane and the LL image plane in the direction of the second normal vector within a perspective projection range.
7. 7. The method of claim 6, wherein the first origin is a center point of the first volume of interest and the second origin is a center point of the second volume of interest.
8. the image alignment device determining a first region of interest for each of the AP image and the LL image of the DRR image; and the image alignment device determines a second region of interest corresponding to the first region of interest for each of the AP image and the LL image of the 2D image; the first reference location is located within the first region of interest; 2. The method of claim 1, wherein said second reference location is located within said second region of interest.
9. determining a first volume of interest from the geometric spatial relationship of the source and the detector to the DRR image, the first volume of interest being intersected by a plane as the image alignment device moves through the region of interest in the AP image and the region of interest in the LL image in the direction of the first normal vector; The image matching method of claim 8, further comprising a step of determining a second volume of interest in which a plane intersects as the image matching device moves the region of interest of the AP image and the region of interest of the LL image in the direction of the second normal vector within the perspective projection range, wherein the geometric spatial relationship of the source and the detector to the 2D image is a perspective projection relationship.
10. 10. The method of claim 9, wherein the first origin is a center point of the first volume of interest and the second origin is a center point of the second volume of interest.
11. 10. The method of claim 9, wherein the first origin is a midpoint between target locations of the patient's spinal pedicle screws.
12. The first rotation angle is a line segment that connects a midpoint between pedicle screw entry points and the first origin, with the first origin as a center; 12. The method of claim 11, wherein the first normal vector passes through a center point of the first volume of interest.
13. each of the first regions of interest for the AP image and the LL image of the DRR image is rectangular; Regarding the DRR image, a first step in which the image alignment device calculates a first intersection point between an epipolar line on the LL image for each vertex of a region of interest in the AP image and a midline connecting an outer perimeter or a midpoint of a side of the region of interest in the LL image; a second step in which the image matching device orthogonally projects the first intersection point onto a normal vector extending from each vertex of the region of interest of the AP image in a normal direction of the AP image to obtain four restoration points; a third step in which the image alignment device determines a second intersection point between an epipolar line on the AP image for each vertex of the region of interest of the LL image and a midline connecting an outer perimeter or a midpoint of a side of the region of interest of the AP image; a fourth step in which the image alignment device orthogonally projects the second intersection point onto a normal vector extending from each vertex of the region of interest on the LL image in a normal direction of the LL image to obtain four restoration points; The image alignment method of claim 8, further comprising a fifth step in which the image alignment device calculates a first hexahedral volume of interest formed based on the eight restoration points obtained through the first to fourth steps.
14. The step of determining the second volume of interest comprises the steps of: a first step in which the image alignment device calculates a first intersection point between an epipolar line on the LL image for each vertex of a region of interest in the AP image and a midline connecting an outer perimeter or a midpoint of a side of the region of interest in the LL image; a second step in which the image matching device perspectively projects the first intersection point onto a perspective projection vector from each vertex of the region of interest of the AP image toward the source to obtain four restoration points; a third step in which the image alignment device determines a second intersection point between an epipolar line on the AP image for each vertex of the region of interest of the LL image and a midline connecting an outer perimeter or a midpoint of a side of the region of interest of the AP image; a fourth step in which the image matching device perspectively projects the second intersection point onto a perspective projection vector from each vertex of the region of interest on the LL image toward a source to obtain four restoration points; The image alignment method of claim 9, further comprising a fifth step in which the image alignment device determines a second hexahedral volume of interest based on the eight restoration points obtained in the first to fourth steps.
15. 1. A method for image alignment, wherein each step is performed by an image alignment device including a processor, obtaining a 3D image of a surgical site on a patient from a pre-operative 3D imaging device; Extracting DRR images in the AP direction and the LL direction from the 3D image; obtaining 2D images of AP and LL views of the surgical site on the patient from an intra-operative 2D imaging device; determining a first region of interest for each of the AP image and the LL image of the DRR image; determining a second region of interest corresponding to the first region of interest for each of the AP image and the LL image of the 2D image; determining a first volume of interest formed by intersecting planes when translating a region of interest in the AP image and a region of interest in the LL image in a direction of a first normal vector relative to the planes of the AP image and the LL image from a geometric spatial relationship of a source and a detector with respect to the DRR image; a second volume of interest formed by intersecting a plane when moving a region of interest of the AP image and a region of interest of the LL image in a direction of a second normal vector to the AP image and the LL image of the 2D image within a perspective projection range, the second volume of interest being a perspective projection relationship of the source and the detector with respect to the 2D image; determining a first fiducial position within the first volume of interest corresponding to a first predetermined fiducial position within the first region of interest and a first displacement of a predetermined reference position corresponding to the first fiducial position; determining a second displacement between a second reference position in the second volume of interest and the reference position relative to a second predetermined reference position in the second volume of interest corresponding to the reference position; determining a transformation relationship between the 2D image and the DRR image from a geometric spatial relationship of the DRR image and the 2D image to the source and detector, respectively, based on the first displacement and the second displacement, wherein the transformation relationship is determined such that a Euclidean distance between vertices of the first region of interest and vertices of the second region of interest is minimized according to the transformation relationship.
16. The step of determining the first displacement includes a step of determining an angle between the reference position and the first reference position as a first rotation angle based on a first rotation axis that is parallel to a cross product vector of the first normal vector with respect to a plane of the AP image and the LL image and passes through a first predetermined origin; The image alignment method of claim 15, wherein the step of determining the second displacement includes a step of determining an angle between the reference position and a second reference position as a second rotation angle based on a second rotation axis that is parallel to a cross product vector of the second normal vector to the plane of the AP image and the LL image and passes through a predetermined second origin.
17. 16. The image matching method of claim 15, wherein the steps of determining the first and second volumes of interest respectively include projecting epipolar lines of vertices of the first and second regions of interest onto the first and second normal vectors to form a polyhedron.
18. In an image alignment apparatus, 18. An image registration apparatus comprising a processor for carrying out the image registration method of any preceding claim.
19. In a surgical robot system, a 2D imaging device for obtaining a 2D image of a surgical site on a patient during surgery; a robot arm including an end effector to which a surgical tool can be attached; a position sensor for sensing a real-time position of the surgical tool or end effector; a controller that controls the robot arm in accordance with a predetermined surgical plan; A display and 18. A surgical robot system comprising a navigation system that performs the image matching method of any one of claims 1 to 17 to display planning information for a surgical tool or implant on a 2D image obtained during surgery via the display, and displays the real-time position of the surgical tool or implant on the 2D image or a 3D image obtained before surgery.
20. A computer program medium storing software for performing the image registration method of any one of claims 1 to 17.
Citation Information
Patent Citations
Method and system for improving 2d-3d registration convergence
JP2018079304A
Image-based navigation method and apparatus
JP2019514547A
C-arm based medical imaging system and method for aligning 2D images with 3D space
JP2022523587A
Pedicle screw fixation planning system and method
JP2022524607A
Guiding method of interventional procedure using medical images and system for interventional procedure for the same
KR1020170030687A
