Method for using vascular marker in surgical navigation system
Patent Information
- Application Number
- PCT/KR2023/004639
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-03-24
- Filing Date
- 2023-04-06
- Publication Date
- 2024-11-21
AI Technical Summary
Existing surgical navigation systems face challenges with fiducial markers due to low precision caused by large Target Registration Error (TRE) resulting from the distance between the marker and the surgical target, which can lead to physical burden and instability, whether using invasive or non-invasive markers.
The method involves using blood vessels near the surgical target as fiducial markers by injecting a fluorescent contrast agent, acquiring CT images, generating 3D models, and matching them using adaptive threshold technology and ICP algorithms to reduce TRE and improve precision.
This approach reduces patient burden and enhances surgical precision by minimizing TRE and stabilizing the guidance indicator, allowing for more accurate surgical navigation.
Smart Images

Figure KR2023004639_21112024_PF_FP_ABST
Abstract
Description
How to Use Vascular Markers in Surgical Navigation Systems
[0001] The present invention relates to a surgical navigation system, and more particularly, to a method of using a marker in a surgical navigation system.
[0002] The medical community is increasingly using surgical navigation to enable more precise and safer surgeries. This surgical navigation requires the use of fiducial markers. Fiducial markers are markers that can be used as reference points in medical imaging.
[0003] Conventional surgical navigation systems use fiducial markers placed on the patient. These fiducial markers can be categorized as invasive or non-invasive.
[0004] Non-invasive fiducial markers are attached to the skin, so they do not cause any burden to the patient, but they contain potential errors because they cannot be stably fixed due to the fluidity of the skin.
[0005] On the other hand, invasive fiducial markers are stable because they are fixed to the patient, but they can be burdensome to the patient because they must be attached to hard tissues such as bones in the body.
[0006] In common, both methods have the problem of low accuracy due to the large Target Registration Error (TRE) because the distance between the surgical target and the fiducial marker is far.
[0007] Figure 1 is an example diagram for explaining FLE, FRE, and TRE.
[0008] Referring to Figure 1, FLE is an abbreviation for Fiducial Localization Error, which means the error between each actual reference point (fiducial) and each measured point.
[0009] FRE stands for Fiducial Registration, and it refers to the error between the fiducial points in two spaces measured after registration. In other words, it refers to the error between each registered fiducial point.
[0010] TRE stands for Target Registration and refers to the error between the actual surgical target point and the surgical target point in the aligned model.
[0011] If the distance between the fiducial marker set and the surgical target is far, the TRE increases due to the distance between the target and the fiducial marker, even if the FRE has a small error. In other words, the smaller the distance between the fiducial marker set and the target, the smaller the TRE.
[0012] In this way, in the case of fiducial markers installed on patients for surgical navigation, there are problems such as physical burden or low stability as a guidance indicator depending on the marker used.
[0013] The present invention has been devised to solve the above problems, and its purpose is to provide a method of using a blood vessel close to a surgical target as a fiducial marker using a fluorescent contrast agent in a surgical navigation system.
[0014] The purpose of the present invention is not limited to the purposes mentioned above, and other purposes not mentioned will be clearly understood by those skilled in the art from the description below.
[0015] To achieve the above object, the present invention relates to a surgical navigation system, and a method for using a vascular marker, comprising: a CT process step of acquiring a CT image through CT (Computed Tomography) imaging of a patient; a CT image-based 3D reconstruction step of generating a CT 3D model using 3D reconstruction technology based on the CT image; a fluorescence process step of generating a fluorescence image by photographing a fluorescent contrast agent injected into a blood vessel near a surgical target of the patient moving to the surgical target through the blood vessel; a multi-camera based 3D reconstruction step of generating a fluorescence 3D model using 3D reconstruction technology based on the fluorescence image; a step of generating a vascular marker by photographing a blood vessel around the surgical target in the fluorescence image; and a registration step of aligning the CT 3D model and the fluorescence 3D model into one model based on the vascular marker.
[0016] In the step of generating the above blood vessel marker, the step may include a step of designating a region of interest (ROI) in the fluorescent image and a step of extracting blood vessels from the ROI using adaptive threshold technology and morphology.
[0017] In the above CT image-based 3D reconstruction step, the CT image can be restored into a 3D model using a 3D reconstruction technology based on the Marching Cube algorithm.
[0018] The above alignment step may include a rough alignment step of aligning the CT 3D model and the fluorescent 3D model into one model based on the blood vessel marker, and a fine alignment step of more precisely aligning the resulting model of the rough alignment step.
[0019] According to the present invention, by using a vascular marker in a surgical navigation system, the physical burden on the patient is reduced and surgical precision can be improved.
[0020] Figure 1 is an example diagram for explaining FLE, FRE, and TRE.
[0021] FIG. 2 is a flowchart showing a method of using a vascular marker in a surgical navigation system according to one embodiment of the present invention.
[0022] Figure 3 illustrates a process of 3D restoration from a CT scan image in one embodiment of the present invention.
[0023] FIG. 4 is a flowchart illustrating a CT 3D restoration process according to one embodiment of the present invention.
[0024] Figures 5 and 6 illustrate a fluorescence 3D restoration process according to one embodiment of the present invention.
[0025] Figure 7 is a flowchart illustrating a fluorescence 3D restoration process according to one embodiment of the present invention.
[0026] Figure 8 is a photograph using a fluorescent contrast agent.
[0027] Figure 9 illustrates a process for generating a vascular marker according to one embodiment of the present invention.
[0028] Figure 10 is a palm image captured with a near-infrared camera for vein recognition.
[0029] Figure 11 is a flowchart illustrating a marker generation process according to one embodiment of the present invention.
[0030] Figure 12 illustrates a rough alignment process according to one embodiment of the present invention.
[0031] Figure 13 illustrates a matching process using the ICP algorithm.
[0032] Figure 14 shows images before and after applying the ICP algorithm technique.
[0033] FIG. 2 is a flowchart showing a method of using a vascular marker in a surgical navigation system according to one embodiment of the present invention.
[0034] Referring to FIG. 2, there are provided a CT process step (S110) for acquiring a CT image through CT (Computed Tomography) shooting of a patient, a CT image-based 3D reconstruction step (S120) for generating a CT 3D model using 3D reconstruction technology based on the CT image, a fluorescence process step (S130) for generating a fluorescence image by shooting a fluorescent contrast agent injected into a blood vessel near a surgical target of the patient moving toward the surgical target through the blood vessel, a multi-camera based 3D reconstruction step (S140) for generating a fluorescence 3D model using 3D reconstruction technology from the fluorescence image, a step (S150) for generating a vessel marker by shooting a blood vessel around the surgical target, a coarse registration step (S160) for aligning a CT 3D model and a fluorescence 3D model into one model based on a vessel marker, and a fine registration step for more precisely aligning the resulting model after performing the coarse registration step. Includes step (S170).
[0035] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated and described in detail in the drawings. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention.
[0036] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.
[0037] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.
[0038] In addition, when describing with reference to the attached drawings, identical components will be assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted. When describing the present invention, if a detailed description of a related known technology is judged to unnecessarily obscure the gist of the present invention, the detailed description will be omitted.
[0039] The present invention relates to a method for using a vascular marker in a surgical navigation system.
[0040] In the present invention, the entity performing the method for using a vascular marker may be referred to as a surgical navigation system, or may be a control unit or processor that controls the surgical navigation system as a whole. That is, the method for using a vascular marker of the present invention may be composed of an algorithm, which is a type of software, and such an algorithm may be executed in the surgical navigation system, control unit, or processor that performs the method for using a vascular marker.
[0041] In the present invention, 3D reconstruction refers to a technology for restoring a two-dimensional image into a three-dimensional image. Registration is a technology for converting different data sets into a single coordinate system.
[0042] A fiducial marker refers to a marker that can be used as a reference in medical imaging. A surgical target refers to a specific area or tissue to be operated on.
[0043] A vessel marker is used as a reference marker, and a vessel near the surgical target is selected as a marker.
[0044] The present invention includes a CT process step of acquiring a CT image through CT (Computed Tomography) shooting of a patient, a CT image-based 3D reconstruction step of generating a CT 3D model using 3D reconstruction technology based on the CT image, a fluorescence process step of generating a fluorescence image by photographing a fluorescent contrast agent injected into a blood vessel near a surgical target of the patient moving toward the surgical target through the blood vessel, a multi-camera based 3D reconstruction step of generating a fluorescence 3D model using 3D reconstruction technology based on the fluorescence image, a step of generating a blood vessel marker by photographing a blood vessel around the surgical target in the fluorescence image, and a registration step of aligning the CT 3D model and the fluorescence 3D model into one model based on the blood vessel marker.
[0045] In the step of generating the above blood vessel marker, the step may include a step of designating a region of interest (ROI) in the fluorescent image and a step of extracting blood vessels from the ROI using adaptive threshold technology and morphology.
[0046] In the above CT image-based 3D reconstruction step, the CT image can be restored into a 3D model using a 3D reconstruction technology based on the Marching Cube algorithm.
[0047] The above alignment step may include a rough alignment step of aligning the CT 3D model and the fluorescent 3D model into one model based on the blood vessel marker, and a fine alignment step of more precisely aligning the resulting model of the rough alignment step.
[0048]
[0049] FIG. 2 is a flowchart showing a method of using a vascular marker in a surgical navigation system according to one embodiment of the present invention.
[0050] Referring to FIG. 2, there are provided a CT process step (S110) for acquiring a CT image through CT (Computed Tomography) shooting of a patient, a CT image-based 3D reconstruction step (S120) for generating a CT 3D model using 3D reconstruction technology based on the CT image, a fluorescence process step (S130) for generating a fluorescence image by shooting a fluorescent contrast agent injected into a blood vessel near a surgical target of the patient moving toward the surgical target through the blood vessel, a multi-camera based 3D reconstruction step (S140) for generating a fluorescence 3D model using 3D reconstruction technology from the fluorescence image, a step (S150) for generating a vessel marker by shooting a blood vessel around the surgical target, a coarse registration step (S160) for aligning a CT 3D model and a fluorescence 3D model into one model based on a vessel marker, and a fine registration step for more precisely aligning the resulting model after performing the coarse registration step. Includes step (S170).
[0051] FIG. 3 illustrates a process of 3D restoration from a CT scan image in one embodiment of the present invention, and FIG. 4 is a flowchart illustrating a CT 3D restoration process according to one embodiment of the present invention.
[0052] Referring to FIGS. 3 and 4, the CT process step (S110) includes a step (S111) of injecting a CT contrast agent into a surgical target area of a patient, a step (S113) of waiting for the CT contrast agent to take effect, and a step (S115) of taking a CT (Computed Tomography) image of the surgical target and its surrounding blood vessels. Here, a contrast agent refers to a drug that helps to clearly distinguish tissues or surgical targets by increasing the contrast of an image during an imaging diagnostic test.
[0053] In one embodiment of the present invention, in the CT image-based 3D reconstruction step (S120), a CT image can be restored into a 3D model using a 3D reconstruction technology based on a Marching Cube algorithm.
[0054] In Figure 3, the process of injecting CT contrast agent into the surgical target area of a patient to take a CT scan and the process of 3D restoring the CT scanned image are illustrated.
[0055] In the present invention, the fluorescence process step (S130) and the multi-camera based 3D reconstruction step (S140) highlight blood vessels near the surgical target with a fluorescence contrast agent, and create a fluorescence 3D model using a multi-camera based 3D reconstruction technology.
[0056] Figures 5 and 6 illustrate a fluorescence 3D restoration process according to one embodiment of the present invention.
[0057] Referring to FIGS. 5 and 6, in one embodiment of the present invention, in the fluorescence process step (S130) and the multi-camera-based 3D reconstruction step (S140), a fluorescent contrast agent is injected into a blood vessel near a surgical target of a patient, and then the image is captured using a multi-camera as it moves through the blood vessel to the surgical target, and this is 3D reconstructed.
[0058] Figure 7 is a flowchart illustrating a fluorescence 3D restoration process according to one embodiment of the present invention.
[0059] Referring to FIG. 7, the fluorescent process step (S130) includes a step of injecting a fluorescent contrast agent into a blood vessel near the surgical target of the patient (S131), a step of waiting until the fluorescent contrast agent is deposited (S133), and a step of photographing with a multiple near-infrared camera once the fluorescent contrast agent is deposited (S135).
[0060] Figure 8 is a photograph using a fluorescent contrast agent.
[0061] Referring to Figure 8, a fluorescent contrast agent (FCA) is a contrast agent capable of highlighting any tissue within the body in near-infrared images. Using this CFA, a target can be highlighted by simultaneously irradiating it with light of a specific wavelength and capturing it with a near-infrared camera.
[0062] Figure 9 illustrates a process for generating a vascular marker according to one embodiment of the present invention.
[0063] Referring to Figure 9, the process of generating a vascular marker in the present invention involves photographing blood vessels surrounding a surgical target to generate a vascular marker. Thus, the present invention utilizes blood vessels surrounding the surgical target as vascular markers to reduce TRE.
[0064] Figure 10 is a palm image captured with a near-infrared camera for vein recognition.
[0065] As shown in Figure 10, human blood vessels are complex and have many features that can be used as fiducial markers if they can be continuously observed, and vein recognition technology that uses human blood vessels as a security measure is already being verified to the extent that it is being utilized.
[0066] In the present invention, a vascular marker refers to administering a fluorescent contrast agent to a blood vessel surrounding a surgical target, and then using the blood vessel highlighted with the fluorescent contrast agent as a fiducial marker.
[0067] Figure 11 is a flowchart illustrating a marker generation process according to one embodiment of the present invention.
[0068] Referring to FIG. 11, the fluorescent process step (S130) includes a step of injecting a fluorescent contrast agent into a blood vessel near a surgical target of a patient (S131), a step of waiting until the fluorescent contrast agent is deposited (S133), and a step of photographing with a multiple near-infrared camera once the fluorescent contrast agent is deposited (S135).
[0069] The blood vessel marker generation step (S150) includes a step (S151) of specifying a region of interest (ROI) in a fluorescent image generated in the fluorescent process step (S130) and a step (S153, S155) of extracting blood vessels from the ROI using adaptive threshold technology and morphology.
[0070] Here, ROI refers to the area within the image to be used for calculations or extraction. Because using the entire image increases the computational load and affects processing speed, ROI is used.
[0071] In the present invention, the adaptive threshold technology is a technology used when extracting blood vessels from an image. When binarizing an image, rather than simply using a fixed value, it is a technology that calculates a threshold value adaptively suited to the image being used and binarizes it.
[0072] Morphology is a technique that deals with images from a morphological perspective, and modifies the results of the image to facilitate the detection of objects of interest.
[0073] In the present invention, the rough alignment step (S160) and the fine alignment step (S170) are processes for aligning a CT image-based 3D model and a fluorescence 3D model based on a vessel marker. At this time, the alignment result determines the quality of the final 3D model.
[0074] First, the CT-based 3D model and the fluorescence 3D model are aligned in the rough alignment step (S160).
[0075] And, in the detailed matching step (S170), the result model of the rough matching step (S150) is matched more finely using the ICP (Iterative Closest Point) technique.
[0076] Figure 12 illustrates a rough alignment process according to one embodiment of the present invention.
[0077] Referring to FIG. 12, the rough alignment step (S160) is a process of merging a CT-based 3D model and a fluorescent 3D model into one model, and aligns the CT-based 3D model and the fluorescent 3D model into one model based on a vessel marker.
[0078] Figure 13 illustrates a matching process using the ICP algorithm.
[0079] As shown in Figure 13, ICP is a technique for matching points that make up a model by repeatedly reducing the distance between them.
[0080] Figure 14 illustrates images before and after applying the ICP algorithm technique. In Figure 14, (a) is an image before applying the ICP algorithm technique, and (b) is an image after applying the ICP algorithm technique.
[0081] Referring to FIG. 14, the fine alignment step (S170) of the present invention is a process of more finely aligning the result model of the rough alignment step (S160) using the ICP (Iterative Closest Point) technique. In one embodiment of the present invention, the fine alignment step (S170) may be aligned using a point-to-plane method G-ICP or a linear optimization method ICP technique.
[0082] While the present invention has been described using several preferred embodiments, these embodiments are illustrative and not limiting. Those skilled in the art will appreciate that various changes and modifications can be made without departing from the spirit and scope of the invention as set forth in the appended claims.
Claims
1. In the method of using a vascular marker in a surgical navigation system, A CT process step that acquires CT images through CT (Computed Tomography) imaging of a patient; A CT image-based 3D reconstruction step of creating a CT 3D model using a 3D reconstruction technology based on the above CT image; A fluorescence process step of generating a fluorescence image by photographing the movement of a fluorescent contrast agent injected into a blood vessel near the surgical target of the patient through the blood vessel toward the surgical target; A multi-camera based 3D reconstruction step of generating a fluorescence 3D model using a 3D reconstruction technique from the above fluorescence image; A step of generating a blood vessel marker by photographing blood vessels around the surgical target in the above fluorescence image; and A matching step for matching the CT 3D model and the fluorescent 3D model into one model based on the above vascular markers. A method of using a vascular marker comprising:
2. In claim 1, In the step of generating the above vascular marker, A method for using a blood vessel marker, characterized in that it comprises a step of designating a region of interest (ROI) in the above fluorescence image and a step of extracting blood vessels from the ROI using adaptive threshold technology and morphology.
3. In claim 1, A method for using a blood vessel marker, characterized in that, in the above CT image-based 3D reconstruction step, the CT image is restored into a 3D model using a 3D reconstruction technology based on a Marching Cube algorithm.
4. In claim 1, The above matching step is, A method for using a vascular marker, characterized in that it comprises a rough alignment step of aligning the CT 3D model and the fluorescent 3D model into one model based on the vascular marker, and a fine alignment step of more precisely aligning the resulting model after performing the rough alignment step.
Citation Information
Patent Citations
Image matching method between computed tomography angiography image and X-Ray angiography image based on hemisphere shaped radiopaque 3D Marker
KR101485900B1
Registration method of images for surgery
KR1020130045774A
System and Method For Non-Invasive Patient-Image Registration
KR1020150019311A
Anatomical scanning, targeting, and visualization
US20230015717A1
