Method and system for determining the position of an object or a surgical instrument

Abstract

According to various embodiments of the present disclosure, a three-dimensional medical image of a subject is acquired from an external device, and a plurality of first X-ray images of the subject are acquired using an X-ray device. Based on the respective position coordinates of a plurality of X-ray sources and X-ray detectors in a first coordinate system, the plurality of first X-ray images, and the three-dimensional medical image, a second coordinate system of the three-dimensional medical image is matched to the first coordinate system. Based on the matching result, the position coordinates of the subject in the first coordinate system are determined. Using the X-ray device, a plurality of second X-ray images of a surgical tool including an electrode are acquired, and based on the plurality of second X-ray images, the position coordinates of the surgical tool in the first coordinate system are determined. It may be configured as such.

Description

Technical Field

[0001] The present disclosure relates to a method and a system for determining the position of an object or a surgical instrument.

[0002] This research is the result of a research conducted as part of the "Mid-sized Enterprise DNA Fusion Industry-Academia Collaboration Project" of the Ministry of Trade, Industry and Energy and the Korea Institute of Industrial Technology. [Project name: Development of Low-dose Multi-source C-arm CT Technology for AI-based Surgery, Project number: P0021346]

Background Art

[0003] To confirm the position of a diseased part of an object (e.g., a patient) in a three-dimensional space, a three-dimensional medical image such as a CT (Computed Tomography) image or an MRI (Magnetic Resonance Imaging) image can be taken in advance. Then, a doctor performs an operation on the object with reference to the three-dimensional medical image taken in advance. To accurately confirm the diseased part of the object, an X-ray image may be further utilized.

Summary of the Invention

Problems to be Solved by the Invention

[0004] When using both a three-dimensional medical image such as a CT image or an MRI image and a two-dimensional medical image such as an X-ray image, the accurate position of the diseased part in the object can be determined by aligning their respective coordinate systems. However, when a doctor performs an operation with reference to the three-dimensional medical image taken in advance, whether the surgical instrument is accurately positioned at the diseased part depends on the discretionary judgment of the doctor.

Means for Solving the Problems

[0005] A system for determining the position of an object or a surgical instrument according to various embodiments of the present disclosure may include an X-ray device including a plurality of X-ray sources configured to irradiate the object with X-rays and an X-ray detector configured to detect the X-rays transmitted through the object, a memory configured to store the position coordinates of each of the plurality of X-ray sources and the X-ray detector in a first coordinate system related to the system, and a processor. The processor according to various embodiments may acquire a three-dimensional medical image of the object from an external device, acquire a plurality of first X-ray images of the object using the X-ray device, match a second coordinate system of the three-dimensional medical image to the first coordinate system based on the position coordinates of each of the plurality of X-ray sources and the X-ray detector in the first coordinate system, the plurality of first X-ray images, and the three-dimensional medical image, determine the position coordinates of the object in the first coordinate system based on the matching result, acquire a plurality of second X-ray images of a surgical instrument including an electrode using the X-ray device, and determine the position coordinates of the surgical instrument in the first coordinate system based on the plurality of second X-ray images.

[0006] The processor according to various embodiments may project the three-dimensional medical image onto a two-dimensional plane to acquire a plurality of projection images, compare the plurality of projection images with the plurality of first X-ray images, determine a projection image having the highest similarity to the plurality of first X-ray images among the plurality of projection images, and match the second coordinate system of the three-dimensional medical image to the first coordinate system based on the plurality of first X-ray images and the determined projection image.

[0007] In various embodiments, the plurality of X-ray sources may be arranged on the same plane.

[0008] In various embodiments, the plurality of X-ray sources may be arranged in a straight line at equal intervals, and the angles at which each of the plurality of X-ray sources irradiates the object with X-rays may be different from each other.

[0009] In various embodiments, the plurality of X-ray sources may be X-ray sources using carbon nanotubes.

[0010] In various embodiments, the X-ray apparatus may further include a single power supply unit configured to supply a high voltage to the plurality of X-ray sources.

[0011] In various embodiments, the X-ray apparatus further includes a marker attached to a designated position, the system further includes a tracking sensor configured to track the position of the marker, and the memory can store a first coordinate transformation relationship between the plurality of X-ray sources and the X-ray detector, and a second coordinate transformation relationship between the plurality of X-ray sources and the marker. The processor according to various embodiments may obtain the position coordinates of the marker in the first coordinate system from the tracking sensor, and based on the position coordinates of the marker, obtain the position coordinates of each of the plurality of X-ray sources and the X-ray detector in the first coordinate system, and may be configured to store the position coordinates of each of the plurality of X-ray sources and the X-ray detector in the first coordinate system in the memory.

[0012] In various embodiments, the X-ray apparatus may further include a connecting member connected to the plurality of X-ray sources and the X-ray detector, a first rotating portion on which the plurality of X-ray sources are arranged and configured to rotate about a first rotation axis, and a second rotating portion connected to the connecting member and configured to rotate about a second rotation axis.

[0013] In various embodiments, the marker is attached to a designated position of the connecting member of the X-ray device, and the designated position may be within the viewing angle of the tracking sensor.

[0014] In various embodiments, the marker is attached to a designated position of the first rotating part of the X-ray device, and the designated position may be within the viewing angle of the tracking sensor.

[0015] A method for determining the position of an object or a surgical instrument in a system including an X-ray device including a plurality of X-ray sources configured to irradiate an object with X-rays and an X-ray detector configured to detect the X-rays transmitted through the object, a memory, and a processor includes the operations of receiving a three-dimensional medical image of the object from an external device, acquiring a plurality of first X-ray images of the object using the X-ray device, matching a second coordinate system of the three-dimensional medical image to the first coordinate system based on the respective position coordinates of the plurality of X-ray sources and the X-ray detector on the first coordinate system related to the system stored in the memory, the plurality of first X-ray images, and the three-dimensional medical image, determining the position coordinates of the object on the first coordinate system based on the matching result, acquiring a plurality of second X-ray images of a surgical instrument including an electrode using the X-ray device, and determining the position coordinates of the surgical instrument on the first coordinate system based on the plurality of second X-ray images.

[0016] The operation of performing the matching according to various embodiments may include: an operation of projecting the three-dimensional medical image onto a two-dimensional plane to obtain a plurality of projection images; an operation of comparing the plurality of projection images with the plurality of first X-ray images; an operation of determining, among the plurality of projection images, a projection image having the highest similarity to the plurality of first X-ray images; and an operation of matching the second coordinate system of the three-dimensional medical image to the first coordinate system based on the plurality of first X-ray images and the determined projection image.

[0017] In various embodiments, the plurality of X-ray sources may be arranged on the same plane.

[0018] In various embodiments, the plurality of X-ray sources may be arranged at equal intervals in a straight line, and each of the plurality of X-ray sources may have a different angle of irradiating the object with X-rays.

[0019] In various embodiments, the plurality of X-ray sources may be X-ray sources using carbon nanotubes.

[0020] In various embodiments, the X-ray device may further include a single power supply unit configured to supply a high voltage to the plurality of X-ray sources.

[0021] In various embodiments, the X-ray device further includes a marker attached to a designated position, the system further includes a tracking sensor configured to track the position of the marker, and the memory can store a first coordinate transformation relationship between the plurality of X-ray sources and the X-ray detector, and a second coordinate transformation relationship between the plurality of X-ray sources and the marker. The method according to various embodiments may further include obtaining the position coordinates of the marker on the first coordinate system from the tracking sensor, obtaining the respective position coordinates of the plurality of X-ray sources and the X-ray detector on the first coordinate system based on the position coordinates of the marker, and storing the respective position coordinates of the plurality of X-ray sources and the X-ray detector on the first coordinate system in the memory.

[0022] In various embodiments, the X-ray device may further include a connecting member connected to the plurality of X-ray sources and the X-ray detector, a first rotating portion on which the plurality of X-ray sources are arranged and configured to rotate about a first rotation axis, and a second rotating portion connected to the connecting member and configured to rotate about a second rotation axis.

[0023] In various embodiments, the marker is attached to a designated position of the connecting member of the X-ray device, and the designated position may be located within the viewing angle of the tracking sensor.

[0024] In various embodiments, the marker is attached to a designated position of the first rotating portion of the X-ray device, and the designated position may be located within the viewing angle of the tracking sensor.

Advantages of the Invention

[0025] The system according to various embodiments of the present disclosure can accurately confirm the position of the object in the three-dimensional space coordinate system by aligning the coordinate systems of the three-dimensional medical image of the object and the two-dimensional X-ray image of the object with each other.

[0026] The systems according to various embodiments of the present disclosure can confirm the accurate position of the surgical instrument in a three-dimensional space coordinate system by taking an X-ray image of the surgical instrument.

[0027] According to various embodiments of the present disclosure, since the plurality of X-ray sources are X-ray sources using digital carbon nanotubes, the imaging time can be shortened, the X-ray exposure time of the object can be shortened, and the power consumption can be reduced.

Brief Description of the Drawings

[0028]

Figure 1

Figure 2A

Figure 2B

Figure 3A

Figure 3B

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Mode for Carrying Out the Invention

[0029] The embodiments of the present disclosure are exemplified for the purpose of explaining the technical idea of the present disclosure. The scope of rights related to the present disclosure is not limited to the embodiments presented below or the specific descriptions related to these embodiments.

[0030] All technical terms and scientific terms used in the present disclosure have meanings generally understood by those having ordinary knowledge in the technical field to which the present disclosure belongs, unless otherwise specified. All terms used in the present disclosure are selected for the purpose of more clearly explaining the present disclosure and are not selected for the purpose of limiting the scope of rights related to the present disclosure.

[0031] Expressions such as "including", "comprising", "having", etc. used in the present disclosure should be understood as open-ended terms that may include other embodiments, unless otherwise specified in the clauses or sentences in which these expressions are included.

[0032] The singular expressions described in the present disclosure may include plural meanings, unless otherwise specified, and this also applies equally to the singular expressions described in the claims.

[0033] Expressions such as "first", "second", etc. used in the present disclosure are used to distinguish multiple components from each other and do not limit the order or importance of these components.

[0034] As used herein, the term "component" refers to a software or a hardware component such as a field-programmable gate array (FPGA) or an application specific integrated circuit (ASIC). However, "component" is not limited to hardware and software. A "component" may be configured to be on an addressable storage medium or configured to cause one or more processors to execute. Thus, by way of example, a "component" includes components such as software components, object-oriented software components, class components, and task components, and processors, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within a component and "component" may be combined into a smaller number of components and "components", or further separated into additional components and "components".

[0035] As used herein, the phrase "based on" is used to describe one or more factors that affect a decision, judgment, or action described in a clause or sentence in which the phrase appears, and this phrase does not exclude additional factors that affect the decision, judgment, or action.

[0036] As used herein, when one component is referred to as being "coupled" or "connected" to another component, it may be understood that the one component is directly coupled or connected to the other component, or it may be understood that the one component is coupled or connected through another new component.

[0037] In the flowcharts shown in this document, process steps, method steps, algorithms, etc. are described in a sequential order, but those processes, methods, and algorithms may be configured to operate in any suitable order. In other words, the steps of the processes, methods, and algorithms described in various embodiments of the present disclosure need not be performed in the order described in the present disclosure. Also, even if some steps are described as being performed non-simultaneously, in other embodiments, some of those steps may be performed simultaneously. Further, the illustration of the processes in the drawings does not mean that the illustrated processes exclude other changes and modifications thereto, nor does it mean that any of the illustrated processes or steps thereof are essential to one or more of the various embodiments of the present disclosure, nor does it mean that the illustrated processes are preferred.

[0038] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In the accompanying drawings, the same or corresponding components are denoted by the same reference numerals. Also, in the following description of the embodiments, duplicate descriptions of the same or corresponding components may be omitted. However, even if the description of a component is omitted, it is not intended that the component is not included in a certain embodiment.

[0039] FIG. 1 is a block diagram of a system 10 according to an embodiment of the present disclosure. FIG. 2A is a diagram showing an X-ray apparatus 110 according to an embodiment of the present disclosure, and FIG. 2B is a diagram schematically showing a plurality of X-ray sources 111 and X-ray detectors 113 of the X-ray apparatus 110. FIGS. 3A and 3B are diagrams showing the system 10 according to an embodiment of the present disclosure.

[0040] A system 10 for determining the position of a subject or a surgical instrument may include an X-ray apparatus 110, a processor 120, a tracking sensor 130, and / or a memory 140. In one embodiment, the system 10 may further include a display 150. Some parts of the configuration shown in FIG. 1 may be omitted or replaced in order to embody the various embodiments disclosed in this document.

[0041] The X-ray apparatus 110 may be a device for obtaining an image of the inside of a subject by fluoroscoping the subject S with X-rays and analyzing the subject. As shown in FIG. 2A, the X-ray apparatus 110 may include an X-ray source 111 (X-ray sources) and / or an X-ray detector 113 (X-ray detector). In one embodiment, the X-ray apparatus 110 may further include a marker 115 and / or a power supply unit 117. The marker 115 may be tracked by a tracking sensor 130 described later.

[0042] The X-ray source 111 of the X-ray apparatus 110 is a light source capable of emitting X-rays and can irradiate the subject with X-rays under the control of the processor 120. The subject may be located between the X-ray source 111 and the X-ray detector 113. In one embodiment, the X-ray apparatus 110 may include a plurality of X-ray sources 111. For example, the X-ray apparatus 110 may include three X-ray sources 111, may include two X-ray sources 111, or may include four or more X-ray sources 111. Hereinafter, for the sake of convenience of explanation, it will be described as including three X-ray sources 111, but the number of X-ray sources 111 is not limited to this description. The plurality of X-ray sources 111 may be, for example, X-ray sources 111 using carbon nanotubes (carbon nanotube, CNT). The plurality of X-ray sources 111 may be configured by, for example, a digital X-ray tube with a cold cathode structure using carbon nanotubes.

[0043] The X-ray detector 113 of the X-ray apparatus 110 may be a detection device that detects the amount (or intensity) of X-rays. The X-ray detector 113 can detect the amount of X-rays that have passed through the object among the X-rays irradiated from the X-ray source 111 to the object. When the internal density of the object is not uniform, the amount of X-rays absorbed by the object may vary depending on the direction in which the X-rays are irradiated. The X-ray detector 113 can measure the amount of X-rays that decreases while the X-rays irradiated at various angles pass through the object, and the processor 120 can generate a two-dimensional X-ray image that projects the inside of the object based on the value measured by the X-ray detector 113. For example, the processor 120 can convert the X-rays measured by the X-ray detector 113 into visible light rays, further convert them into digital signals, and image them. The X-ray detector 113 may be in the form of a plate, but may also be embodied in various forms capable of detecting X-rays.

[0044] The plurality of X-ray sources 111 may be arranged on one plane. For example, as shown in FIG. 2B, the plurality of X-ray sources 111 may all be arranged on one plane (plane P-P') parallel to the surface of the X-ray detector 113. The plurality of X-ray sources 111 may be arranged at equal intervals in a straight line, or may be arranged in a straight line at different intervals from each other. For example, the interval between the plurality of X-ray sources 111 may be determined within 50 to 200 nm. For example, the interval between the plurality of X-ray sources 111 may be 150 nm. The irradiation angles of the plurality of X-ray sources 111 can be set to be different from each other so that an object can be positioned within the field of view of the X-ray beams irradiated from all of the plurality of X-ray sources 111. For example, as shown in FIG. 2B, among the plurality of X-ray sources 111, the first X-ray source 111a located on the left side is set to irradiate X-rays at an angle inclined 15° to the right with respect to the normal of the plane (plane P-P'), the second X-ray source 111b located in the center is set to irradiate X-rays parallel to the normal of the plane (plane P-P'), and the third X-ray source 111c located on the right side may be set to irradiate X-rays at an angle inclined 15° to the left with respect to the normal of the plane (plane P-P'). In this case, the object S may be located in the region A where the field of view of each of the plurality of X-ray sources 111 overlaps. It is obvious that the above-described angles are exemplary, and the plurality of X-ray sources 111 may be changed to various angles at which they can irradiate the object with X-rays. When the plurality of X-ray sources 111 have the above-described arrangement, for example, even if the plurality of X-ray sources 111 are not arranged on a rotating plane, a plurality of X-ray images of the object imaged from different directions can be obtained.

[0045] Each of the plurality of X-ray sources 111 can be selectively and sequentially driven at regular time intervals (e.g., several ms). For example, the processor 120 first drives and then stops the first X-ray source 111a among the plurality of X-ray sources 111, and after a preset time has elapsed (e.g., after 5 ms), it drives and then stops the second X-ray source 111b, and further, after the set time has elapsed, it can drive and then stop the third X-ray source 111c. That is, the processor 120 controls the plurality of X-ray sources 111 to be selectively and sequentially driven within a short time instead of driving them simultaneously, thereby reducing the X-ray exposure amount of the object.

[0046] The X-ray apparatus 110 according to various embodiments may further include a connecting member 112, a first rotating part 114a, a second rotating part 114b, and / or a stage 116. The connecting member 112 may be configured to connect the plurality of X-ray sources 111 and the X-ray detector 113. For example, the connecting member 112 may be a C-shaped robot arm. The first rotating part 114a may be configured such that the plurality of X-ray sources 111 are arranged and rotate about the first rotation axis. For example, the first rotating part 114a may be configured to be capable of yaw rotation about the z-axis. By yaw-rotating the first rotating part 114a, the plurality of X-ray sources 111 can be yaw-rotated, and thereby, a longitudinal X-ray image or a lateral X-ray image of the object can be acquired.

[0047] The second rotating part 114b of the X-ray device 110 may be connected to the connecting member 112 and configured to rotate about the second rotation axis. For example, the second rotating part 114b may be configured to be capable of roll rotation about the x-axis. That is, by rotating the second rotating part 114b in a roll manner, the plurality of X-ray sources 111 and the X-ray detectors 113 can be rotated together. A user (e.g., a doctor) can adjust the positions of the first rotating part 114a and the second rotating part 114b in order to obtain an accurate X-ray image of the subject.

[0048] The stage 116 of the X-ray device 110 may be where the subject S is located. The stage 116 can move in the x-axis direction. After the subject S is located on the stage 116, the stage 116 may move between the plurality of X-ray sources 111 and the X-ray detectors 113. For example, as shown in FIG. 3B, the subject S is located on the stage 116, and the surgical site of the subject S may be located within the viewing angle T_FOV of the plurality of X-ray sources 111. The power supply unit 117 of the X-ray device 110 can supply the electric power necessary to operate each component of the X-ray device 110. The power supply unit 117 can supply the electric power necessary for the plurality of X-ray sources 111 to output X-rays.

[0049] The tracking sensor 130 may be a device for tracking the position and / or orientation of an object. For example, the tracking sensor 130 can track a target (e.g., the X-ray device 110) by measuring the position and / or orientation of a marker 115 attached to the target. The marker 115 can generate energy or a signal so that it can be sensed by the tracking sensor 130. The marker 115 may be attached to a predetermined position of the X-ray device 110. The marker 115 may be attached to various positions of each component of the X-ray device 110. According to one embodiment, the marker 115 may be attached to a designated position of the connecting member 112 of the X-ray device 110 as shown in FIG. 2A, or may be attached to a designated position of the first rotating part 114a of the X-ray device 110 as shown in FIG. 3A. Here, the designated position is located within the viewing angle of the tracking sensor 130. For example, as shown in FIG. 3A, the marker 115 is located within the viewing angle T_FOV of the tracking sensor 130.

[0050] The tracking method using the tracking sensor 130 is not particularly limited, but generally, an optical tracking method based on optical technology or an electromagnetic tracking method based on electromagnetic wave technology may be used. Also, various tracking methods may be used in combination.

[0051] The position measured by the tracking sensor 130 may be defined as three-dimensional space coordinates, such as coordinates on the x, y, and z axes of a rectangular coordinate system. Also, the orientation measured by the tracking sensor 130 may be defined as rotational information such as roll, pitch, and yaw. For accurate tracking of an object, the six degrees of freedom of the position and orientation of the object thus defined may be measured. According to one embodiment, the tracking sensor 130 can confirm the position of the x-ray device 110 by measuring the position of the marker 115 attached to the designated position of the x-ray device 110.

[0052] The processor 120 may be configured to perform operations or data processing related to the control and / or communication of each component included in the system 10. The processor 120 may be operatively coupled to, for example, the components of the x-ray device 110, the tracking sensor 130, the memory 140, and / or the display 150. The processor 120 can load instructions or data received from other components of the x-ray device 110 into the memory 140, process the instructions or data stored in the memory 140, and store the resulting data.

[0053] The memory 140 can store instructions for the operation of the processor 120. The memory 140 can store a first coordinate transformation relationship between a plurality of x-ray sources 111 and x-ray detectors 113, and a second coordinate transformation relationship between a plurality of x-ray sources 111 and markers 115. For example, the first coordinate transformation relationship and the second coordinate transformation relationship may be matrix vectors. The memory 140 can store the respective position coordinates of a plurality of x-ray sources 111 and x-ray detectors 113 on the first coordinate system related to the system 10.

[0054] The display 150 can display various screens based on the control of the processor 120. The display 150 can display, for example, a three-dimensional medical image or a two-dimensional x-ray image.

[0055] The processor 120 can obtain the position coordinates of the marker 115 on the first coordinate system related to the system 10 by using the tracking sensor 130. Here, the first coordinate system may be a three-dimensional space coordinate system used in the system 10. The processor 120 can obtain the position coordinates of the plurality of X-ray sources 111 and the X-ray detectors 113 on the first coordinate system based on the position coordinates of the marker 115. For example, the processor 120 can obtain the position coordinates of the plurality of X-ray sources 111 based on the second coordinate transformation relationship between the plurality of X-ray sources 111 and the marker 115 and the position coordinates of the marker 115. Then, the processor 120 can obtain the position coordinates of the X-ray detector 113 based on the first coordinate transformation relationship between the plurality of X-ray sources 111 and the X-ray detector 113 and the position coordinates of the plurality of X-ray sources 111. Here, the position coordinates of the plurality of X-ray sources 111 and the position coordinates of the X-ray detector 113 are the position coordinates defined on the first coordinate system of the system 10. The processor 120 can store the obtained position coordinates of each of the plurality of X-ray sources and the X-ray detector in the memory 140.

[0056] The processor 120 can obtain a three-dimensional medical image of the subject from an external device. The user can obtain a three-dimensional medical image (e.g., an MRI image or a CT image) of the subject in advance using an MRI device or a CT device before the surgery, and the processor 120 can receive the three-dimensional medical image of the subject from the MRI device or the CT device. For example, the processor 120 can receive the three-dimensional medical image of the subject from an MRI device or a CT device connected by wired or wireless communication. According to another embodiment, the processor 120 can also receive the three-dimensional medical image of the subject from a server device. In this case, the user can upload the three-dimensional medical image of the subject to the server device, and then transmit the three-dimensional medical image of the subject to the system 10.

[0057] The processor 120 can acquire a plurality of first X-ray images of the object using the X-ray device 110. The processor 120 can sequentially drive a plurality of X-ray sources 111 of the X-ray device 110 to acquire a plurality of first X-ray images of the object. For example, the processor 120 can acquire a plurality of first X-ray images respectively using the plurality of X-ray sources 111. In this case, the plurality of first X-ray images may be X-ray images obtained by photographing the object from different directions.

[0058] The processor 120 can match the second coordinate system of the three-dimensional medical image to the first coordinate system based on the three-dimensional medical image of the object and the plurality of first X-ray images. For example, the processor 120 can match the second coordinate system of the three-dimensional medical image to the first coordinate system based on the respective position coordinates of the plurality of X-ray sources 111 and the X-ray detector 113 on the first coordinate system, the plurality of first X-ray images of the object, and the three-dimensional medical image. The second coordinate of the three-dimensional medical image can mean a coordinate system used to indicate the position of each point of the object represented in three dimensions in the three-dimensional medical image. The three-dimensional medical image of the object can indicate the position of the object on the second coordinate system. A specific method for matching the second coordinate system of the three-dimensional medical image to the first coordinate system of the system 10 using the three-dimensional medical image and the X-ray image will be described later. The processor 120 can determine the position of the object on the first coordinate system based on the matching result. That is, the positional relationship between the plurality of X-ray sources 111 and the X-ray detector 113 may be defined on the first coordinate system by a predetermined calibration process, and based on the plurality of first X-ray images of the object obtained from such a plurality of X-ray sources 111 and the X-ray detector 113, the second coordinate system of the three-dimensional medical image of the object can be aligned with the first coordinate system. Thereby, even only with the X-ray image of the object, the position of the object on the first coordinate system, which is a three-dimensional space coordinate, can be accurately determined.

[0059] The processor 120 can also determine the position coordinates of the surgical instrument using the matching result. The processor 120 can use the X-ray device 110 to obtain a plurality of second X-ray images of the surgical instrument including the electrode during the surgery. The processor 120 can sequentially drive a plurality of X-ray sources 111 of the X-ray device 110 to obtain a plurality of second X-ray images of the surgical instrument. The processor 120 can determine the position of the surgical instrument in the first coordinate system based on the plurality of second X-ray images. In this case, since the X-ray image is aligned with the first coordinate system, the position of the electrode of the surgical instrument in the first coordinate system can be determined using the position of the electrode of the surgical instrument displayed on the X-ray image.

[0060] In FIG. 1, the X-ray device 110, the processor 120, the tracking sensor 130, the memory 140, and the display 150 are shown as being separately configured, but the present invention is not limited thereto. According to one embodiment, the processor 120, the memory 140, and the display 150 may be integrated with the X-ray device 110 and embodied as one device. According to one embodiment, the processor 120, the tracking sensor 130, the memory 140, and the display 150 may be embodied as one electronic device separately from the X-ray device 110. According to one embodiment, the processor 120, the memory 140, and the display 150 may be embodied as an electronic device separate from the X-ray device 110, and the tracking sensor 130 may also be embodied as a separate imaging device and may be communicatively connected to each other.

[0061] FIG. 4 is a flowchart of the operation of the processor 120 according to various embodiments of the present disclosure. Referring to the flowchart 400, in operation 410, the processor 120 according to various embodiments can receive a three-dimensional medical image of a subject from an external device. The three-dimensional medical image of the subject may be an MRI image or a CT image of the subject acquired in advance before the surgery. The external device may be, for example, an MRI device, a CT device, or a server device.

[0062] In operation 420, the processor 120 according to various embodiments can acquire a plurality of first X-ray images of the subject using the X-ray device 110. The processor 120 can acquire a plurality of first X-ray images of the subject by sequentially and selectively driving a plurality of X-ray sources 111. The plurality of first X-ray images may be X-ray images of the subject taken from different directions.

[0063] In operation 430, the processor 120 according to various embodiments can match the second coordinate system of the three-dimensional medical image to the first coordinate system. The processor 120 can match the second coordinate system of the three-dimensional medical image onto the first coordinate system based on the respective position coordinates of the plurality of X-ray sources 111 and the X-ray detector 113 on the first coordinate system, the plurality of first X-ray images of the subject, and the three-dimensional medical image. The method of matching the coordinate systems may be performed in various ways using the three-dimensional medical image and the X-ray image. For example, the processor 120 can virtually acquire a projection image in which the three-dimensional medical image acquired in advance before the surgery is projected onto a two-dimensional plane. This projection image may be a DRR (Digitally Reconstructed Radiograph) image. The processor 120 can perform coordinate system matching by comparing the generated virtual projection image with the actual X-ray image.

[0064] Specifically, the processor 120 can obtain a plurality of projection images in which the 3D medical images are projected from different directions, and can compare the plurality of projection images with a plurality of first X-ray images. The processor 120 can determine the projection image with the highest similarity among the plurality of projection images to the plurality of first X-ray images. That is, the processor 120 can find the projection image that is most similar to the X-ray image of the actual object. After that, when the processor 120 searches for the projection image with the highest similarity to the X-ray image, based on the projection angle when generating the projection image, etc., it can estimate the position and posture of the object on the X-ray image in the 3D space. In one embodiment, the processor 120 can also select points that are likely to be features on the X-ray image to calculate a rough position, and search for the projection image that is most similar to the actual X-ray image based on the virtual projection images around the position. In this case, the time taken to search for the matching projection image can be shortened. According to one embodiment, the above-described matching process between the X-ray image and the 3D medical image may be possible with only one X-ray image, but if the coordinate system matching process is performed using two or more X-ray images, a more accurate matching result can be obtained.

[0065] In operation 440, the processor 120 according to various embodiments can determine the position of the object on the first coordinate system based on the matching result. By aligning the second coordinate system of the 3D medical image using the above-described 3D medical image and the plurality of first X-ray images with the first coordinate system of the system 10, the position and / or posture of the object on the first coordinate system can be determined. That is, the user can accurately confirm the position of the object on the first coordinate system that appears in the 3D medical image.

[0066] In operation 450, the processor 120 according to various embodiments can obtain a plurality of second X-ray images of a surgical tool including an electrode using the X-ray device 110. For example, the processor 120 can obtain a plurality of second X-ray images of the surgical tool by sequentially and selectively driving a plurality of X-ray sources 111.

[0067] In operation 460, the processor 120 according to various embodiments can determine the position of the surgical tool in the first coordinate system based on the plurality of second X-ray images. By the matching operation performed in operation 450 of FIG. 4, it becomes possible to determine which position on the first coordinate system corresponds to a specific position on the X-ray image. Therefore, the position of the surgical tool in the first coordinate system corresponding to the position of the electrode of the surgical tool appearing in the plurality of second X-ray images can be determined. The user (for example, a doctor) can confirm the position of the surgical tool in the first coordinate system by taking an X-ray image during the operation and perform an accurate operation.

[0068] FIG. 5 is a flowchart of the operations of the processor 120 according to various embodiments of the present disclosure. Referring to the flowchart 500, in operation 510, the processor 120 according to various embodiments can obtain the position coordinates of the marker 115 in the first coordinate system from the tracking sensor 130. The tracking sensor 130 can obtain the position coordinates of the marker 115 in the first coordinate system by measuring the position of the marker 115 attached to a predetermined position of the X-ray device 110. Here, the first coordinate system may be a three-dimensional space coordinate system used in the system 10.

[0069] Processors 120 according to various embodiments can, in operation 520, obtain the position coordinates of each of the plurality of x-ray sources 111 and x-ray detectors 113 on the first coordinate system based on the obtained position coordinates of the marker 115. Here, the position coordinates of the plurality of x-ray sources 111 can mean the position coordinates of each of the plurality of x-ray sources 111. The memory 140 stores a first coordinate transformation relationship between the plurality of x-ray sources 111 and the x-ray detector 113, and a second coordinate transformation relationship between the plurality of x-ray sources 111 and the marker 115. The coordinate transformation relationship may be represented by a matrix vector. For example, the processor 120 can obtain the position coordinates of the plurality of x-ray sources 111 based on the position coordinates of the marker 115 and the second coordinate transformation relationship, and can obtain the position coordinates of the x-ray detector 113 based on the obtained position coordinates of the plurality of x-ray sources 111 and the first coordinate transformation relationship. Specific methods for obtaining the first coordinate transformation relationship and the second coordinate transformation relationship will be described later.

[0070] Processors 120 according to various embodiments can, in operation 530, store the position coordinates of the plurality of x-ray sources 111 and the x-ray detector 113 on the first coordinate system in the memory 140. The stored position coordinates of the plurality of x-ray sources 111 and the x-ray detector 113 may be used in the process of matching the second coordinate system of the three-dimensional medical image of the object to the first coordinate system of the system 10.

[0071] FIG. 6 is an x-ray image 600 of a surgical instrument 610 according to various embodiments of the present disclosure, and FIG. 7 is a diagram showing the position of the surgical instrument on a three-dimensional image 700 having a first coordinate system.

[0072] Referring to FIG. 6, the user can use the X-ray device 110 to capture a two-dimensional X-ray image 600 of the surgical tool 610 in order to confirm whether the surgical tool 610 has been accurately inserted into the affected part of the subject during the operation. The surgical tool 610 including the electrode 611 is displayed on the X-ray image 600. Then, the processor 120 can determine which position on the three-dimensional first coordinate system the position of the surgical tool 610 on the two-dimensional X-ray image 600 corresponds to. Specifically, by the matching operation performed in operation 430 of FIG. 4, the second coordinate system of the three-dimensional medical image is matched with the first coordinate system of the subject using the X-ray image of the subject and the three-dimensional medical image. Therefore, the processor 120 can determine the position on the first coordinate system corresponding to the position of the surgical tool 610 on the X-ray image 600. FIG. 7 shows the positions of the surgical tool and its electrode 711 on the three-dimensional image 700. For example, the position of the electrode 711 of the surgical tool in the first coordinate system corresponding to the position of the electrode 611 of the surgical tool 610 in the X-ray image 600 may be determined on the three-dimensional image 700. Therefore, only by using the X-ray image 600 of the surgical tool 610, the accurate position of the electrode 711 of the surgical tool on the three-dimensional image 700 having the three-dimensional first coordinate system can be grasped.

[0073] FIG. 8 is a flowchart of the operation of the processor 120 according to various embodiments of the present disclosure. Specifically, the flowchart 800 is a specific flowchart regarding operation 430 of FIG. 4.

[0074] In operation 810, the processor 120 according to various embodiments can obtain a plurality of projection images by projecting the three-dimensional medical image onto a two-dimensional plane. The plurality of projection images may be two-dimensional images obtained assuming that the three-dimensional medical image is projected onto the two-dimensional plane. This projection image may be a DRR image.

[0075] In operation 820, the processor 120 according to various embodiments can compare the plurality of projection images with a plurality of first X-ray images.

[0076] In operation 830, for various embodiments, the processor 120 can determine the projection image among the plurality of projection images that has the highest similarity to the plurality of first x-ray images. The processor 120 can determine the projection image among the plurality of projection images that has the highest similarity to the plurality of first x-ray images. When the processor 120 searches for the projection image that has the highest similarity to the x-ray image, based on, for example, the projection angle when the projection image is generated, the processor 120 can estimate the position and orientation of the object on the x-ray image in three-dimensional space.

[0077] In operation 840, for various embodiments, the processor 120 can match the second coordinate system of the three-dimensional medical image to the first coordinate system by comparing the plurality of first x-ray images with the determined projection image. That is, the processor 120 can determine the position of the object on the first coordinate system by matching the second coordinate system of the three-dimensional medical image to the first coordinate system of the system 10. Further, by aligning the x-ray image with the determined projection image, the position on the first coordinate system corresponding to the specific position displayed on the x-ray image can also be determined.

[0078] FIGS. 9 to 11 are diagrams for explaining the process of obtaining the first coordinate transformation relationship. Specifically, FIG. 9 is a diagram showing a method of obtaining the coordinate relationship between the x-ray source 111 and the x-ray detector 113, FIG. 10 is a plan view showing a calibration tool used to obtain the coordinate relationship between the x-ray source 111 and the x-ray detector 113, and FIG. 11 is a diagram showing the position coordinates of a plurality of x-ray sources 111 with respect to the coordinate system of the x-ray detector 113.

[0079] Referring to FIG. 9, a calibration tool can be used to obtain the first coordinate transformation relationship. The calibration tool may include a first correction plate 910 and a second correction plate 920. As shown in FIG. 9, the calibration tool may have a structure in which the first correction plate 910 and the second correction plate 920 are arranged in two layers at a predetermined interval, and may have a structure that can be arranged on the X-ray detector 113. For example, the first correction plate 910 and the second correction plate 920 may be arranged in parallel in two layers, and their areas may be the same. Each of the first correction plate 910 and the second correction plate 920 may include a plurality of balls 911, 921 arranged in a grid. The plurality of balls 911, 921 may be metal balls displayed in the X-ray image. The calibration tool can be called a calibration jig (zig), for example.

[0080] The plurality of first balls 911 arranged on the first correction plate 910 and the plurality of second balls 921 arranged on the second correction plate 920 are different in both the number and the arrangement position from each other. For example, as shown in FIG. 10(a), the plurality of first balls 911 are arranged on the first correction plate 910 in a 5*5 form, the interval between the plurality of first balls 911 is d1, and the distance from the outermost first ball 911 to the edge of the first correction plate 910 may be d2. The plurality of second balls 921 are arranged on the second correction plate 920 in a 4*4 form, the interval between the plurality of second balls 921 is d1, and the distance from the outermost second ball 921 to the edge of the second correction plate 920 may be d3. Also, the distance between the first correction plate 910 and the second correction plate 920 may be h. The above-mentioned d1, d2, d3, and h are values set by the user.

[0081] A plurality of X-ray images of the calibration tool can be obtained using a plurality of X-ray sources 111. For example, when three X-ray sources 111 are used, three X-ray images of the calibration tool can be obtained.

[0082] Thereafter, the processor 120 can obtain the coordinate relationship between the X-ray source 111 and the X-ray detector 113 using triangulation. Specifically, the processor 120 can obtain the position coordinates of a plurality of X-ray sources 111 centered on the X-ray detector 113. Since the method of calculating the coordinate relationship between the X-ray detector 113 and the plurality of X-ray sources 111 using triangulation may use general triangulation, specific description is omitted.

[0083] According to one embodiment, the processor 120 can obtain the respective position coordinates of the plurality of X-ray sources 111 based on a detector center coordinate system centered on the center of the X-ray detector 113. According to one embodiment, the processor 120 can obtain the respective position coordinates of the plurality of X-ray sources 111 based on a detector corner coordinate system centered on one vertex of the X-ray detector 113. As described above, the coordinate relationship between the X-ray detector 113 and each of the plurality of X-ray sources 111 can be obtained using a calibration tool. Therefore, if the processor 120 knows the respective position coordinates of the plurality of X-ray sources 111, the position coordinates of the X-ray detector 113 can be calculated using the above-described coordinate relationship, and vice versa. The above-described coordinate relationship between the X-ray detector 113 and each of the plurality of X-ray sources 111 may be expressed in the form of a matrix vector or text.

[0084] FIGS. 12 to 14 are diagrams showing a method of obtaining a second coordinate transformation relationship according to various embodiments of the present disclosure. Specifically, FIG. 12 is a diagram showing a pivoting tool 1200 used to obtain the second coordinate transformation relationship, FIG. 13 is a plurality of X-ray images obtained by photographing a correction phantom using a plurality of X-ray sources 111, and FIG. 14 is a diagram showing the position coordinates of the marker 115 based on the first X-ray source 111.

[0085] Referring to FIG. 12, the pivoting mechanism 1200 can be called a pivoting phantom or a calibration phantom. The pivoting mechanism 1200 may include a plurality of pivoting balls formed at specified positions. The pivoting balls may be balls formed of metal so as to be detected in an X-ray image, and may be called pivoting points. For example, four pivoting balls may be formed on the upper surface of the pivoting mechanism, four on the first side surface, and four on the second side surface, for a total of 12. However, the number and positions of the plurality of pivoting balls are not limited to this.

[0086] The user can position the pivoting mechanism 1200 on the X-ray detector 113. Thereafter, the user can use the plurality of X-ray sources 111 to acquire a plurality of X-ray images 1310, 1320, 1330 of the pivoting mechanism 1200. Since each of the plurality of X-ray sources 111 irradiates X-rays at different angles at different positions, the positions of the pivoting balls displayed in each of the plurality of X-ray images 1310, 1320, 1330 are different from each other.

[0087] The processor 120 can extract, for example, the position coordinates of the plurality of pivoting balls on the X-ray image 1310 acquired by the first X-ray source 111a. The processor 120 can extract, for example, the position coordinates of the plurality of pivoting balls centered on the first X-ray source 111 using triangulation. In a similar manner, the processor 120 can extract the position coordinates of the plurality of pivoting balls in the X-ray images 1320, 1330 acquired by the second X-ray source 111b and the third X-ray source 111c.

[0088] Thereafter, the user can position a marker probe on a plurality of pivot balls. For example, the marker probe may be positioned on any one of the plurality of pivot balls. Since this marker probe may be a marker that generates energy or a signal so as to be sensed by the tracking sensor 130, it can be sensed by the tracking sensor 130. The marker probe may be a portable and small-sized marker.

[0089] The processor 120 can use the tracking sensor 130 to obtain the position coordinates of the marker probe on the first coordinate system and the position coordinates of the marker 115 attached to the designated position of the X-ray device 110. Thereby, the processor 120 can obtain the positional relationship between the marker 115 attached to the designated position of the X-ray device 110 and the marker probe. Further, since the position coordinates of the marker probe are the same as the position coordinates of the pivot ball on which the marker probe is located, the processor 120 can obtain the positional relationship between the marker 115 attached to the designated position of the X-ray device 110 and the first X-ray source 111. That is, the processor 120 can obtain the coordinate conversion relationship between the first X-ray source 111 and the marker 115. By applying the above-described method identically to the second X-ray source 111 and the third X-ray source 111, the processor 120 can obtain the first coordinate conversion relationship among the plurality of X-ray sources 111 and the marker 115.

[0090] The processor 120 can store the obtained first coordinate conversion relationship and second coordinate conversion relationship in the memory 140. The calibration method described with reference to FIGS. 9 to 14 may be performed once immediately after the system 10 is newly set, and may be performed each time the arrangement position of the X-ray device 110 is changed. The first coordinate conversion relationship and the second coordinate conversion relationship stored in the memory 140 may be used to align the three-dimensional medical image of the object and the plurality of first X-ray images of the object.

[0091] The above method has been described by way of specific examples, but the above method can also be embodied as computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes any type of recording device on which data readable by a computer system is stored. Examples of the computer-readable recording medium may include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. Also, the computer-readable recording medium may be distributed over a computer system connected by a network, and the computer-readable code may be stored and executed in a distributed manner. And functional programs, codes, and code segments for implementing the above examples can be easily inferred by programmers in the technical field to which the present disclosure pertains.

Claims

1. A system for determining the position of an object or a surgical instrument, comprising: An X-ray device including a plurality of X-ray sources configured to irradiate the object with X-rays, and an X-ray detector configured to detect the X-rays transmitted through the object; A memory configured to store the respective position coordinates of the plurality of X-ray sources and the X-ray detector on a first coordinate system related to the system; A processor, wherein the processor: Obtains a three-dimensional medical image of the object from an external device; Obtains a plurality of first X-ray images of the object using the X-ray device; Based on the respective position coordinates of the plurality of X-ray sources and the X-ray detector on the first coordinate system, the plurality of first X-ray images, and the three-dimensional medical image, matches the second coordinate system of the three-dimensional medical image to the first coordinate system; Determines the position coordinates of the object on the first coordinate system based on the matching result; Obtains a plurality of second X-ray images of a surgical instrument including an electrode using the X-ray device; Determines the position coordinates of the surgical instrument on the first coordinate system based on the plurality of second X-ray images.

2. The processor: Projects the three-dimensional medical image onto a two-dimensional plane to obtain a plurality of projected images; Compares the plurality of projected images with the plurality of first X-ray images; Determines the projected image with the highest similarity to the plurality of first X-ray images among the plurality of projected images; Based on the plurality of first X-ray images and the determined projected image, matches the second coordinate system of the three-dimensional medical image to the first coordinate system.

3. The system according to claim 1, wherein the plurality of X-ray sources are arranged on the same plane.

4. The plurality of X-ray sources are arranged at equal intervals in a straight line, and each of the plurality of X-ray sources irradiates the object with X-rays at different angles.

5. The system according to claim 1, wherein the plurality of X-ray sources are X-ray sources using carbon nanotubes.

6. The X-ray device is The system according to claim 5, further comprising a single power supply unit configured to supply a high voltage to the plurality of X-ray sources.

7. The X-ray apparatus further includes a marker attached at a predetermined position, The system further includes a tracking sensor configured to track the position of the marker, The memory stores a first coordinate transformation relationship between the plurality of X-ray sources and the X-ray detector, and a second coordinate transformation relationship between the plurality of X-ray sources and the marker, The processor, acquires the position coordinates of the marker on the first coordinate system from the tracking sensor, acquires the respective position coordinates of the plurality of X-ray sources and the X-ray detector on the first coordinate system based on the position coordinates of the marker, The system according to claim 1, wherein the respective position coordinates of the plurality of X-ray sources and the X-ray detector on the first coordinate system are stored in the memory.

8. The X-ray apparatus, a connecting member connected to the plurality of X-ray sources and the X-ray detector, a first rotating part on which the plurality of X-ray sources are arranged and configured to rotate about a first rotation axis, The system according to claim 7, further comprising a second rotating part connected to the connecting member and configured to rotate about a second rotation axis.

9. The marker is attached to a specified position of the connecting member of the X-ray apparatus, The specified position is located within the viewing angle of the tracking sensor. The system according to claim 8.

10. The marker is attached to a specified position of the first rotating part of the X-ray apparatus, The specified position is located within the viewing angle of the tracking sensor. The system according to claim 8.

11. A method for determining the position of an object or a surgical instrument in a system including an X-ray apparatus including a plurality of X-ray sources configured to irradiate an object with X-rays and an X-ray detector configured to detect X-rays transmitted through the object, a memory, and a processor, comprising: receiving a three-dimensional medical image of the object from an external device; acquiring a plurality of first X-ray images of the object using the X-ray apparatus; Based on the position coordinates of each of the plurality of X-ray sources and the X-ray detector on the first coordinate system related to the system stored in the memory, the plurality of first X-ray images, and the three-dimensional medical image, an operation of matching the second coordinate system of the three-dimensional medical image to the first coordinate system, Based on the matching result, an operation of determining the position coordinates of the object on the first coordinate system, An operation of using the X-ray device to acquire a plurality of second X-ray images of a surgical tool including an electrode, Based on the plurality of second X-ray images, an operation of determining the position coordinates of the surgical tool on the first coordinate system, the method comprising:

12. The operation of matching includes: An operation of projecting the three-dimensional medical image onto a two-dimensional plane to obtain a plurality of projection images, An operation of comparing the plurality of projection images with the plurality of first X-ray images, Among the plurality of projection images, an operation of determining the projection image with the highest similarity to the plurality of first X-ray images, Based on the plurality of first X-ray images and the determined projection image, an operation of matching the second coordinate system of the three-dimensional medical image to the first coordinate system, the method according to claim 11.

13. The plurality of X-ray sources are arranged on the same plane, the method according to claim 11.

14. The plurality of X-ray sources are arranged at equal intervals in a straight line, Each of the plurality of X-ray sources has a different angle of irradiating the object with X-rays, the method according to claim 13.

15. The plurality of X-ray sources are X-ray sources using carbon nanotubes, the method according to claim 11.

16. The X-ray device Further includes one power supply unit configured to supply a high voltage to the plurality of X-ray sources, the method according to claim 15.

17. The X-ray device further includes a marker attached to a predetermined position, The system further includes a tracking sensor configured to track the position of the marker, The memory stores a first coordinate transformation relationship between the plurality of X-ray sources and the X-ray detector, and a second coordinate transformation relationship between the plurality of X-ray sources and the marker, The method An operation of acquiring the position coordinates of the marker on the first coordinate system from the tracking sensor, An operation of acquiring the position coordinates of each of the plurality of X-ray sources and the X-ray detector on the first coordinate system based on the position coordinates of the marker, An operation of storing the position coordinates of each of the plurality of X-ray sources and the X-ray detector on the first coordinate system in the memory, The method according to claim 11, further comprising:

18. The X-ray device includes: A connecting member connected to the plurality of X-ray sources and the X-ray detector, A first rotating part on which the plurality of X-ray sources are arranged and configured to rotate about a first rotation axis, A second rotating part connected to the connecting member and configured to rotate about a second rotation axis, The method according to claim 17, further comprising:

19. The marker is attached to a designated position of the connecting member of the X-ray device, The designated position is located within the viewing angle of the tracking sensor, The method according to claim 18.

20. The marker is attached to a designated position of the first rotating part of the X-ray device, The designated position is located within the viewing angle of the tracking sensor, The method according to claim 18.

Images