Method and system for determining the position of a target body or surgical instrument

Abstract

According to various embodiments of the present disclosure, a three-dimensional medical image of a subject is acquired from an external device, and a plurality of first X-ray images of the subject are acquired using an X-ray device. Based on the respective position coordinates of a plurality of X-ray sources and X-ray detectors in a first coordinate system, the plurality of first X-ray images, and the three-dimensional medical image, a second coordinate system of the three-dimensional medical image is matched to the first coordinate system. Based on the matching result, the position coordinates of the subject in the first coordinate system are determined. Using the X-ray device, a plurality of second X-ray images of a surgical tool including an electrode are acquired, and based on the plurality of second X-ray images, the position coordinates of the surgical tool in the first coordinate system are determined. It may be configured as such.

Description

Technical Field

[0001] The present disclosure relates to a method and system for determining the position of an object or a surgical instrument.

[0002] This research is the result of a research conducted as part of the "Mid - sized Enterprise DNA - integrated Industry - Academia Collaboration Project" of the Ministry of Trade, Industry and Energy and the Korea Institute of Industrial Technology. [Title of the project: Development of Low - dose Multi - source C - arm CT Technology for AI - based Surgery, Project number: P0021346]

Background Art

[0003] To confirm the position of a diseased part of an object (e.g., a patient) in a three - dimensional space, three - dimensional medical images such as CT (Computed Tomography) images or MRI (Magnetic Resonance Imaging) images can be taken in advance. Then, a doctor performs surgery on the object by referring to the pre - taken three - dimensional medical images. To accurately confirm the diseased part of the object, X - ray images may be further utilized.

Summary of the Invention

Problems to be Solved by the Invention

[0004] When using both three - dimensional medical images such as CT images or MRI images and two - dimensional medical images such as X - ray images, the accurate position of the diseased part within the object can be determined by aligning their respective coordinate systems. However, when a doctor performs surgery by referring to the pre - taken three - dimensional medical images, whether the surgical instrument is accurately positioned at the diseased part depends on the discretionary judgment of the doctor.

Means for Solving the Problems

[0005] A system for determining the position of an object or surgical instrument according to various embodiments of the present disclosure may include an X-ray apparatus including a plurality of X-ray sources configured to irradiate the object with X-rays, and an X-ray detector configured to detect X-rays that have passed through the object; a memory configured to store the position coordinates of the plurality of X-ray sources and the X-ray detector on a first coordinate system relating to the system; and a processor. The processor according to various embodiments may be configured to acquire a three-dimensional medical image of the object from an external device, acquire a plurality of first X-ray images of the object using the X-ray device, match the second coordinate system of the three-dimensional medical image to the first coordinate system based on the position coordinates of the plurality of X-ray sources and the X-ray detector on the first coordinate system, the plurality of first X-ray images, and the three-dimensional medical image, determine the position coordinates of the object on the first coordinate system based on the matching result, acquire a plurality of second X-ray images of a surgical instrument including electrodes using the X-ray device, and determine the position coordinates of the surgical instrument on the first coordinate system based on the plurality of second X-ray images.

[0006] The processor according to various embodiments may be configured to project the three-dimensional medical image onto a two-dimensional plane to acquire a plurality of projected images, compare the plurality of projected images with the plurality of first X-ray images, determine the projection image with the highest degree of similarity to the plurality of first X-ray images, and match the second coordinate system of the three-dimensional medical image to the first coordinate system based on the plurality of first X-ray images and the determined projection image.

[0007] In various embodiments, the multiple X-ray sources may be arranged on the same plane.

[0008] In various embodiments, the plurality of X-ray sources are arranged in a straight line at equal intervals, and the angles at which each of the plurality of X-ray sources irradiates the target object may differ from each other.

[0009] In various embodiments, the plurality of X-ray sources may be X-ray sources using carbon nanotubes.

[0010] In various embodiments, the X-ray apparatus may further include a power supply unit configured to supply high voltage to the plurality of X-ray sources.

[0011] In various embodiments, the X-ray apparatus further includes markers attached to predetermined positions, the system further includes tracking sensors configured to track the positions of the markers, and the memory can store a first coordinate transformation relationship between the plurality of X-ray sources and the X-ray detector, and a second coordinate transformation relationship between the plurality of X-ray sources and the marker. The processor in various embodiments may be configured to obtain the position coordinates of the markers in the first coordinate system from the tracking sensors, obtain the respective position coordinates of the plurality of X-ray sources and the X-ray detectors in the first coordinate system based on the position coordinates of the markers, and store the respective position coordinates of the plurality of X-ray sources and the X-ray detectors in the first coordinate system in the memory.

[0012] In various embodiments, the X-ray apparatus may further include a connecting member connected to the plurality of X-ray sources and the X-ray detector, a first rotating part on which the plurality of X-ray sources are arranged and configured to rotate about a first rotation axis, and a second rotating part connected to the connecting member and configured to rotate about a second rotation axis.

[0013] In various embodiments, the marker is attached to a designated position on the connecting member of the X-ray apparatus, and the designated position may be located within the field of view of the tracking sensor.

[0014] In various embodiments, the marker is attached to a designated position on the first rotating part of the X-ray apparatus, and the designated position may be located within the field of view of the tracking sensor.

[0015] A method for determining the position of a target object or surgical instrument in a system including an X-ray apparatus, a memory, and a processor, according to various embodiments of the present disclosure, which includes a plurality of X-ray sources configured to irradiate a target object with X-rays, and an X-ray detector configured to detect X-rays that have passed through the target object, may include: receiving a three-dimensional medical image of the target object from an external device; acquiring a plurality of first X-ray images of the target object using the X-ray apparatus; matching a second coordinate system of the three-dimensional medical image to the first coordinate system based on the position coordinates of the plurality of X-ray sources and the X-ray detector on a first coordinate system relating to the system stored in the memory, the plurality of first X-ray images, and the three-dimensional medical image; determining the position coordinates of the target object on the first coordinate system based on the matching result; acquiring a plurality of second X-ray images of a surgical instrument including electrodes using the X-ray apparatus; and determining the position coordinates of the surgical instrument on the first coordinate system based on the plurality of second X-ray images.

[0016] The matching operation according to various embodiments may include: an operation to project the three-dimensional medical image onto a two-dimensional plane to acquire a plurality of projected images; an operation to compare the plurality of projected images with the plurality of first X-ray images; an operation to determine the projection image among the plurality of projected images that has the highest similarity to the plurality of first X-ray images; and an operation to match the second coordinate system of the three-dimensional medical image to the first coordinate system based on the plurality of first X-ray images and the determined projection image.

[0017] In various embodiments, the multiple X-ray sources may be arranged on the same plane.

[0018] In various embodiments, the plurality of X-ray sources are arranged in a straight line at equal intervals, and the angles at which each of the plurality of X-ray sources irradiates the target object may differ from each other.

[0019] In various embodiments, the plurality of X-ray sources may be X-ray sources using carbon nanotubes.

[0020] In various embodiments, the X-ray apparatus may further include a power supply unit configured to supply high voltage to the plurality of X-ray sources.

[0021] In various embodiments, the X-ray apparatus further includes markers attached to predetermined positions, the system further includes tracking sensors configured to track the positions of the markers, and the memory can store a first coordinate transformation relationship between the plurality of X-ray sources and the X-ray detector, and a second coordinate transformation relationship between the plurality of X-ray sources and the marker. The method according to various embodiments may further include the operation of obtaining the position coordinates of the markers in the first coordinate system from the tracking sensor, the operation of obtaining the respective position coordinates of the plurality of X-ray sources and the X-ray detectors in the first coordinate system based on the position coordinates of the markers, and the operation of storing the respective position coordinates of the plurality of X-ray sources and the X-ray detectors in the first coordinate system in the memory.

[0022] In various embodiments, the X-ray apparatus may further include a connecting member connected to the plurality of X-ray sources and the X-ray detector, a first rotating part on which the plurality of X-ray sources are arranged and configured to rotate about a first rotation axis, and a second rotating part connected to the connecting member and configured to rotate about a second rotation axis.

[0023] In various embodiments, the marker is attached to a designated position on the connecting member of the X-ray apparatus, and the designated position may be located within the field of view of the tracking sensor.

[0024] In various embodiments, the marker is attached to a designated position on the first rotating part of the X-ray apparatus, and the designated position may be located within the field of view of the tracking sensor. [Effects of the Invention]

[0025] The systems according to various embodiments of this disclosure can accurately determine the position of an object in a three-dimensional spatial coordinate system by aligning the coordinate systems of the object's three-dimensional medical image and its two-dimensional X-ray image.

[0026] The systems according to various embodiments of the present disclosure can confirm the accurate position of a surgical instrument in a three-dimensional space coordinate system by taking an X-ray image of the surgical instrument.

[0027] According to various embodiments of the present disclosure, since the plurality of X-ray sources are X-ray sources using digital carbon nanotubes, the imaging time can be shortened, the X-ray exposure time of the object can be shortened, and the power consumption can be reduced.

Brief Description of Drawings

[0028] [Figure 1] It is a block diagram of a system according to an embodiment of the present disclosure. [Figure 2A] It is a diagram showing an X-ray apparatus according to an embodiment of the present disclosure. [Figure 2B] It is a diagram schematically showing a plurality of X-ray sources and an X-ray detector of an X-ray apparatus. [Figure 3A] It is a diagram showing a system according to an embodiment of the present disclosure. [Figure 3B] It is a diagram showing a system according to an embodiment of the present disclosure. [Figure 4] It is a flowchart of the operation of a processor according to an embodiment of the present disclosure. [Figure 5] It is a flowchart of the operation of a processor according to an embodiment of the present disclosure. [Figure 6] It is an X-ray image of a surgical instrument according to an embodiment of the present disclosure. [Figure 7] It is a diagram showing the position of a surgical instrument on a first coordinate system. [Figure 8] It is a flowchart of the operation of a processor according to an embodiment of the present disclosure. [Figure 9] It is a diagram for explaining the process of obtaining a first coordinate transformation relationship. [Figure 10] It is a diagram for explaining the process of obtaining a first coordinate transformation relationship. [Figure 11]This diagram illustrates the process of obtaining the first coordinate transformation relationship. [Figure 12] This figure shows a method for obtaining a second coordinate transformation relationship according to one embodiment of the present disclosure. [Figure 13] A diagram showing a method for obtaining a second coordinate transformation relationship according to one embodiment of the present disclosure. [Figure 14] A diagram showing a method for obtaining a second coordinate transformation relationship according to one embodiment of the present disclosure. [Modes for carrying out the invention]

[0029] The embodiments described herein are illustrative for the purpose of illustrating the technical concept of this disclosure. The scope of rights relating to this disclosure is not limited to the embodiments presented below or the specific descriptions relating thereto.

[0030] All technical and scientific terms used in this disclosure have meanings that would be generally understood by a person with ordinary skill in the art to which this disclosure pertains, unless otherwise specified. All terms used in this disclosure have been chosen for the purpose of making this disclosure clearer, and not to limit the scope of rights relating to this disclosure.

[0031] Expressions such as "includes," "equipped with," and "possess" used in this disclosure should be understood as open-ended terms that may include other embodiments, unless otherwise specified in the phrase or sentence containing such expression.

[0032] Unless otherwise specified, singular expressions described in this disclosure may include plural meanings, and this applies equally to singular expressions described in the claims.

[0033] The terms "First," "Second," etc., used in this disclosure are used to distinguish between multiple components and do not limit the order or importance of those components.

[0034] As used in this disclosure, the term “part” means software or hardware components such as FPGAs (field-programmable gate arrays) and ASICs (application-specific integrated circuits). However, “part” is not limited to hardware and software. A “part” may be configured to reside on an addressable storage medium and may be configured to regenerate one or more processors. Thus, as an example, a “part” includes components such as software components, object-oriented software components, class components and task components, and processors, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables. Components and the functions provided within a “part” may be combined as a smaller number of components and “parts,” or further separated as additional components and “parts.”

[0035] The expression "based on" as used in this disclosure is used to describe one or more factors that influence an act or action of decision, judgment, or action described in the phrase or sentence containing the expression, and the expression does not exclude any additional factors that influence an act or action of decision, judgment, or action.

[0036] Where it is referred to in this disclosure that one component is “connected” or “linked” to another component, it may be understood that the one component is directly connected or linked to the other component, or that it is connected or linked through a new other component.

[0037] While the flowcharts in this document describe process stages, method stages, and algorithms in a sequential order, these processes, methods, and algorithms may be configured to operate in any appropriate order. In other words, the stages of the processes, methods, and algorithms described in the various embodiments of this disclosure do not need to be performed in the order described herein. Furthermore, even if some stages are described as being performed non-simultaneously, in other embodiments, some of those stages may be performed simultaneously. Moreover, the examples of processes shown in the drawings do not mean that the illustrated processes are excluded from other variations and modifications, nor do they mean that any illustrated process or any of its stages is essential to one or more of the various embodiments of this disclosure, nor do they mean that the illustrated processes are preferred.

[0038] Embodiments of this disclosure will be described below with reference to the attached drawings. In the attached drawings, identical or corresponding components are denoted by the same reference numerals. In the following description of embodiments, redundant descriptions of identical or corresponding components may be omitted. However, the omission of a description of a component does not mean that the component is not included in a particular embodiment.

[0039] Figure 1 is a block diagram of system 10 according to one embodiment of the present disclosure. Figure 2A is a diagram showing an X-ray apparatus 110 according to one embodiment of the present disclosure, and Figure 2B is a diagram illustrating the multiple X-ray sources 111 and X-ray detectors 113 of the X-ray apparatus 110. Figures 3A and 3B show system 10 according to one embodiment of the present disclosure.

[0040] The system 10 for determining the position of a target body or surgical instrument may include an X-ray device 110, a processor 120, a tracking sensor 130, and / or memory 140. In one embodiment, the system 10 may further include a display 150. Some of the configurations shown in Figure 1 may be omitted or replaced in order to embody the various embodiments disclosed herein.

[0041] The X-ray apparatus 110 may be a device for analyzing an object S by using X-rays to visualize the inside of the object. As shown in Figure 2A, the X-ray apparatus 110 may include X-ray sources 111 and / or an X-ray detector 113. In one embodiment, the X-ray apparatus 110 may further include a marker 115 and / or a power supply unit 117. The marker 115 may be tracked by a tracking sensor 130, which will be described later.

[0042] The X-ray source 111 of the X-ray apparatus 110 is a light source capable of emitting X-rays and can irradiate a target object with X-rays under the control of the processor 120. The target object may be located between the X-ray source 111 and the X-ray detector 113. In one embodiment, the X-ray apparatus 110 may include a plurality of X-ray sources 111. For example, the X-ray apparatus 110 may include three X-ray sources 111, two X-ray sources 111, or four or more X-ray sources 111. For the sake of explanation, the following description will assume that the apparatus includes three X-ray sources 111, but the number of X-ray sources 111 is not limited to this description. The plurality of X-ray sources 111 may be, for example, X-ray sources 111 using carbon nanotubes (CNTs). The multiple X-ray sources 111 may be composed of, for example, digital X-ray tubes with a cold cathode structure using carbon nanotubes.

[0043] The X-ray detector 113 of the X-ray apparatus 110 may be a detection device that detects the amount (or intensity) of X-rays. The X-ray detector 113 can detect the amount of X-rays that pass through the target object from the X-ray source 111. If the internal density of the target object is not uniform, the amount of X-rays absorbed by the target object may differ depending on the direction from which the X-rays are irradiated. The X-ray detector 113 can measure the amount of X-rays that decreases as X-rays irradiated at various angles pass through the target object, and the processor 120 can generate a two-dimensional X-ray image projecting the interior of the target object based on the value measured by the X-ray detector 113. For example, the processor 120 can convert the X-rays measured by the X-ray detector 113 into visible light, and further convert this into a digital signal to create an image. The X-ray detector 113 may be in the form of a plate, but it may be implemented in various forms that can detect X-rays.

[0044] Multiple X-ray sources 111 may be arranged on a single plane. For example, as shown in Figure 2B, multiple X-ray sources 111 may all be arranged on a single plane (plane P-P') parallel to the surface of the X-ray detector 113. Multiple X-ray sources 111 may be arranged in a straight line at equal intervals, or they may be arranged in a straight line at different intervals from each other. For example, the interval between multiple X-ray sources 111 may be determined to be within 50 to 200 nm. For example, the interval between multiple X-ray sources 111 may be 150 nm. The irradiation angles of each of the multiple X-ray sources 111 can be set to be different from each other so that the target object can be located within the field of view of the X-ray beams irradiated from all of the multiple X-ray sources 111. For example, as shown in Figure 2B, among the multiple X-ray sources 111, the first X-ray source 111a located on the left is set to emit X-rays at an angle tilted 15° to the right with respect to the normal to the plane (plane P-P'), the second X-ray source 111b located in the center is set to emit X-rays parallel to the normal to the plane (plane P-P'), and the third X-ray source 111c located on the right is set to emit X-rays at an angle tilted 15° to the left with respect to the normal to the plane (plane P-P'). In this case, the target object S may be located in region A where the field of view angles of the multiple X-ray sources 111 overlap. The angles described above are illustrative, and it is clear that the multiple X-ray sources 111 may be changed to various angles that allow X-rays to be emitted onto the target object. When multiple X-ray sources 111 are arranged as described above, for example, even if the multiple X-ray sources 111 are not arranged in a rotating plane, multiple X-ray images of an object imaged from different directions can be acquired.

[0045] Each of the multiple X-ray sources 111 can be selectively and sequentially driven at regular time intervals (e.g., a few ms). For example, the processor 120 can first drive and then stop the first X-ray source 111a, then drive and then stop the second X-ray source 111b after a predetermined time has elapsed (e.g., 5 ms), and then drive and then stop the third X-ray source 111c after another predetermined time has elapsed. In other words, the processor 120 can reduce the amount of X-ray exposure to the target object by controlling the multiple X-ray sources 111 to be driven selectively and sequentially within a short time interval, rather than driving them simultaneously.

[0046] The X-ray apparatus 110 according to various embodiments may further include a connecting member 112, a first rotating part 114a, a second rotating part 114b, and / or a stage 116. The connecting member 112 may be configured to connect a plurality of X-ray sources 111 and an X-ray detector 113. For example, the connecting member 112 may be a C-shaped robot arm. The first rotating part 114a may be configured to hold a plurality of X-ray sources 111 and rotate around a first rotation axis. For example, the first rotating part 114a may be configured to be capable of yaw rotation around the z axis. By yaw rotating the first rotating part 114a, the plurality of X-ray sources 111 can be yaw rotated, thereby enabling the acquisition of a longitudinal or transverse X-ray image of the target object.

[0047] The second rotating part 114b of the X-ray apparatus 110 may be connected to a connecting member 112 and configured to rotate about a second rotation axis. For example, the second rotating part 114b may be configured to rotate in a roll motion about the x-axis. That is, by rotating the second rotating part 114b in a roll motion, multiple X-ray sources 111 and X-ray detectors 113 can be rotated together. The user (e.g., a physician) can adjust the positions of the first rotating part 114a and the second rotating part 114b to obtain an accurate X-ray image of the target body.

[0048] The stage 116 of the X-ray apparatus 110 may be on which the subject S is positioned. The stage 116 can move in the x-axis direction. After the subject S is positioned on the stage 116, the stage 116 may move between the multiple X-ray sources 111 and the X-ray detector 113. For example, as shown in Figure 3B, the subject S may be positioned on the stage 116, and the surgical site of the subject S may be located within the field of view T_FOV of the multiple X-ray sources 111. The power supply unit 117 of the X-ray apparatus 110 can supply the power necessary to operate each component of the X-ray apparatus 110. The power supply unit 117 can supply the power necessary for the multiple X-ray sources 111 to output X-rays.

[0049] The tracking sensor 130 may be a device for tracking the position and / or orientation of an object. For example, the tracking sensor 130 can track a target (e.g., an X-ray apparatus 110) by measuring the position and / or orientation of a marker 115 attached to the target. The marker 115 can generate energy or a signal that can be sensed by the tracking sensor 130. The marker 115 may be attached to a predetermined position on the X-ray apparatus 110. The marker 115 may be attached to various positions on each component of the X-ray apparatus 110. According to one embodiment, the marker 115 may be attached to a predetermined position on the connecting member 112 of the X-ray apparatus 110, as shown in Figure 2A, or to a predetermined position on the first rotating part 114a of the X-ray apparatus 110, as shown in Figure 3A. Here, the predetermined position is located within the field of view of the tracking sensor 130. For example, as shown in Figure 3A, the marker 115 is located within the field of view T_FOV of the tracking sensor 130.

[0050] The tracking method using the tracking sensor 130 is not particularly limited, but generally, an optical tracking method based on optical technology or an electromagnetic tracking method based on electromagnetic wave technology may be used. Furthermore, various tracking methods may be used in combination.

[0051] The position measured by the tracking sensor 130 may be defined as three-dimensional spatial coordinates, such as coordinates on the x, y, and z axes of a Cartesian coordinate system. The orientation measured by the tracking sensor 130 may be defined as rotational information such as roll, pitch, and yaw. For accurate tracking of an object, the six degrees of freedom of the position and orientation of the object, as defined in this way, may be measured. In one embodiment, the tracking sensor 130 can confirm the position of the X-ray apparatus 110 by measuring the position of a marker 115 attached to a specified position on the X-ray apparatus 110.

[0052] The processor 120 may be configured to perform calculations or data processing related to the control and / or communication of each component included in the system 10. The processor 120 may be operationally coupled to, for example, components of the X-ray apparatus 110, a tracking sensor 130, a memory 140, and / or a display 150. The processor 120 can load instructions or data received from other components of the X-ray apparatus 110 into the memory 140, process the instructions or data stored in the memory 140, and store the resulting data.

[0053] Memory 140 can store instructions for the operation of the processor 120. Memory 140 can store a first coordinate transformation relationship between multiple X-ray sources 111 and X-ray detectors 113, and a second coordinate transformation relationship between multiple X-ray sources 111 and markers 115. For example, the first and second coordinate transformation relationships may be matrix vectors. Memory 140 can store the respective position coordinates of multiple X-ray sources 111 and X-ray detectors 113 on a first coordinate system with respect to the system 10.

[0054] The display 150 can display various screens based on the control of the processor 120. For example, the display 150 can display a three-dimensional medical image or a two-dimensional X-ray image.

[0055] The processor 120 can use the tracking sensor 130 to obtain the position coordinates of the marker 115 on a first coordinate system relating to the system 10. Here, the first coordinate system may be a three-dimensional spatial coordinate system used in the system 10. Based on the position coordinates of the marker 115, the processor 120 can obtain the position coordinates of multiple X-ray sources 111 and X-ray detectors 113 on the first coordinate system. For example, the processor 120 can obtain the position coordinates of multiple X-ray sources 111 based on a second coordinate transformation relationship between the multiple X-ray sources 111 and the marker 115, and the position coordinates of the marker 115. Subsequently, the processor 120 can obtain the position coordinates of the X-ray detectors 113 based on a first coordinate transformation relationship between the multiple X-ray sources 111 and the X-ray detectors 113, and the position coordinates of the multiple X-ray sources 111. Here, the position coordinates of the multiple X-ray sources 111 and the position coordinates of the X-ray detectors 113 are position coordinates defined on the first coordinate system of the system 10. The processor 120 can store the acquired position coordinates of multiple X-ray sources and X-ray detectors in the memory 140.

[0056] The processor 120 can acquire a 3D medical image of the subject from an external device. The user may acquire a 3D medical image of the subject (e.g., an MRI image or a CT image) in advance using an MRI or CT scanner before surgery, and the processor 120 can receive the 3D medical image of the subject from the MRI or CT scanner. For example, the processor 120 can receive a 3D medical image of the subject from an MRI or CT scanner connected via wired or wireless communication. According to another embodiment, the processor 120 can also receive a 3D medical image of the subject from a server device. In this case, the user may upload the 3D medical image of the subject to the server device and then transmit the 3D medical image of the subject to the system 10.

[0057] The processor 120 can acquire multiple first X-ray images of an object using the X-ray apparatus 110. The processor 120 can acquire multiple first X-ray images of an object by sequentially driving multiple X-ray sources 111 of the X-ray apparatus 110. For example, the processor 120 can acquire multiple first X-ray images using multiple X-ray sources 111, and in this case, the multiple first X-ray images may be X-ray images of the object taken from different directions.

[0058] The processor 120 can match the second coordinate system of a 3D medical image to the first coordinate system based on the 3D medical image of the object and multiple first X-ray images. For example, the processor 120 can match the second coordinate system of a 3D medical image to the first coordinate system based on the position coordinates of multiple X-ray sources 111 and X-ray detectors 113 on the first coordinate system, multiple first X-ray images of the object, and the 3D medical image. The second coordinate system of a 3D medical image can mean the coordinate system used to indicate the position of each point of the object represented in 3D in the 3D medical image. The 3D medical image of the object can indicate the position of the object on the second coordinate system. A specific method for matching the second coordinate system of a 3D medical image to the first coordinate system of the system 10 using the 3D medical image and X-ray images will be described later. Based on the matching result, the processor 120 can determine the position of the object on the first coordinate system. In other words, the positional relationship between multiple X-ray sources 111 and X-ray detectors 113 may be defined on a first coordinate system by a predetermined calibration process, and based on multiple first X-ray images of the object obtained from such multiple X-ray sources 111 and X-ray detectors 113, the second coordinate system of the object's three-dimensional medical image can be aligned with the first coordinate system. As a result, the position of the object on the first coordinate system, which is a three-dimensional spatial coordinate system, can be accurately determined using only the object's X-ray image.

[0059] The processor 120 can also determine the position coordinates of the surgical instrument using the matching results. The processor 120 can acquire multiple second X-ray images of the surgical instrument, including electrodes, during surgery using the X-ray device 110. The processor 120 can sequentially drive multiple X-ray sources 111 of the X-ray device 110 to acquire multiple second X-ray images of the surgical instrument. Based on the multiple second X-ray images, the processor 120 can determine the position of the surgical instrument in the first coordinate system. In this case, since the X-ray images are aligned to the first coordinate system, the position of the electrodes of the surgical instrument in the first coordinate system can be determined using the position of the electrodes of the surgical instrument displayed on the X-ray images.

[0060] In Figure 1, the X-ray device 110, processor 120, tracking sensor 130, memory 140, and display 150 are shown as being configured separately, but the invention is not limited to this configuration. In one embodiment, the processor 120, memory 140, and display 150 may be integrated with the X-ray device 110 and implemented as a single device. In another embodiment, the processor 120, tracking sensor 130, memory 140, and display 150 may be implemented as separate electronic devices from the X-ray device 110. In yet another embodiment, the processor 120, memory 140, and display 150 may be implemented as separate electronic devices from the X-ray device 110, and the tracking sensor 130 may also be implemented as a separate imaging device, and they may be communicated and connected to each other.

[0061] Figure 4 is an operational flowchart of the processor 120 according to various embodiments of the present disclosure. Referring to the operational flowchart 400, in operation 410, the processor 120 according to various embodiments can receive a three-dimensional medical image of the target body from an external device. The three-dimensional medical image of the target body may be an MRI image or CT image of the target body acquired in advance before surgery. The external device may be, for example, an MRI machine, a CT machine, or a server device.

[0062] In operation 420, the processor 120 according to various embodiments can acquire multiple first X-ray images of the target object using the X-ray apparatus 110. The processor 120 can acquire multiple first X-ray images of the target object by sequentially and selectively driving multiple X-ray sources 111. The multiple first X-ray images may be X-ray images of the target object taken from different directions.

[0063] In operation 430, the processor 120 according to various embodiments can match the second coordinate system of the 3D medical image to the first coordinate system. The processor 120 can match the second coordinate system of the 3D medical image onto the first coordinate system based on the position coordinates of multiple X-ray sources 111 and X-ray detectors 113 on the first coordinate system, multiple first X-ray images of the target body, and the 3D medical image. There are various methods for matching coordinate systems using 3D medical images and X-ray images. For example, the processor 120 can virtually acquire a projected image in which a 3D medical image acquired in advance before surgery is projected onto a 2D plane. This projected image may be a DRR (Digitally Reconstructed Radiograph) image. The processor 120 can perform coordinate system matching by comparing the generated virtual projected image with the actual X-ray image.

[0064] Specifically, the processor 120 can acquire multiple projection images of a 3D medical image projected from different directions and compare these multiple projection images with multiple first X-ray images. The processor 120 can determine which projection image has the highest similarity to the multiple first X-ray images. In other words, the processor 120 can find the projection image that is most similar to the X-ray image of the actual object. After finding the projection image with the highest similarity to the X-ray image, the processor 120 can estimate the position and orientation of the object in the 3D space based on the projection angle used to generate that projection image. In one embodiment, the processor 120 can also select points that are likely to be featureful on the X-ray image, calculate their approximate positions, and then search for the projection image that is most similar to the actual X-ray image based on a virtual projection image around those positions. In this case, the time required to find a matching projection image can be reduced. According to one embodiment, the matching process between the X-ray image and the 3D medical image described above may be possible with only one X-ray image, but a more precise matching result can be obtained by performing a coordinate system matching process using two or more X-ray images.

[0065] In operation 440, the processor 120 according to various embodiments can determine the position of the object in the first coordinate system based on the matching results. The position and / or orientation of the object in the first coordinate system can be determined by aligning the second coordinate system of the 3D medical image, which uses the aforementioned 3D medical image and multiple first X-ray images, with the first coordinate system of the system 10. In other words, the user can accurately confirm the position of the object in the first coordinate system that appears in the 3D medical image.

[0066] In operation 450, the processor 120 according to various embodiments can acquire multiple second X-ray images of surgical instruments, including electrodes, using the X-ray apparatus 110. For example, the processor 120 can acquire multiple second X-ray images of surgical instruments by sequentially and selectively driving multiple X-ray sources 111.

[0067] In operation 460, the processor 120, according to various embodiments, can determine the position of the surgical instrument in the first coordinate system based on multiple second X-ray images. The matching operation performed in operation 450 of Figure 4 makes it possible to determine the position in the first coordinate system corresponding to a specific position in the X-ray image. Therefore, the position of the surgical instrument in the first coordinate system corresponding to the electrode positions of the surgical instrument appearing in multiple second X-ray images can be determined. The user (e.g., a physician) can confirm the position of the surgical instrument in the first coordinate system by taking X-ray images during surgery, enabling accurate surgery.

[0068] Figure 5 is an operational flowchart of the processor 120 according to various embodiments of the present disclosure. Referring to the operational flowchart 500, in operation 510, the processor 120 according to various embodiments can acquire the position coordinates of the marker 115 on the first coordinate system from the tracking sensor 130. The tracking sensor 130 can acquire the position coordinates of the marker 115 on the first coordinate system by measuring the position of the marker 115 attached to a predetermined position on the X-ray apparatus 110. Here, the first coordinate system may be a three-dimensional spatial coordinate system used in system 10.

[0069] In various embodiments, the processor 120 can, in operation 520, acquire the position coordinates of multiple X-ray sources 111 and X-ray detectors 113 on a first coordinate system based on the acquired position coordinates of the marker 115. Here, the position coordinates of multiple X-ray sources 111 can mean the position coordinates of each of the multiple X-ray sources 111. The memory 140 stores the first coordinate transformation relationship between the multiple X-ray sources 111 and X-ray detectors 113, and the second coordinate transformation relationship between the multiple X-ray sources 111 and the marker 115. The coordinate transformation relationship may be represented by a matrix vector. For example, the processor 120 can acquire the position coordinates of multiple X-ray sources 111 based on the position coordinates of the marker 115 and the second coordinate transformation relationship, and can acquire the position coordinates of the X-ray detectors 113 based on the acquired position coordinates of multiple X-ray sources 111 and the first coordinate transformation relationship. Specific methods for acquiring the first and second coordinate transformation relationships will be described later.

[0070] In various embodiments, the processor 120 can, in operation 530, store the position coordinates of multiple X-ray sources 111 and X-ray detectors 113 in the first coordinate system in the memory 140. The stored position coordinates of the multiple X-ray sources 111 and X-ray detectors 113 may be used in the process of matching the second coordinate system of the 3D medical image of the object to the first coordinate system of the system 10.

[0071] Figure 6 is an X-ray image 600 of a surgical instrument 610 according to various embodiments of the present disclosure, and Figure 7 is a diagram showing the position of the surgical instrument on a three-dimensional image 700 having a first coordinate system.

[0072] Referring to Figure 6, the user can use the X-ray device 110 to take a two-dimensional X-ray image 600 of the surgical instrument 610 to confirm that the surgical instrument 610 was accurately inserted into the affected area of ​​the target body during surgery. The surgical instrument 610, including the electrode 611, is displayed on the X-ray image 600. Subsequently, the processor 120 can determine which position on the three-dimensional first coordinate system corresponds to the position of the surgical instrument 610 on the two-dimensional X-ray image 600. Specifically, the matching operation performed in operation 430 of Figure 4 matches the second coordinate system of the three-dimensional medical image to the first coordinate system of the target body using the X-ray image of the target body and the three-dimensional medical image, so that the processor 120 can determine the position on the first coordinate system that corresponds to the position of the surgical instrument 610 on the X-ray image 600. Figure 7 shows the positions of the surgical instrument and its electrode 711 on the three-dimensional image 700. For example, the position of the electrode 711 of the surgical instrument in the first coordinate system, which corresponds to the position of the electrode 611 of the surgical instrument 610 in the X-ray image 600, may be determined on the 3D image 700. Therefore, the precise position of the electrode 711 of the surgical instrument on the 3D image 700, which has a 3D first coordinate system, can be determined using only the X-ray image 600 of the surgical instrument 610.

[0073] Figure 8 is an operational flowchart of the processor 120 according to various embodiments of this disclosure. Specifically, operational flowchart 800 is a specific operational flowchart relating to operation 430 in Figure 4.

[0074] In operation 810, the processor 120 according to various embodiments can acquire multiple projected images by projecting a 3D medical image onto a 2D plane. The multiple projected images may be 2D images acquired when it is assumed that the 3D medical image is projected onto a 2D plane. These projected images may be DRR images.

[0075] In operation 820, the processor 120 according to various embodiments can compare multiple projection images with multiple first X-ray images.

[0076] In operation 830, the processor 120, according to various embodiments, can determine the projection image that is most similar to the multiple first X-ray images from among the multiple projection images. The processor 120 can determine the projection image that is most similar to the multiple first X-ray images from among the multiple projection images. Once the processor 120 finds the projection image that is most similar to the X-ray image, it can estimate the position and orientation of the object on the X-ray image in three-dimensional space, based on the projection angle when the projection image is generated.

[0077] In operation 840, the processor 120 according to various embodiments can match the second coordinate system of the 3D medical image to the first coordinate system by comparing multiple first X-ray images with the determined projection image. That is, the processor 120 can determine the position of the object on the first coordinate system by matching the second coordinate system of the 3D medical image to the first coordinate system of the system 10. Furthermore, by aligning the X-ray image with the determined projection image, the position on the first coordinate system corresponding to a specific position displayed in the X-ray image can also be determined.

[0078] Figures 9 to 11 illustrate the process of obtaining the first coordinate transformation relationship. Specifically, Figure 9 shows how to obtain the coordinate relationship between the X-ray source 111 and the X-ray detector 113; Figure 10 is a plan view showing the calibration tool used to obtain the coordinate relationship between the X-ray source 111 and the X-ray detector 113; and Figure 11 shows the position coordinates of multiple X-ray sources 111 relative to the coordinate system of the X-ray detector 113.

[0079] Referring to Figure 9, a calibration tool can be used to obtain the first coordinate transformation relationship. The calibration tool may include a first correction plate 910 and a second correction plate 920. The calibration tool may have a structure in which the first correction plate 910 and the second correction plate 920 are arranged in two layers separated by a predetermined distance, as shown in Figure 9, and may be a structure that can be placed on the X-ray detector 113. For example, the first correction plate 910 and the second correction plate 920 may be arranged in two layers parallel to each other, and their areas may be the same. Each of the first correction plate 910 and the second correction plate 920 may include a plurality of balls 911, 921 arranged in a grid. The plurality of balls 911, 921 may be metal balls that appear in the X-ray image. The calibration tool may be called, for example, a calibration jig.

[0080] The multiple first balls 911 arranged on the first correction plate 910 and the multiple second balls 921 arranged on the second correction plate 920 differ from each other in both number and position. For example, Figure 10 to As shown, multiple first balls 911 are arranged on the first correction plate 910 in a 5*5 configuration, the spacing between the multiple first balls 911 is d1, and the distance from the outermost first ball 911 to the edge of the first correction plate 910 may be d2. Multiple second balls 921 are arranged on the second correction plate 920 in a 4*4 configuration, the spacing between the multiple second balls 921 is d1, and the distance from the outermost second ball 921 to the edge of the second correction plate 920 may be d3. Also, the distance between the first correction plate 910 and the second correction plate 920 may be h. The above values ​​d1, d2, d3, and h are user-defined values.

[0081] Multiple X-ray sources 111 can be used to acquire multiple X-ray images of the calibration tool. For example, when using three X-ray sources 111, three X-ray images of the calibration tool can be acquired.

[0082] Subsequently, the processor 120 can obtain the coordinate relationship between the X-ray source 111 and the X-ray detector 113 using triangulation. Specifically, the processor 120 can obtain the position coordinates of multiple X-ray sources 111 centered on the X-ray detector 113. The method for calculating the coordinate relationship between the X-ray detector 113 and the multiple X-ray sources 111 using triangulation can be the general method of triangulation, so a detailed explanation is omitted.

[0083] In one embodiment, the processor 120 can obtain the position coordinates of each of the multiple X-ray sources 111 with reference to a detector center coordinate system based on the center of the X-ray detector 113. In another embodiment, the processor 120 can obtain the position coordinates of each of the multiple X-ray sources 111 with reference to a detector corner coordinate system centered on one vertex of the X-ray detector 113. As described above, the coordinate relationship between the X-ray detector 113 and each of the multiple X-ray sources 111 can be obtained using a calibration tool. Therefore, if the processor 120 knows the position coordinates of each of the multiple X-ray sources 111, it can calculate the position coordinates of the X-ray detector 113 using the above-described coordinate relationship, and vice versa. The above-described coordinate relationship between the X-ray detector 113 and each of the multiple X-ray sources 111 may be expressed in matrix vector or text format.

[0084] Figures 12 to 14 illustrate methods for obtaining a second coordinate transformation relationship according to various embodiments of the present disclosure. Specifically, Figure 12 shows a pivoting tool 1200 used to obtain the second coordinate transformation relationship, Figure 13 shows multiple X-ray images obtained by photographing a correction phantom using multiple X-ray sources 111, and Figure 14 shows the position coordinates of a marker 115 relative to the first X-ray source 111.

[0085] Referring to Figure 12, the pivot mechanism 1200 can be called a pivoting phantom or calibration phantom. The pivot mechanism 1200 may include a plurality of pivot balls formed at specified positions. The pivot balls may be balls made of metal so as to be detectable in an X-ray image, and may be called pivot points. The plurality of pivot balls may be, for example, four formed on the top surface of the pivot mechanism, four on the first side, and four on the second side, for a total of twelve, but the number and positions of the plurality of pivot balls are not limited thereto.

[0086] The user can position the pivot mechanism 1200 on the X-ray detector 113. The user can then acquire multiple X-ray images 1310, 1320, and 1330 of the pivot mechanism 1200 using multiple X-ray sources 111. Since each of the multiple X-ray sources 111 emits X-rays at different positions and angles, the positions of the pivot balls displayed in each of the multiple X-ray images 1310, 1320, and 1330 are different.

[0087] The processor 120 can, for example, extract the position coordinates of multiple pivot balls on the X-ray image 1310 acquired by the first X-ray source 111a. The processor 120 can, for example, extract the position coordinates of multiple pivot balls centered on the first X-ray source 111 using triangulation. In a similar manner, the processor 120 can extract the position coordinates of multiple pivot balls in the X-ray images 1320 and 1330 acquired by the second X-ray source 111b and the third X-ray source 111c.

[0088] Subsequently, the user can position the marker probe on multiple pivot balls. For example, the marker probe may be positioned on any one of the multiple pivot balls. This marker probe may be a marker that generates energy or a signal so that it can be sensed by the tracking sensor 130, and thus can be sensed by the tracking sensor 130. The marker probe may be a portable and compact marker.

[0089] The processor 120 can use the trekking sensor 130 to obtain the position coordinates of the marker probe in the first coordinate system and the position coordinates of the marker 115 attached to a specified position on the X-ray apparatus 110. This allows the processor 120 to obtain the positional relationship between the marker 115 attached to the specified position on the X-ray apparatus 110 and the marker probe. Furthermore, since the position coordinates of the marker probe are the same as the position coordinates of the pivot ball on which the marker probe is located, the processor 120 can obtain the positional relationship between the marker 115 attached to the specified position on the X-ray apparatus 110 and the first X-ray source 111. In other words, the processor 120 can obtain the coordinate transformation relationship between the first X-ray source 111 and the marker 115. By applying the above method to the second X-ray source 111 and the third X-ray source 111 as well, the processor 120 can obtain the first coordinate transformation relationship between multiple X-ray sources 111 and markers 115.

[0090] The processor 120 can store the acquired first and second coordinate transformation relationships in the memory 140. The calibration method described in Figures 9 to 14 may be performed once immediately after setting up the system 10, or it may be performed each time the position of the X-ray apparatus 110 is changed. The first and second coordinate transformation relationships stored in the memory 140 may be used to align the three-dimensional medical image of the object with multiple first X-ray images of the object.

[0091] Although the above method has been described with reference to specific embodiments, the method can also be embodied as computer-readable code on a computer-readable recording medium. A computer-readable recording medium includes any type of recording device on which data readable by a computer system is stored. Examples of computer-readable recording media may include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. Furthermore, the computer-readable recording medium may be distributed across a network of connected computer systems, and the computer-readable code may be stored and executed in a distributed manner. Functional programs, code, and code segments for embodying the above embodiments can be readily inferred by programmers in the art to which this disclosure belongs.

Claims

1. A system for determining the position of a target object or surgical instrument, An X-ray apparatus including a plurality of X-ray sources configured to irradiate the target object with X-rays, an X-ray detector configured to detect X-rays that have passed through the target object, and markers attached to predetermined positions on the X-ray apparatus, A tracking sensor configured to track the position of the aforementioned marker, A memory configured to store the position coordinates of the plurality of X-ray sources and the X-ray detectors on a first coordinate system relating to the system, a first coordinate transformation relationship between the plurality of X-ray sources and the X-ray detectors, and a second coordinate transformation relationship between the plurality of X-ray sources and the markers, Includes a processor, The aforementioned processor, A three-dimensional medical image of the target object is acquired from an external device. Using the aforementioned X-ray apparatus, multiple first X-ray images of the target object are acquired. Based on the position coordinates of the plurality of X-ray sources and X-ray detectors on the first coordinate system, the plurality of first X-ray images, and the three-dimensional medical image, the second coordinate system of the three-dimensional medical image is matched to the first coordinate system to obtain the matching result. Based on the matching results, the position coordinates of the object on the first coordinate system are determined. Using the aforementioned X-ray apparatus, multiple second X-ray images of surgical instruments, including electrodes, are acquired. The system is configured to determine the position coordinates of the surgical instrument on the first coordinate system based on the plurality of second X-ray images. The aforementioned processor, The position coordinates of the marker on the first coordinate system are obtained from the tracking sensor. Based on the position coordinates of the marker, the first coordinate transformation relationship, and the second coordinate transformation relationship, the position coordinates of the plurality of X-ray sources and the X-ray detectors on the first coordinate system are obtained. The system is configured to store the position coordinates of the plurality of X-ray sources and the X-ray detectors on the first coordinate system in the memory. system.

2. The aforementioned processor, The three-dimensional medical image is projected onto a two-dimensional plane to obtain multiple projected images. The plurality of projection images are compared with the plurality of first X-ray images. From among the plurality of projection images, the projection image with the highest similarity to the plurality of first X-ray images is determined. The system according to claim 1, configured to match the second coordinate system of the three-dimensional medical image to the first coordinate system based on the plurality of first X-ray images and the determined projection image.

3. The system according to claim 1, wherein the plurality of X-ray sources are arranged on the same plane.

4. The aforementioned multiple X-ray sources are arranged in a straight line at equal intervals. The system according to claim 3, wherein each of the plurality of X-ray sources irradiates the target object at a different angle from the others.

5. The system according to claim 1, wherein the plurality of X-ray sources are X-ray sources using carbon nanotubes.

6. The aforementioned X-ray apparatus is The system according to claim 5, further comprising a power supply unit configured to supply high voltage to the plurality of X-ray sources.

7. The aforementioned X-ray apparatus is A connecting member connected to the plurality of X-ray sources and the X-ray detector, The aforementioned plurality of X-ray sources are arranged in a first rotating section configured to rotate around a first rotation axis, The system according to claim 1, further comprising a second rotating part connected to the connecting member and configured to rotate about a second rotation axis.

8. The marker is attached to the designated position on the connecting member of the X-ray apparatus. The system according to claim 7, wherein the designated position is located within the field of view of the tracking sensor.

9. The marker is attached to a designated position on the first rotating part of the X-ray apparatus. The system according to claim 7, wherein the designated position is located within the field of view of the tracking sensor.

10. A method for operating a system for determining the position of a target or surgical instrument, which includes an X-ray apparatus including a plurality of X-ray sources configured to irradiate a target with X-rays, an X-ray detector configured to detect X-rays that have passed through the target, and markers placed at predetermined positions on the X-ray apparatus, a memory, and a processor, The processor performs the operation of receiving a three-dimensional medical image of the target object from an external device, The processor performs the operation of acquiring multiple first X-ray images of the target object using the X-ray apparatus, The processor performs the operation of matching the second coordinate system of the three-dimensional medical image to the first coordinate system and obtaining a matching result based on the position coordinates of the plurality of X-ray sources and the X-ray detectors on the first coordinate system of the system stored in the memory, the plurality of first X-ray images, and the three-dimensional medical image. The processor performs the operation of determining the position coordinates of the object on the first coordinate system based on the matching result, The processor performs the operation of acquiring multiple second X-ray images of surgical instruments, including electrodes, using the X-ray apparatus. The processor includes the operation of determining the position coordinates of the surgical instrument on the first coordinate system based on the plurality of second X-ray images, The processor performs the operation of obtaining the position coordinates of the marker on the first coordinate system from a tracking sensor configured to track the position of the marker. The processor performs the operation of obtaining the position coordinates of the multiple X-ray sources and the X-ray detector on the first coordinate system based on the position coordinates of the marker, the first coordinate transformation relationship between the multiple X-ray sources and the X-ray detector, and the second coordinate transformation relationship between the multiple X-ray sources and the marker, and A method comprising the operation of the processor storing the position coordinates of the plurality of X-ray sources and the X-ray detectors on the first coordinate system in the memory.

11. The aforementioned operation of matching and obtaining the matching result is: The processor performs the operation of projecting the three-dimensional medical image onto a two-dimensional plane and acquiring multiple projected images, The processor performs the operation of comparing the plurality of projection images with the plurality of first X-ray images, The processor performs the operation of determining the projection image that has the highest similarity to the plurality of first X-ray images from among the plurality of projection images, The method according to claim 10, wherein the processor performs the operation of matching the second coordinate system of the three-dimensional medical image to the first coordinate system based on the plurality of first X-ray images and the determined projection image.

12. The method according to claim 10, wherein the plurality of X-ray sources are arranged on the same plane.

13. The aforementioned multiple X-ray sources are arranged in a straight line at equal intervals. The method according to claim 12, wherein each of the plurality of X-ray sources irradiates the target object at an angle different from that of the others.

14. The method according to claim 10, wherein the plurality of X-ray sources are X-ray sources using carbon nanotubes.

15. The aforementioned X-ray apparatus is The method according to claim 14, further comprising a power supply unit configured to supply high voltage to the plurality of X-ray sources.

16. The aforementioned X-ray apparatus is A connecting member connected to the plurality of X-ray sources and the X-ray detector, The aforementioned plurality of X-ray sources are arranged in a first rotating section configured to rotate around a first rotation axis, The method according to claim 10, further comprising: a second rotating part connected to the connecting member and configured to rotate about a second rotation axis.

17. The marker is attached to the designated position on the connecting member of the X-ray apparatus. The method according to claim 16, wherein the designated position is located within the field of view of the tracking sensor.

18. The marker is attached to a designated position on the first rotating part of the X-ray apparatus. The method according to claim 16, wherein the designated position is located within the field of view of the tracking sensor.

Images