Medical 3D Image Measurement Device and Medical Image Integration System
The medical three-dimensional image measuring device and system address the obstruction issue by allowing the marker to change position and orientation, ensuring smooth tracking and improved surgical navigation.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- KOHYOUNG TECH
- Filing Date
- 2024-09-17
- Publication Date
- 2026-07-01
Smart Images

Figure 0007883546000001 
Figure 0007883546000002 
Figure 0007883546000003
Abstract
Description
Technical Field
[0001] The present disclosure relates to a medical three-dimensional video measurement device and a medical video integration system.
[0002] The present disclosure is derived from research conducted as part of the technical development support for the WC300 project, which is a national research and development project.
[0003] [Problem ID: S2482672, Research Project Name: WC300 Project Technical Development Support, Research Problem Name: Development of a Surgical Navigation Integrated Head and Neck Surgery Robot System with an Alignment Accuracy of 1 mm or Less, Contribution Rate: 1 / 1, Administrative Agency: Koyon Technology Co., Ltd., Research Period: March 1, 2017 to December 31, 2021]
Background Art
[0004] Recently, surgical navigation technology has been utilized to support surgeons in performing surgeries. By placing markers on a medical three-dimensional video measurement device that photographs surgical instruments and the surgical site of a patient, and tracking the markers through an optical tracking system having an imaging device such as a camera, the position and orientation of the surgical instruments can be tracked, and the video of the surgical site photographed through the medical three-dimensional video measurement device can be aligned with previously photographed medical images of the patient (e.g., CT images, MRI images). Through this, the position and orientation information of the surgical instruments on the medical images of the patient can be recognized on the system.
[0005] Using the images of the markers obtained through the optical tracking system, the position information and orientation information of the surgical instruments or the medical three-dimensional video measurement device on which the markers are placed can be obtained. For example, the position information can be defined by spatial coordinates such as coordinates on the X, Y, and Z axes of a rectangular coordinate system, and the orientation information can be defined by roll, pitch, and yaw.
[0006] Medical 3D image measuring devices can measure 3D images of a patient's surgical site in order to acquire and process image information related to the surgical site. For example, medical 3D image measuring devices utilize a method in which a specific pattern of light is irradiated onto the surgical site, and the resulting pattern is measured to acquire a 3D image of the target body. [Overview of the project] [Problems that the invention aims to solve]
[0007] When a medical 3D imaging device changes its position and / or orientation to image a surgical site, there is a limitation that the optical tracking system cannot change its position and / or orientation due to positions and / or orientations that make it impossible for the optical tracking system to track the markers of the medical 3D imaging device. In conventional technology, various surgical positions of the patient (e.g., Parkbench position, Prone position, Supine position) can cause interference from the patient or other objects (e.g., support table, surgical instruments, etc.), obstructing the field of view of the optical tracking system and making it difficult to image the markers. Embodiments of this disclosure solve the problems of the conventional technology described above. [Means for solving the problem]
[0008] One aspect of this disclosure provides an embodiment of a medical three-dimensional image measuring device. A typical embodiment of a medical three-dimensional image measuring device includes a light source that emits light; a camera configured to receive reflected light from an object and generate three-dimensional image information; a housing in which the camera is located and which forms an opening into which the reflected light flows; and a marker having a tracking surface, which is positioned in the housing so as to be able to change at least one of its relative position and relative orientation with respect to the opening, and is configured to be imaged by an external imaging device for tracking of its position and orientation.
[0009] Other aspects of this disclosure provide various embodiments of medical image matching systems. A medical image matching system according to a typical embodiment includes a light source that emits light, a camera configured to receive reflected light from an object and generate three-dimensional image information, a housing in which the camera is placed and which forms an opening into which the reflected light flows, and a medical three-dimensional image measuring device including a marker that is positioned in the housing so as to be able to change at least one of its relative position and relative orientation with respect to the opening and has a tracking surface configured to be imaged by an external imaging device for tracking of position and orientation; and an imaging device that images at least a portion of the tracking surface of the marker and images a tracking image, and includes an external electronic device configured to receive the three-dimensional image information and to determine the position and orientation of the marker using the tracking image and to determine the coordinates of the three-dimensional image information. [Effects of the Invention]
[0010] According to the embodiments of this disclosure, the position and / or orientation of the markers are changed by the patient fixation device, surgical instruments, the medical 3D image measuring device itself, and / or the operator, so as not to obstruct the field of view of the imaging device for the markers, and the tracking of the markers by an external electronic device is made smooth.
[0011] According to the embodiments of this disclosure, convenience can be improved by reducing the inconvenience of the operator having to ensure that their field of view of the markers of the optical tracking system is not obstructed, and conditions can be provided that allow the operator to concentrate more on imaging the surgical site and manipulating surgical instruments. For example, even in difficult surgical conditions where the patient is in a prone position and the surgical site is facing downwards, the operator can easily change the position and / or orientation of the markers of the medical 3D image measuring device to ensure a clear field of view of the markers of the optical tracking system.
[0012] According to embodiments of this disclosure, the position and / or orientation of a marker can be changed so that tracking of the marker by a medical 3D image measuring device is performed without interruption. [Brief explanation of the drawing]
[0013] [Figure 1] This is a block diagram showing a medical image integration system (10) according to one embodiment of the present disclosure. [Figure 2] This figure shows how a medical image integration system (10) according to one embodiment of the present disclosure is used. [Figure 3] This is a cross-sectional view of a medical 3D image measuring device (100) according to one embodiment of the present disclosure. [Figure 4] This is a perspective view of a medical three-dimensional image measuring device (101) according to the first embodiment of this disclosure. [Figure 5] Figure 4 is a cross-sectional view of the medical 3D image measuring device (101) cut along line S1-S1'. [Figure 6] This is an elevation view of a medical three-dimensional image measuring device (102) according to a second embodiment of the present disclosure. [Figure 7] This is an elevation view of a medical three-dimensional image measuring device (103) according to a third embodiment of the present disclosure. [Figure 8] This is an elevation view of a medical three-dimensional image measuring device (104) according to the fourth embodiment of this disclosure. [Figure 9] This is an elevation view of a medical three-dimensional image measuring device (105) according to the fifth embodiment of this disclosure. [Figure 10] This is an elevation view of a medical three-dimensional image measuring device (106) according to the sixth embodiment of this disclosure. [Modes for carrying out the invention]
[0014] The embodiments described herein are illustrative for the purpose of illustrating the technical concept of this disclosure. The scope of rights under this disclosure is not limited by the embodiments presented below or the specific descriptions thereof.
[0015] All technical and scientific terms used in this disclosure shall have the meanings commonly understood by those of ordinary skill in the art to which this disclosure pertains, unless otherwise defined. All terms used in this disclosure are selected for the purpose of more clearly explaining this disclosure and are not selected to limit the scope of rights under this disclosure.
[0016] Expressions such as "comprising", "including", "having", etc. used in this disclosure should be understood as open-ended terms that subsume the possibility of including other embodiments, unless otherwise specifically mentioned in the clauses or sentences in which such expressions are included.
[0017] The singular forms described in this disclosure may include the meanings of the plural forms unless otherwise specified, and this also applies to the singular forms described in the claims.
[0018] Expressions such as "first", "second", etc. used in this disclosure are used to distinguish multiple components from each other and do not limit the order or importance of such components.
[0019] In this disclosure, when a certain component is referred to as being "connected to" or "coupled to" another component, it should be understood that the certain component may be directly connected or coupled to the other component, or may be connected or coupled through another new component.
[0020] Hereinafter, each embodiment of this disclosure will be described with reference to the accompanying drawings. In the accompanying drawings, the same or corresponding components are given the same reference numerals. Also, in the description of the following embodiments, repeated descriptions of the same or corresponding components may be omitted. However, even if the description of a component is omitted, it is not intended that such a component is not included in a certain embodiment.
[0021] Figure 1 is a block diagram showing a medical image matching system (10) according to one embodiment of the present disclosure. Figure 2 is a block diagram showing how the medical image matching system (10) according to one embodiment of the present disclosure is used. Figure 3 is a cross-sectional view of a medical 3D image measuring device (100) according to one embodiment of the present disclosure.
[0022] Referring to Figures 1 to 3, the medical image matching system (10) may include a medical 3D image measuring device (100) and an external electronic device (20). The medical 3D image measuring device (100) and the external electronic device (20) can communicate with each other via wired or wireless means to send and receive various types of data (e.g., images). Even if some of the configuration shown in Figure 1 is omitted or replaced, it is unlikely that this will hinder the realization of the various embodiments disclosed in this document.
[0023] The medical 3D imaging device (100) may include a processor (110). The processor (110) can perform calculations and data processing related to the control and / or communication of each component of the medical 3D imaging device (100). The processor (110) can process signals received from the components of the medical 3D imaging device (100). The processor (110) can control the medical 3D imaging device (100) to send signals to an external electronic device (20). The processor (110) can load instructions or data received from other components of the medical 3D imaging device (100) into memory (not shown), process the instructions or data stored in memory, and store result data.
[0024] A medical three-dimensional image measuring device (100) may include a light source (120). The light source (120) may output patterned light. The patterned light output from the light source (120) may be shone onto an object (e.g., a patient) (P). The patterned light may be light having a specific pattern for measuring a three-dimensional image of the object (P), or light having a pattern of a constant or specific period. The patterned light may include, for example, random dot patterned light, checkered patterned light, patterned light with a sinusoidal brightness pattern, on-off patterned light with repeated bright and dark areas, or triangular wave patterned light with a triangular wave variation in brightness, but the form of the patterned light is not limited thereto.
[0025] The light source (120) may include a patterned section on which multiple patterns are formed and an LED that illuminates the patterned section with light. The light source (120) may also include a condensing lens (125) configured to focus the light output from the LED (121) and illuminate the patterned section. The light output from the LED (121) may pass through the patterned section (123) on which multiple patterns are formed, so that the patterns are reflected. The LED (121) may, for example, emit infrared light, but is not limited to this.
[0026] The medical 3D image measuring device (100) may include a camera (130). The camera (130) may be configured to capture an image of an object (P). The camera (130) can capture an image of the object (P) to acquire 3D image data of the object (P), and can process the acquired image data to obtain a 3D image of the object (P).
[0027] For example, the camera (130) can capture an image of an object (P) illuminated with patterned light. The processor (110) can generate a three-dimensional image of the object (P) based on a phase shift scheme using the patterned light. For example, when patterned light of a certain shape is shone onto an object (P) via a light source (120), the intensity of the light appearing on the surface of the object (P) may change due to the refraction of light on the surface. The processor (110) can generate phase data from the image generated through the camera (130) and generate a three-dimensional image.
[0028] In one embodiment, the camera (130) may be a light field camera (130) that generates a light field image. The light field camera (130) may be configured to retrospectively determine the depth of the object (P) after capturing the object (P) and to combine images having different object (P) depths. The image sensor of the light field camera (130) may have a retrospectively variable object (P) depth. The camera (130) can generate a light field image of the object (P) that reflects the pattern. The processor (110) can generate a three-dimensional image of the surface of the object (P) by generating phase data from the light field image and calculating the height of each point constituting the surface of the object (P).
[0029] The camera (130) may include a focusing lens (137), a lens array (135), and an image sensor (131). The focusing lens (137) can focus light coming from an object (P). The lens array (135) may be a lens in which multiple microlenses are arranged. The image sensor (131) can capture light that has passed through the lens array (135) and generate a light field image using the captured light. The image sensor (131) may be divided into regions corresponding to each of the multiple microlenses. For example, the image sensor (131) may include a CCD (charge-coupled device) sensor or a CMOS (complementary metal-oxide semiconductor) sensor.
[0030] A light field image generated by a camera (130) according to one embodiment may include multiple sub-images that store both color information and direction information of the light. For example, when a pattern light is shone on an object (P) and the reflected light reflected from the object (P) is received by the camera (130), the light field image may be an image formed by combining multiple sub-images that contain color information and direction information of the reflected light. The camera (130) can use the multiple sub-images included in the light field image to carry out a refocusing process. For example, in the refocusing process, the camera (130) can combine information from pixels in the light field image that corresponds to the depth of the desired object (P), and the optical path and direction calculated inversely, to generate an image of a desired depth. For example, the camera (130) can also generate an image in which the entire area of the object (P) is in focus during the refocusing process. In order for the camera (130) to form an accurate image of the target area, the distance between the medical 3D image measuring device (100) and the target area of the object (P) needs to be adjusted. However, when using a camera (130) that generates a light field image, the depth of the object (P) can be determined retrospectively, and a light field image that is in focus over the entire area of the object (P) can be generated, so there is no need to adjust the focal length in advance. Compared to a camera (130) that uses a general lens, a camera (130) that generates a light field image has a wider measurable depth range, and an image of the object (P) can be acquired in a single shot.
[0031] The medical 3D image measuring device (100) may include an optical path control element (140). The optical path control element (140) can reflect patterned light output from a light source (120) in a specific direction so that the patterned light is directed onto an object (P). The optical path control element (140) can transmit reflected light so that the reflected light from the object (P) reaches a camera (130). The optical path control element (140) may be, for example, a semi-transparent mirror. As an example, the light source (120) and the camera (130) may be arranged perpendicular to each other with respect to the optical path control element (140).
[0032] The medical three-dimensional image measuring device (100) may include housings (161, 162) that form the exterior. The housings (161, 162) may include a first housing (161) and a second housing (162) that are coupled to each other. The second housing (162) may be coupled to the first housing (161) in a manner that allows for relative movement. A marker (180) may be placed in the first housing (161).
[0033] A camera (130) is placed inside the housing (161, 162). An opening (162h) is formed in the housing (161, 162) through which the reflected light flows into the interior. A lens (not shown) made of a light-transmitting material may be placed in the opening (162h). A light source (120) may be placed inside the housing (161, 162). An optical path control element (140) may be placed inside the housing (161, 162).
[0034] In the embodiment disclosed in Figure 3, a light source (120), a camera (130), and an optical path control element (140) are arranged inside a first housing (161), and an aperture (162h) is formed in the second housing (162) so that patterned light output from the light source (120) illuminates the target object (P), but the disclosure is not limited thereto. In other embodiments, the light source (120), camera (130), and optical path control element (140) may be arranged in the second housing (162), and an aperture (162h) may also be formed therein.
[0035] In one embodiment, the user can use the medical three-dimensional image measuring device (100) while holding the first housing (161) or the second housing (162). The first housing (161) or the second housing (162) may include a configuration (e.g., a handle) that facilitates the movement, transport, and use of the medical three-dimensional image measuring device (100) by the user.
[0036] In other embodiments, the first housing (161) or the second housing (162) may be supported by other external devices (e.g., a stand (not shown) fixed to an operating table (30) or the floor). The stand may be configured to operate in a way that changes the position and orientation of the medical three-dimensional image measuring device (100).
[0037] A medical 3D image measuring device (100) may be configured to transmit information to an external electronic device (20) wirelessly or via a wired connection. The medical 3D image measuring device (100) may include a communication circuit (150). The communication circuit (150) may be configured to transmit information to the external electronic device (20). The communication circuit (150) can establish a communication channel with the external electronic device (20) and send and receive various data with the external electronic device (20). According to one embodiment, the communication circuit (150) may include a cellular communication module and be configured to connect to a cellular network (e.g., 3G, LTE, 5G, Wibro, or WiMAX). According to another embodiment, the communication circuit (150) may include a short-range communication module and be able to send and receive data with the external electronic device (20) using short-range communication (e.g., Wi-Fi, Bluetooth®, Bluetooth® Low Energy (BLE), UWB). The medical 3D image measuring device (100) may further include wired communication lines (151, 152, 153) for sending and receiving information with external electronic devices (20) (see Figures 4 to 10).
[0038] In one embodiment, the processor (110) can generate a three-dimensional image of the surface of the object (P) using a light field image of the object (P) acquired through the camera (130). For example, the intensity of the light appearing on the surface of the actual target area may change due to the bending of the surface of the object (P) when irradiated with patterned light. The processor (110) can use the light field image of the object (P) to measure the intensity of the light that has changed due to the bending of the surface of the object (P), generate phase data from this data, and calculate the height of each point that makes up the surface. By calculating the height of each point that makes up the surface of the object (P), the processor (110) can generate a three-dimensional image of the surface of the object (P). The processor (110) can transmit the three-dimensional image of the surface of the object (P) to an external electronic device (20) via a communication circuit (150).
[0039] The external electronic device (20) may include a controller (21). The controller (21) can perform calculations and data processing related to the control and / or communication of each component of the external electronic device (20). The controller (21) can process signals received from the components of the external electronic device (20). The processor (110) can process signals received from the medical 3D image measuring device (100) to send them out. The controller (21) can load received instructions or data into memory (not shown), process the instructions or data stored in memory, and store the resulting data.
[0040] The external electronic device (20) may include an imaging device (23). The imaging device (23) can image at least a portion of the tracking surface of a marker (180) attached to a medical 3D image measuring device (100) and form a tracking image on at least a portion of the tracking surface. For example, the tracking surface may be a patterned surface, in which case the tracking image is a patterned image. The imaging device (23) may include, for example, at least two or more cameras (23a, 23b) capable of forming images on at least a portion of the marker. The external electronic device (20) can determine the position and / or orientation of the marker (180) using the formed tracking image.
[0041] In an embodiment in which the marker (180) has a patterned surface as the tracking surface, when the external electronic device (20) acquires a pattern image of the marker (180), at least one sub-pattern may be extracted from the pattern image as the basic unit constituting the pattern of the marker (180). The position of the extracted at least one sub-pattern within the overall pattern may be determined, and the orientation of the marker (180) may be determined based on the determined position of the sub-pattern within the overall pattern. Here, the orientation of the marker (180) may mean the three-dimensional direction or orientation of the marker (180) relative to the imaging device (23). For example, the position of the marker (180) or the medical three-dimensional image measuring device (100) may be determined using triangulation based on two images having a stereoscopic relationship from images formed by an imaging device (23) including at least two cameras (23a, 23b). Once the position and orientation of the marker (180) are determined as described above, the position and orientation of the medical 3D image measuring device (100) to which the marker (180) is attached can be determined based on the geometric relationship between the marker (180) and the medical 3D image measuring device (100) to which the marker (180) is attached.
[0042] The external electronic device (20) may include a storage (25). The storage (25) can store various data used by at least one component of the external electronic device (20) (e.g., a controller (21)). For example, the storage (25) can store a three-dimensional image of the surface of an object (P) received by the controller (21) from a medical three-dimensional imaging device (100). For example, the storage (25) can store medical images (e.g., CT images, MRI images, etc.) received by the controller (21) from a medical device (not shown).
[0043] The external electronic device (20) can send and receive information with the medical 3D image measuring device (100) wirelessly or via a wired connection. The external electronic device (20) may include a communication circuit (27). The communication circuit (27) of the external electronic device (20) can establish a communication channel with the medical 3D image measuring device (100) and send and receive information with the medical 3D image measuring device (100). According to one embodiment, the communication circuit (27) of the external electronic device (20) may include a cellular communication module and be configured to connect to a cellular network (e.g., 3G, LTE, 5G, Wibro, or WiMAX). According to another embodiment, the communication circuit (27) of the external electronic device (20) may include a short-range communication module and be able to send and receive data with the medical 3D image measuring device (100) using short-range communication (e.g., Wi-Fi, Bluetooth®, Bluetooth® Low Energy (BLE), UWB). The external electronic device (20) may further include a wired communication line (not shown) for sending and receiving information with a medical 3D image measuring device (100).
[0044] The controller (21) can perform image matching between the image of the surface of the object (P) received from the medical 3D image measuring device (100) and the medical image of the object (P). The image of the surface of the object (P) generated by the medical 3D image measuring device (100) may be the external surface or a part thereof of the target previously included in the medical image. For example, if the medical image is an image modeling the 3D shape of the head of the object (P), the 3D image of the surface of the object (P) may be an image measuring the external shape of the eyes, nose, mouth, ears, etc. on the surface of the head of the object (P).
[0045] In one embodiment, the three-dimensional image of the surface of the object (P) may have a unique coordinate system (e.g., x1y1z1 coordinate system) related to the medical three-dimensional image measuring device (100). The coordinate system of the three-dimensional image of the surface of the object (P) may differ from the coordinate system of the medical image (e.g., x2y2z2) and may differ from the coordinate system of the external electronic device (20) (e.g., x0y0z0). The coordinate system of the external electronic device (20) may, for example, mean the coordinate system of the imaging device (23) of the external electronic device (20).
[0046] Referring to Figure 2, a user (e.g., a physician) (D) can acquire a three-dimensional image of the surface of an object (P) using a medical three-dimensional image measuring device (100). For example, the user (D) can irradiate the surface of the object (P) with patterned light using the medical three-dimensional image measuring device (100). The irradiated patterned light may form a pattern (PA) on the surface of the object (P).
[0047] In one embodiment, a medical three-dimensional image measuring device (100) can receive reflected light from an object (P) and generate a light field image of the object (P). The light field image of the object (P) may be, for example, an image formed by combining multiple sub-images relating to an irradiated pattern (PA). The medical three-dimensional image measuring device (100) can use the light field image of the object (P) to generate a three-dimensional image of the surface of the object (P). The medical three-dimensional image measuring device (100) can transmit the generated three-dimensional image of the surface of the object (P) to an external electronic device (20).
[0048] The external electronic device (20) can image at least a portion of the tracking surface of the marker (180) attached to the medical 3D image measuring device (100) via an imaging device, and can form a tracking image on at least a portion of the tracking surface. Based on the formed tracking image, the external electronic device (20) can determine the position and orientation of the medical 3D image measuring device (100) to which the marker (180) is attached.
[0049] The external electronic device (20) can convert the coordinate system of the 3D image relative to the surface of the object (P) to the coordinate system of the external electronic device (20). For example, the external electronic device (20) can convert the coordinate system of the 3D image relative to the surface of the object (P) to the coordinate system of the external electronic device (20) based on the position and orientation of the medical 3D image measuring device (100) determined through the marker (180).
[0050] The external electronic device (20) can convert the coordinate system of the medical image of the target body (P) received from the medical device to the coordinate system of the external electronic device (20). In various embodiments, the external electronic device (20) can perform image matching by unifying the coordinate systems between the 3D image of the surface of the target body (P) and the medical image of the target body (P).
[0051] Referring to Figure 3, the medical 3D image measuring device (100) may include at least one focusing lens (171, 172) for focusing light. The focusing lens (171, 172) may be positioned on the path of light. The focusing lens (171, 172) may be positioned around the optical path control element (140). At least one focusing lens (171, 172) may include a first focusing lens (171) positioned on the path of light from the light source (120) toward the optical path control element (140). At least one focusing lens (171, 172) may include a second focusing lens (172) positioned on the path of light from the optical path control element (140) toward the object (P).
[0052] A medical three-dimensional image measuring device (100) includes a light source (120) that emits light. The light source (120) may include an LED (121). The light source (120) may include a pattern section (123) on which multiple patterns are formed. Light emitted from the LED (121) can be irradiated onto the pattern section (123). The light source (120) may include a focusing lens (125) between the pattern section (123) and the LED (121) that is configured to focus the light emitted from the LED (121) and irradiate onto the pattern section (123). Light emitted from the LED (121) can pass through the pattern section (123) and reflect the pattern. According to one embodiment, light emitted from the light source (120) may be incident on an optical path control element (140). Light incident on the optical path control element (140) may be reflected in the direction in which a reflector (176) is positioned so that it can be irradiated onto an object (P). In other embodiments not shown, the medical three-dimensional image measuring device does not include an optical path control element (140), and the light output from the light source (120) can be immediately incident in the direction of the reflector (176).
[0053] In one embodiment, light can be reflected by a reflector (176) and irradiated onto the object (P) through an aperture (162h) of the second housing (162). In other embodiments not shown, the medical three-dimensional image measuring device does not include a reflector (176), and the light can be irradiated onto the object (P) through an aperture formed on the optical path (LA) without reflection by the reflector (176).
[0054] The medical 3D image measuring device (100) may be configured such that the optical path (LA) of light output from the light source (120) and irradiated onto the target object (P), and the optical path (LB) of reflected light reflected from the target object (P) and reaching the camera (130) are coaxial. The optical paths (LA) and (LB) may be superimposed coaxially in the section between the optical path control element (140) and the target object (P).
[0055] Light irradiated onto the object (P) can be reflected by the object (P). The reflected light from the object (P) can then be re-entered into the second housing (162) through the aperture (162h). In one embodiment, the reflected light can be reflected by a mirror (176) and incident on the optical path control element (140). In other embodiments not shown, the medical 3D image measuring device does not include a mirror (176), and the reflected light can be immediately incident on the optical path control element (140) without additional reflection.
[0056] Reflected light incident on the optical path control element (140) can pass through the optical path control element (140) and reach the camera (130). In other embodiments not shown, the medical 3D image measuring device does not include the optical path control element (140), and the reflected light can immediately be incident on the camera (130).
[0057] The medical three-dimensional image measuring device (100) may include a camera (130) configured to receive the reflected light and generate image information. The camera (130) may include a focusing lens (137) through which the reflected light passes. The camera (130) may include a lens array (135) through which a plurality of microlenses are arranged. The camera (130) may include an image sensor (131) that captures the reflected light. The reflected light can pass through the focusing lens (137) and the lens array (135) to reach the image sensor (131).
[0058] For example, the image sensor (131) can capture reflected light and generate a light field image of the object (P). The light field image of the object (P) may be an image relating to a pattern illuminating the object (P). The processor (110) can use the light field image to generate a three-dimensional image of the surface of the object (P). The processor (110) can transmit the three-dimensional image of the surface of the object (P) to an external electronic device (20) via the communication circuit (150).
[0059] A medical three-dimensional image measuring device (100) may include a marker (180) having a tracking surface. The marker (180) is positioned in a housing (161, 162). The marker (180) may be positioned in the first housing (161).
[0060] A marker (180) is positioned in the housing (161, 162) such that at least one of its relative position to the opening (162h) and its relative orientation to the opening (162h) can be changed. Specifically, the marker (180) is fixed to the first housing (161), and the second housing (162) can be coupled to the first housing (161) such that at least one of its relative position to the first housing (161) and its relative orientation to the first housing (161) can be changed. An opening (162h) can be formed in the second housing (162).
[0061] In one embodiment, the marker (180) may include a patterned surface (not shown) on which a pattern is formed as the tracking surface. The marker (180) may include a lens (not shown) configured such that at least a portion of the pattern, which appears uniquely depending on the direction from which the marker (180) is viewed, is identifiable from outside the marker (180). The lens of the marker (180) may be a ball lens. The patterned surface may have a concave curved shape.
[0062] The external electronic device (20) is configured to receive the image information generated by the camera (130) of the medical 3D image measuring device (100). The external electronic device (20) may include a communication circuit (27) for receiving the image information.
[0063] The external electronic device (20) may include an imaging device (23) that images at least a portion of the tracking surface of the marker (180) and forms a tracking image. The external electronic device (20) may be configured to determine the position and orientation of the marker (180) using the tracking image and to determine the coordinates of the image information.
[0064] In one embodiment, the external electronic device (20) can determine the location (or coordinate) and posture (or orientation) of the medical 3D imaging device (100) to which the marker (180) is attached, based on the imaged tracking image. The location of the medical 3D imaging device (100) may be defined in spatial coordinates, such as coordinates on the x, y, and z axes of a Cartesian coordinate system. The posture of the medical 3D imaging device (100) may be defined in terms of roll, pitch, and yaw. The external electronic device (20) can track the location and orientation of the medical 3D imaging device (100) by imaging the marker (180) attached to the medical 3D imaging device (100) via the imaging device (23).
[0065] For example, the imaging device (23) of the external electronic device (20) can form a pattern image of at least a portion of the pattern that is visually identifiable from outside the marker (180) via the ball lens of the marker (180). Once a pattern image of at least a portion of the pattern surface is acquired, the external electronic device (20) can process the information extracted from the pattern image of at least a portion of the pattern surface to determine the position and orientation of the marker (180). Based on the position and orientation of the marker (180), the external electronic device (20) can determine the position and orientation of the medical 3D image measuring device (100) to which the marker (180) is attached. A specific method for calculating the position and orientation of the marker (180) using an image of at least a portion of the pattern surface can be one of the general optical tracking methods.
[0066] The medical three-dimensional image measuring device (100) may be configured to change at least one of the relative position and relative orientation of a marker (180) with respect to an aperture (162h) into which reflected light flows. The medical three-dimensional image measuring device (100) may include a sensor (196) that senses displacement information resulting from the change of at least one of the relative position and relative orientation of the marker (180) with respect to the aperture. A communication circuit (150) may be configured to transmit the displacement information of the marker (180) to an external electronic device (20).
[0067] The external electronic device (20) can receive the displacement information of the marker (180). The external electronic device (20) may be configured to determine the coordinates of the image information generated by the medical 3D image measuring device (100) based on the displacement information. For example, the external electronic device (20) corrects the position or orientation of the medical 3D image measuring device (100) based on the displacement information, and the corrected position or orientation information of the medical 3D image measuring device (100) is used for image matching between the image information (image information generated by the medical 3D image measuring device) and medical images (e.g., CT images, MRI images).
[0068] The image information may have a unique coordinate system (e.g., x1y1z1 coordinate system) related to the medical three-dimensional image measuring device (100). The coordinate system of the image information may differ from the coordinate system of the medical image (e.g., x2y2z2) and may differ from the coordinate system of the external electronic device (20) (e.g., x0y0z0).
[0069] The medical image matching system (10) can convert the coordinate system of the medical image (e.g., x2y2z2) and the coordinate system of the image information (e.g., x1y1z1) to the coordinate system of the external electronic device (20) (e.g., x0y0z0). The external electronic device (20) can match the medical image and the image information, which have different coordinate systems. In order to match the medical image and the image information, the external electronic device (20) can extract a surface image from the medical image and match the extracted surface image with the received image information. Here, the surface image extracted from the medical image may have the same coordinate system as the medical image (e.g., x2y2z2). Furthermore, the external electronic device (20) can convert the coordinate system of the image information (e.g., x1y1z1) to the coordinate system of the external electronic device (20) (e.g., x0y0z0) via a marker (180) attached to the medical 3D image measuring device (100). Furthermore, the medical image and the surface image extracted from the medical image can also be converted to the coordinate system of the external electronic device (20) (for example, x0y0z0). The external electronic device (20) can perform matching between the image information and the medical image using any one of a variety of image matching algorithms. For example, the external electronic device (20) can perform matching using the ICP (interactive closest point) algorithm.
[0070] The second housing (162) may be configured to be movable relative to the first housing (161) so as to be able to change its relative position and / or orientation with respect to the first housing (161). Here, the relative movement includes the movement of the second housing (162) relative to the first housing (161) and the movement of the first housing (161) relative to the second housing (161). Through this, either the relative position or the relative orientation of the marker (180) fixed to the first housing (161) with respect to the opening (162h) can be changed.
[0071] The marker (180) is configured to change at least one of its relative position and orientation with respect to the opening (162h) by performing at least one of the following: (i) relative translational motion with respect to the opening (162h) and (ii) relative rotational motion with respect to the opening (162h) about a predetermined axis of rotation. The second housing (162) is configured to change at least one of the relative position and orientation of the marker (180) with respect to the opening (162h) by performing at least one of the following: (i) relative translational motion with respect to the first housing (161) and (ii) relative rotational motion with respect to the first housing (161) about a predetermined axis of rotation. As used in this disclosure, “axis of rotation” is a hypothetical axis and does not refer to an actual part of the device. In this disclosure, “configuration A performs relative translational motion with respect to configuration B” means that configuration A performs translational motion with respect to configuration B and configuration B performs translational motion with respect to configuration A. Furthermore, in this disclosure, the phrase "component C performs a relative rotational motion with respect to component D about a predetermined axis of rotation" encompasses both component C performing a rotational motion with respect to component D about a predetermined axis of rotation and component D performing a rotational motion with respect to component C about a predetermined axis of rotation.
[0072] For example, the marker (180) may be configured to perform translational motion relative to the opening (162h). Specifically, the marker (180) may perform translational motion relative to the opening (162h), or the opening (162h) may perform translational motion relative to the marker (180). Here, the second housing (162) may be configured to perform translational motion relative to the first housing (161). This allows the distance between the marker (180) and the opening (162h) to be changed. For example, in the third, fourth, and sixth embodiments described later, with reference to Figures 7, 8, and 10, the marker (180) can perform translational motion relative to the opening (162h).
[0073] As another example, the marker (180) can perform relative rotational motion with respect to the opening (162h) about a predetermined axis of rotation. Specifically, the marker (180) may rotate with respect to the opening (162h) about a predetermined axis of rotation, or the opening (162h) may rotate with respect to the marker (180) about a predetermined axis of rotation. Here, the second housing (162) may be configured to perform relative rotational motion with respect to the first housing (161) about a predetermined axis of rotation. This allows the distance and / or orientation of the marker (180) with respect to the opening (162h) to be changed. For example, in the first, second, fifth, and sixth embodiments described later with reference to Figures 4, 5, 6, 9, and 10, the marker (180) can perform relative rotational motion with respect to the opening (162h) about a predetermined axis of rotation.
[0074] In one embodiment with reference to Figure 3, the second housing (162) may be rotatably coupled to the first housing (161). The medical three-dimensional image measuring device (100) may include a bearing (191) positioned between the first housing (161) and the second housing (162) so that the second housing (162) is rotatable relative to the first housing (161). The second housing (162) can rotate relative to the first housing (161) with respect to the central axis of the bearing (191).
[0075] The sensor (196) can sense displacement information resulting from a change in at least one of the relative position and relative orientation of the first housing (161) relative to the second housing (162). The sensor (196) can be any one of a variety of sensors, or it can be embodied by two or more types of sensors.
[0076] In one embodiment, the sensor (196) can sense rotation angle information of the second housing (162) relative to the first housing (161). That is, when the second housing (162) rotates relative to the first housing (161), or when the first housing (161) rotates relative to the second housing (162), the sensor (196) can sense the rotation angle information. For example, the sensor (196) may be a gyro sensor or an encoder. Such a sensor can be applied to the first, second, fifth, and sixth embodiments described later, with reference to Figures 4, 5, 6, 9, and 10.
[0077] In other embodiments, the sensor (196) can sense distance information of the relative movement of the second housing (162) to the first housing (161). That is, when the second housing (162) moves relative to the first housing (161), or when the first housing (161) moves relative to the second housing (162), the sensor (196) can sense the distance information. For example, the sensor (196) may be an infrared sensor, a 3D sensor, an ultrasonic sensor, an RF sensor, a geomagnetic sensor, an encoder, etc. Such a sensor can be applied to the third, fourth, and sixth embodiments described later with reference to Figures 7, 8, and 10.
[0078] The processor (110) can transmit the displacement information to an external electronic device (20) via a communication circuit (150). In one embodiment, the processor (110) can transmit the rotation angle information to an external electronic device (20) via a communication circuit (150). In another embodiment, the processor (110) can transmit the distance information to an external electronic device (20) via a communication circuit (150).
[0079] Figure 4 is a perspective view of a medical 3D image measuring device (101) according to the first embodiment of this disclosure. Figure 5 is a cross-sectional view of the medical 3D image measuring device (101) of Figure 4, cut along line S1-S1'. The medical 3D image measuring device (101) according to the first embodiment will be described below, focusing on the differences from each embodiment of the medical 3D image measuring device (100) described above. The drawings show the direction (PD) in which light is emitted through the aperture (162h) and the direction (MD) in which the marker (180) faces forward.
[0080] Referring to Figures 4 and 5, the second housing (162) of the medical 3D image measuring device (101) is configured to be able to change its relative orientation to the first housing (161) by being positioned so as to be able to rotate relative to the first housing (161) around a predetermined axis of rotation (X1). The direction of rotation (C1) of the rotational movement is shown in the drawings.
[0081] The axis of rotation (X1) can extend parallel to the path (LB) of the reflected light just before it reaches the camera (130) (see Figure 3). For example, the axis of rotation (X1) can coincide with the optical path (LB). Through this, the position of the optical path (LB) can be maintained even if the first housing (161) and the second housing (162) undergo relative rotational motion to each other. A sensor (not shown) of the medical 3D image measuring device (101) may be configured to sense rotation angle information due to the change in the relative orientation of the second housing (162).
[0082] The medical three-dimensional image measuring device (101) may include a first communication line (151) configured to transmit the rotation angle information to an external electronic device (20). The medical three-dimensional image measuring device (101) may include a second communication line (152) configured to transmit the image information generated by the camera (130) to the external electronic device (20). The medical three-dimensional image measuring device (101) may include a third communication line (153) configured to transmit trigger information to the external electronic device (20) for syncing the image information generated by the camera (130) with the image captured by the imaging device (23) of the external electronic device (20).
[0083] The medical three-dimensional image measuring device (101) may be configured so that the rotational motion is performed automatically. For example, the medical three-dimensional image measuring device (101) may include a motor (not shown) that generates a driving force and gears (197a, 162a) to which the driving force is transmitted. A driven gear (162a) may be formed in the second housing (162). For example, the driven gear (162a) may be a ring gear formed on the inner wall surface of the second housing (162) along the direction of rotation (C1). The driving gear (197a) can mesh with the driven gear (162a) and rotate, and can transmit the driving force to the driven gear (162a).
[0084] Figure 6 is an elevation view of the medical 3D image measuring device (102) according to the second embodiment of this disclosure. The medical 3D image measuring device (102) according to the second embodiment will be described below, focusing on the differences from the medical 3D image measuring device (101) according to the first embodiment described above.
[0085] Referring to Figure 6, the medical three-dimensional image measuring device (102) may be configured so that the rotational motion is performed manually by the user. The medical three-dimensional image measuring device (102) includes a bearing (191) positioned between the first housing (161) and the second housing (162) so that the second housing (162) can rotate relative to the first housing (161). For example, the bearing (191) may be a ball bearing.
[0086] The sensor (196) of the medical 3D image measuring device (101) may be configured to sense rotation angle information due to the change in the relative orientation of the second housing (162). For example, the sensor (196) may include an encoder. The encoder may include a target portion (196a) formed in either the first housing (161) or the second housing, and a sensing portion (196b) formed in the other. The target portion (196a) may extend along a rotational direction (C1) about a rotation axis (X1). The sensing portion (196b) may be positioned to face a specific position of the target portion (196a), and the specific position may be configured to change due to the rotational motion. The rotation angle information can be sensed through the information about the specific position of the target portion (196a) sensed by the sensing portion (196b).
[0087] Figure 7 is an elevation view of a medical 3D image measuring device (103) according to the third embodiment of this disclosure. The medical 3D image measuring device (103) according to the third embodiment will be described below, focusing on the differences from the medical 3D image measuring devices (101, 102) according to the first and second embodiments described above.
[0088] Referring to Figure 7, the second housing (162) of the medical 3D image measuring device (103) is configured to allow relative translational motion with respect to the first housing (161), thereby enabling a change in its relative position to the first housing (161). The drawing shows the direction of movement (L1) of the translational motion.
[0089] The medical three-dimensional image measuring device (103) includes a sensor (196) configured to sense distance information that has moved relative to the first housing (161) due to the relative position change of the second housing (162). For example, the sensor (196) may be an infrared sensor. The sensor (196) may include a light-emitting unit (196g) that emits infrared rays (R1) and a light-receiving unit (196h) that senses the infrared rays (R1) and generates the distance information. The first communication line (151) may be configured to transmit the distance information to an external electronic device (20).
[0090] The medical three-dimensional image measuring device (103) may be configured so that the translational motion is performed automatically. For example, the medical three-dimensional image measuring device (103) may include a motor (not shown) that generates a driving force and gears (197c, 162c) to which the driving force is transmitted. A driven gear (162c) may be formed in the second housing (162). For example, the driven gear (162c) is a rack formed on the inner wall surface of the second housing (162) along the direction of movement (L1), and the driving gear (197c) may be a pinion. The driving gear (197c) can mesh with the driven gear (162c) and rotate, and can transmit the driving force to the driven gear (162a).
[0091] Figure 8 is an elevation view of the medical 3D image measuring device (104) according to the fourth embodiment of this disclosure. The medical 3D image measuring device (104) according to the fourth embodiment will be described below, focusing on the differences from the medical 3D image measuring device (103) according to the third embodiment described above.
[0092] Referring to Figure 8, the medical three-dimensional image measuring device (104) may be configured such that the translational motion is performed manually by the user. The medical three-dimensional image measuring device (104) includes a slider (192) formed in either a first housing (161) or a second housing (162), and a guide (193) formed in the other. The guide (193) is formed to extend along the direction of movement (L1). The slider (192) can slide along the guide (193) to move in the direction of movement (L1).
[0093] The sensor (196) of the medical three-dimensional image measuring device (104) may include a sensor (196) configured to sense distance information that has moved relative to the first housing (161) due to the relative position change of the second housing (162). For example, the sensor (196) may include a linear encoder. The linear encoder may include a target portion (196c) formed in either the first housing (161) or the second housing, and a sensing portion (196d) formed in the other. The target portion (196c) may extend along the direction of movement (C1). The sensing portion (196d) may be positioned to face a specific position of the target portion (196c), and the specific position may be configured to change due to the translational motion. The distance information can be sensed through the information about the specific position of the target portion (196c) sensed by the sensing portion (196d).
[0094] Figure 9 is an elevation view of the medical 3D image measuring device (105) according to the fifth embodiment of this disclosure. The medical 3D image measuring device (105) according to the fifth embodiment will be described below, focusing on the differences from the medical 3D image measuring devices (101, 102, 103, 104) according to the first to fourth embodiments described above.
[0095] Referring to Figure 9, the second housing (162) of the medical 3D image measuring device (105) is configured to be able to change its relative orientation to the first housing (161) by being positioned on the first housing (161) so as to be able to rotate relative to it about a predetermined axis of rotation (X2). The direction of rotation (C2) of the rotational movement is shown in the drawing. The medical 3D image measuring device (105) includes a hinge (194) on which the first housing (161) and the second housing (162) are configured to rotate relative to each other. The hinge (194) may be positioned on the axis of rotation (X2).
[0096] The axis of rotation (X2) may extend in a direction that crosses the path (LB) of the reflected light just before it reaches the camera (130). In the medical 3D image measuring device (105), the light source (120), camera (130), and optical path control element (140) may be arranged inside the second housing (162).
[0097] The medical three-dimensional image measuring device (105) may be configured so that the rotational motion is performed automatically. For example, the medical three-dimensional image measuring device (105) may include a motor (not shown) that generates a driving force and gears (197e, 162e) to which the driving force is transmitted. A driven gear (162e) may be formed in the second housing (162). For example, the driven gear (162e) may be formed on the periphery of the hinge (194). The driving gear (197e) can mesh with the driven gear (162e) and rotate, and can transmit the driving force to the driven gear (162e).
[0098] Figure 10 is an elevation view of the medical 3D image measuring device (106) according to the sixth embodiment of this disclosure. The medical 3D image measuring device (106) according to the sixth embodiment will be described below, focusing on the differences from the medical 3D image measuring devices (101, 102, 103, 104, 105) according to the first to fifth embodiments described above.
[0099] Referring to Figure 10, the second housing (162) of the medical 3D image measuring device (106) is configured to allow for relative rotational motion around a rotation axis (X1) relative to the first housing (161) and relative translational motion relative to the first housing (161), thereby enabling changes in its relative position and orientation to the first housing (161). The drawing shows the direction of rotation (C1) of the rotational motion and the direction of movement (L1) of the translational motion.
[0100] The medical three-dimensional image measuring device (106) may be configured so that the rotational and translational motions are performed manually by the user. The medical three-dimensional image measuring device (106) includes a bearing (191) positioned between the first housing (161) and the second housing (162) so that the second housing (162) can perform the relative rotational and translational motions with respect to the first housing (161). For example, the bearing (191) may be a ball bearing. The bearing (191) may be positioned on the inner wall surface of the second housing (162).
[0101] The sensor (196) of the medical 3D image measuring device (106) may be configured to sense distance information moved relative to the first housing due to the relative position change of the second housing (162) and to sense rotation angle information due to the relative orientation change of the second housing (162). For example, the sensor (196) may be an infrared sensor. The sensor (196) may include a light-emitting unit (196e) that emits infrared rays (R1) and a light-receiving unit (196f) that senses the infrared rays (R1) and generates the distance information. Multiple light-receiving units (196f) are arranged along the rotation direction (C1), and the rotation angle information can be generated depending on which of the multiple light-receiving units (196f) senses the infrared rays (R1).
[0102] While the technical concept of this disclosure has been explained above with reference to some embodiments and examples shown in the attached drawings, it is important to understand that various substitutions, modifications, and alterations may be made without departing from the technical concept and scope of this disclosure as understandable to a person with ordinary skill in the art to which this disclosure pertains. Furthermore, such substitutions, modifications, and alterations should be considered to fall within the scope of the attached claims.
Claims
1. A light source that outputs patterned light - The patterned light is light having a pattern with a specific period, A camera configured to receive reflected light from a target object after the aforementioned patterned light is reflected back onto the target object and generate three-dimensional image information, A processor configured to generate a three-dimensional image of the surface of the object using the three-dimensional image information, A housing in which the camera is positioned and which forms an opening into which the reflected light flows, A motor that generates driving force, Multiple gears on which the aforementioned driving force is transmitted, A marker having a tracking surface, which is positioned in the housing so as to be able to change at least one of the position relative to the opening or the orientation relative to the opening, and is configured to be imaged by an imaging device outside the housing for tracking of the position and orientation, The aforementioned housing is The first housing to which the marker is fixed, A second housing having the opening formed therein, coupled to the first housing so as to be able to change at least one of its relative position or relative orientation with respect to the first housing, Includes, The aforementioned multiple gears are, A driven gear formed on the inner wall surface of the second housing along the direction of movement, A driving gear that rotates in mesh with the driven gear and transmits the driving force to the driven gear, Includes, The preceding 2 housing is, The device is configured to allow relative translational movement with respect to the first housing, thereby enabling a change in its relative position to the first housing. Medical 3D imaging measurement device.
2. The sensor further includes a sensor configured to sense displacement information resulting from a change in at least one of the relative position and relative orientation of the marker with respect to the opening, A medical three-dimensional image measuring device according to claim 1.
3. The aforementioned marker, The device is configured to change at least one of the relative position or relative orientation to the opening by performing at least one of the following: relative translational motion with respect to the opening or relative rotational motion with respect to the opening around a predetermined axis of rotation. A medical three-dimensional image measuring device according to claim 1.
4. The preceding 2 housing is, The marker is configured to change at least one of the relative position or relative orientation of the marker with respect to the opening by performing at least one of the following: relative translational motion with respect to the first housing or relative rotational motion with respect to the first housing about a predetermined axis of rotation. A medical three-dimensional image measuring device according to claim 1.
5. The preceding 2 housing is, By being positioned in the first housing so as to be able to rotate relative to it about a predetermined axis of rotation, it is configured to be able to change its relative orientation to the first housing. The system further includes a sensor configured to sense rotational angle information due to the change in the relative orientation of the second housing, A medical three-dimensional image measuring device according to claim 1.
6. The invention further includes a bearing positioned between the first housing and the second housing so that the second housing can rotate relative to the first housing. A medical three-dimensional image measuring device according to claim 5.
7. The sensor is configured to sense distance information that has moved relative to the first housing due to the relative position change of the second housing. A medical three-dimensional image measuring device according to claim 5.
8. The invention further includes a bearing positioned between the first housing and the second housing such that the second housing can perform relative translational motion and relative rotational motion with respect to the first housing. A medical three-dimensional image measuring device according to claim 7.
9. The rotation axis extends parallel to the path of the reflected light just before it reaches the camera. A medical three-dimensional image measuring device according to claim 5.
10. The rotation axis extends in a direction that crosses the path of the reflected light just before it reaches the camera. A medical three-dimensional image measuring device according to claim 5.
11. The system further includes a sensor configured to sense distance information that has moved relative to the first housing due to the relative position change of the second housing, A medical three-dimensional image measuring device according to claim 1.
12. The invention further includes a bearing positioned between the first housing and the second housing so that the second housing can perform translational motion relative to the first housing. A medical three-dimensional image measuring device according to claim 11.
13. A three-dimensional image measuring device comprising: a light source that outputs patterned light; a camera configured to receive reflected light from an object and generate three-dimensional image information; a processor configured to generate a three-dimensional image of the surface of the object using the three-dimensional image information; a housing in which the camera is arranged and which forms an opening into which the reflected light flows; a motor that generates a driving force; a plurality of gears to which the driving force is transmitted; and a marker having a tracking surface arranged in the housing so as to be able to change at least one of the relative position or orientation to the opening, and configured to be imaged by an imaging device outside the housing for tracking the position and orientation. The imaging device includes an imaging device that captures at least a portion of the tracking surface of the marker and forms a tracking image, and an external electronic device configured to receive a three-dimensional image of the surface of the object, and to determine the position and orientation of the marker using the tracking image and to determine the coordinates of the three-dimensional image, The housing includes a first housing to which the marker is fixed, and a second housing having the opening formed therein, which is coupled to the first housing so as to be able to change at least one of its relative position or relative orientation with respect to the first housing. The plurality of gears include a driven gear formed on the inner wall surface of the second housing along the direction of movement, and a driving gear that rotates in mesh with the driven gear and transmits the driving force to the driven gear. The second housing is configured to be able to change its relative position to the first housing by being positioned so that it can perform translational motion relative to the first housing. Medical image integration system.
14. The aforementioned three-dimensional image measuring device is The system further includes a sensor configured to sense displacement information resulting from a change in at least one of the relative position and relative orientation of the marker with respect to the opening, The external electronic device is configured to determine the coordinates of the three-dimensional image based on the displacement information. A medical image matching system according to claim 13.