Medical three-dimensional image measuring device and medical image matching system

Abstract

To provide a medical three-dimensional image measuring device in which the location and / or posture of a marker can be changed so as not to block the field of view for the marker of an imaging device by a patient fixing device, a surgical instrument, the medical three-dimensional image measuring device itself, and / or an operator, so that tracking of the marker by the external electronic device can be performed smoothly.SOLUTION: A medical three-dimensional image measuring device 100 includes a light source configured to output light, a camera configured to generate three-dimensional image information by receiving reflected light generated by reflecting the light from an object, a housing 162 including the camera disposed therein and forming an opening 162h through which the reflected light enters into the housing, and a marker 180 disposed on the housing 162 to be capable of changing at least one of a relative location or a relative posture to the opening 162h and including a tracking surface configured to be imaged by an external imaging device for tracking a location and a posture.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to a medical three-dimensional image measuring device and a medical image registration system.

[0002] This disclosure is derived from research conducted as part of the national research and development program WC300 Project Technology Development Support.

[0003] [Project ID: S2482672, Research project name: WC300 Project Technology Development Support, Research project name: Development of a surgical navigation integrated head and neck surgery robot system with alignment accuracy of less than 1 mm, Contribution rate: 1 / 1, Sponsor: Kohyoung Technology Co., Ltd., Research period: 2017.03.01~2021.12.31] [Background technology]

[0004] Recently, surgical navigation technology has been used to support surgeons in surgery. By placing markers on the surgical instruments and on the medical 3D imaging measurement device that captures the surgical site of the patient, and tracking the markers through an optical tracking system with an imaging device such as a camera, the position and orientation of the surgical instruments can be tracked, and the images of the surgical site captured through the medical 3D imaging measurement device can be aligned with previously captured medical images of the patient (e.g. CT images, MRI images). Through this, the position and orientation information of the surgical instruments on the medical images of the patient can be recognized by the system.

[0005] Using an image of the marker acquired through the optical tracking system, position information and orientation information of the surgical instrument or medical 3D image measuring device on which the marker is placed can be acquired. For example, the position information can be defined by spatial coordinates such as coordinates on the X, Y, and Z axes of a Cartesian coordinate system, and the orientation information can be defined by roll, pitch, and yaw.

[0006] A medical 3D image measuring device can measure a 3D image of a surgical site of a patient in order to obtain and process image information on the surgical site of a patient. For example, a medical 3D image measuring device is used to irradiate a certain pattern of light onto the surgical site, measure the pattern generated, and obtain a 3D image of the target object from the pattern. Summary of the Invention [Problem to be solved by the invention]

[0007] When the medical 3D image measuring device changes its position and / or posture to capture an image of a surgical site, the position and / or posture of the medical 3D image measuring device cannot be changed due to the position and / or posture that makes it impossible for the optical tracking system to track the marker of the medical 3D image measuring device. In the conventional technology, there is a problem that the field of view of the optical tracking system is blocked by interference from the patient or other objects (e.g., a cradle, a surgical instrument, etc.) due to various surgical postures of the patient (e.g., parkbench posture, prone posture, supine posture), making it difficult to capture the image of the marker. The embodiment of the present disclosure solves the above-mentioned problem of the conventional technology. [Means for solving the problem]

[0008] One aspect of the present disclosure provides an embodiment of a medical 3D image measuring device. The medical 3D image measuring device according to a representative embodiment includes a light source that outputs light; a camera configured to receive reflected light of the light reflected on an object and generate 3D image information; a housing in which the camera is disposed and which forms an opening through which the reflected light flows; and a marker having a tracking surface that is disposed in the housing such that at least one of a relative position and a relative orientation with respect to the opening can be changed and is configured to be imaged by an external imaging device for tracking the position and orientation.

[0009] Another aspect of the present disclosure provides various embodiments of a medical image registration system. The medical image registration system according to a representative embodiment includes a medical three-dimensional image measurement device including a light source for outputting light, a camera configured to receive reflected light of the light reflected by a target object and generate three-dimensional image information, a housing in which the camera is disposed and which forms an opening through which the reflected light flows, and a marker disposed in the housing such that at least one of a relative position and a relative orientation with respect to the opening can be changed and having a tracking surface configured to be imaged by an external imaging device for tracking position and orientation; and an external electronic device including an imaging device for imaging at least a part of the tracking surface of the marker to form a tracking image, configured to receive the three-dimensional image information, and configured to determine a position and orientation of the marker using the tracking image and determine coordinates of the three-dimensional image information. Effect of the Invention

[0010] According to an embodiment of the present disclosure, the position and / or orientation of the marker can be changed by the patient fixation device, surgical instruments, the medical 3D image measurement device itself, and / or the operator so as not to obstruct the imaging device's view of the marker, thereby facilitating tracking of the marker by an external electronic device.

[0011] According to the embodiment of the present disclosure, it is possible to improve convenience by reducing the hassle of the operator trying to ensure that the field of view of the marker of the optical tracking system is not obstructed, and to provide the operator with a condition in which he or she can further concentrate on photographing the surgical site and operating the surgical instrument. For example, even in a difficult surgical condition in which the patient is in a prone position and the surgical site faces downward, the operator can easily secure the field of view of the marker of the optical tracking system by changing the position and / or posture of the marker of the medical 3D image measuring device.

[0012] According to an embodiment of the present disclosure, the position and / or orientation of a marker can be changed so that tracking of the marker by the medical three-dimensional image measuring device can be performed without interruption. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a block diagram illustrating a medical image registration system 10 according to one embodiment of the present disclosure. [Diagram 2] FIG. 1 illustrates an embodiment of a medical image registration system (10) in use according to the present disclosure. [Diagram 3] 1 is a cross-sectional view of a medical three-dimensional image measuring device (100) according to an embodiment of the present disclosure. [Figure 4] FIG. 1 is a perspective view of a medical three-dimensional image measuring device (101) according to a first embodiment of the present disclosure. [Diagram 5] 5 is a cross-sectional view of the medical three-dimensional image measuring device (101) of FIG. 4 taken along line S1-S1'. [Figure 6] FIG. 1 is an elevational view of a medical three-dimensional imaging measuring device (102) according to a second embodiment of the present disclosure. [Figure 7] FIG. 11 is an elevational view of a medical three-dimensional image measuring device (103) according to a third embodiment of the present disclosure. [Figure 8] FIG. 11 is an elevational view of a medical three-dimensional imaging measuring device (104) according to a fourth embodiment of the present disclosure. [Figure 9] FIG. 13 is an elevational view of a medical three-dimensional image measuring device (105) according to a fifth embodiment of the present disclosure. [Figure 10] FIG. 13 is an elevation view of a medical three-dimensional imaging measurement device (106) according to a sixth embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] The embodiments of the present disclosure are illustrated for the purpose of explaining the technical idea of ​​the present disclosure, and the scope of the rights of the present disclosure is not limited to the embodiments presented below or the specific explanations of these embodiments.

[0015] All technical and scientific terms used in this disclosure have the meanings commonly understood by those of ordinary skill in the art to which this disclosure belongs, unless otherwise defined. All terms used in this disclosure are selected for the purpose of more clearly describing this disclosure, and are not selected to limit the scope of the rights of this disclosure.

[0016] As used in this disclosure, terms such as "including," "comprising," "having," and the like, should be understood to be open-ended terms that include the possibility of including other embodiments, unless otherwise stated in the phrase or sentence in which the term appears.

[0017] In the present disclosure, the singular expressions may include the plural meaning unless otherwise specified, and this also applies to the singular expressions described in the claims.

[0018] The terms "first", "second", etc. used in this disclosure are used to distinguish multiple elements from each other, and do not limit the order or importance of the elements.

[0019] In this disclosure, when a component is referred to as being "coupled" or "connected" to another component, it should be understood that the component may be directly coupled or connected to the other component, or may be coupled or connected through an intermediary of a new other component.

[0020] Hereinafter, each embodiment of the present disclosure will be described with reference to the accompanying drawings. In the accompanying drawings, the same or corresponding components are given the same reference numerals. In addition, in the following description of the embodiments, duplicated descriptions of the same or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such a component is not included in a certain embodiment.

[0021] Fig. 1 is a block diagram showing a medical image registration system (10) according to an embodiment of the present disclosure. Fig. 2 is a block diagram showing how the medical image registration system (10) according to an embodiment of the present disclosure is used. Fig. 3 is a cross-sectional view of a medical 3D image measuring device (100) according to an embodiment of the present disclosure.

[0022] 1 to 3, a medical image registration system 10 may include a medical 3D image measuring device 100 and an external electronic device 20. The medical 3D image measuring device 100 and the external electronic device 20 are communicatively connected to each other via wire or wirelessly, and may transmit and receive various data (e.g., images). Even if some of the components shown in FIG. 1 are omitted or replaced, it is believed that there will be no problem in realizing various embodiments disclosed herein.

[0023] The medical 3D image measuring device 100 may include a processor 110. The processor 110 may perform calculations and data processing related to control and / or communication of each component of the medical 3D image measuring device 100. The processor 110 may process signals received from components of the medical 3D image measuring device 100. The processor 110 may control the medical 3D image measuring device 100 to send signals to an external electronic device 20. The processor 110 may load commands or data received from other components of the medical 3D image measuring device 100 into a memory (not shown), process the commands or data stored in the memory, and store the result data.

[0024] The medical 3D image measuring device 100 may include a light source 120. The light source 120 may output a patterned light. The patterned light output from the light source 120 may be irradiated onto a target object (e.g., a patient) P. The patterned light may be light having a specific pattern to measure a 3D image of the target object P, or light having a constant or specific periodic pattern. The patterned light may include, for example, a random dot patterned light, a checkered patterned light, a striped patterned light having a sine wave pattern of brightness, an on-off patterned light in which bright and dark parts are repeatedly shown, or a triangular wave patterned light in which the brightness changes in a triangular wave pattern, but the shape of the patterned light is not limited thereto.

[0025] The light source (120) may include a pattern portion on which a plurality of patterns are formed, and an LED for irradiating light onto the pattern portion. The light source (120) may include a condensing lens (125) configured to condense light output from the LED (121) and irradiate the light onto the pattern portion. The light output from the LED (121) may pass through the pattern portion (123) on which a plurality of patterns are formed, and the pattern may be reflected. The LED (121) may emit, for example, infrared light, but is not limited thereto.

[0026] The medical 3D image measuring device 100 may include a camera 130. The camera 130 may be configured to capture an image of a target object P. The camera 130 may capture an image of the target object P to obtain 3D image data of the target object P, and may process the obtained image data to obtain a 3D image of the target object P.

[0027] For example, the camera (130) can capture an image of the object (P) illuminated with pattern light. The processor (110) can generate a 3D image of the object (P) based on a phase shift method using the pattern light. For example, when a certain type of pattern light is illuminated onto the object (P) through the light source (120), the intensity of the light appearing on the surface may change depending on the curvature of the surface of the object (P). The processor (110) can generate phase data from the image generated through the camera (130) to generate a 3D image.

[0028] In one embodiment, the camera 130 may be a light field camera 130 that generates a light field image. The light field camera 130 may be configured to determine the depth of the object P after capturing an image of the object P and combine images having different object P depths. The image sensor of the light field camera 130 may have a variable object P depth after capturing an image of the object P. The camera 130 may generate a light field image of the object P reflecting a pattern. The processor 110 may generate phase data from the light field image and calculate the height of each point constituting the surface of the object P to generate a 3D image of the surface of the object P.

[0029] The camera 130 may include a condenser lens 137, a lens array 135, and an image sensor 131. The condenser lens 137 may condense light entering from a target object P. The lens array 135 may be a lens in which a plurality of microlenses are arranged. The image sensor 131 may capture light passing through the lens array 135 and generate a light field image using the captured light. The image sensor 131 may be divided into regions corresponding to each of the plurality of microlenses. For example, the image sensor 131 may include a charge-coupled device (CCD) sensor or a complementary metal-oxide semiconductor (CMOS) sensor.

[0030] According to an embodiment, the light field image generated by the camera 130 may include a plurality of sub-images that store both color information and direction information of light. For example, when a pattern light is irradiated on the object P and the reflected light reflected from the object P is received by the camera 130, the light field image may be an image obtained by combining a plurality of sub-images including color information and direction information of the reflected light. The camera 130 may perform a refocusing process using a plurality of sub-images included in the light field image. For example, the camera 130 may generate an image of a desired depth by combining information on pixels corresponding to the depth of the desired object P and the light path and direction calculated inversely therefrom among the pixels of the light field image during the refocusing process. For example, the camera 130 may generate an image in which the entire area of ​​the object P is in focus during the refocusing process. In order to allow the camera (130) to accurately form an image of the target area, the distance between the medical 3D image measuring device (100) and the target area of ​​the object (P) needs to be adjusted, but when a camera (130) that generates a light field image is used, the depth of the object (P) can be determined after the fact, and a light field image that is in focus for the entire area of ​​the object (P) can be generated, so there is no need to adjust the focal length in advance. The camera (130) that generates a light field image has a wider measurable depth range than a camera (130) that uses a general lens, and an image of the object (P) can be obtained in one shot.

[0031] The medical 3D image measuring device (100) may include an optical path control element (140). The optical path control element (140) may reflect the pattern light output from the light source (120) in a specific direction so that the pattern light is irradiated onto the object (P). The optical path control element (140) may transmit the reflected light reflected from the object (P) so that the reflected light reaches the camera (130). The optical path control element (140) may be, for example, a semi-transparent mirror. As an example, the light source (120) and the camera (130) may be disposed perpendicular to each other with respect to the optical path control element (140).

[0032] The medical 3D image measuring device (100) may include housings (161, 162) that form the exterior. The housings (161, 162) may include a first housing (161) and a second housing (162) that are coupled to each other. The second housing (162) may be coupled to the first housing (161) so as to be movable relative to the first housing (161). A marker (180) may be disposed on the first housing (161).

[0033] A camera (130) is disposed inside the housings (161, 162). An opening (162h) is formed in the housings (161, 162) through which the reflected light enters. A lens (not shown) made of a light-transmitting material may be disposed in the opening (162h). ​​A light source (120) may be disposed inside the housings (161, 162). A light path control element (140) may be disposed inside the housings (161, 162).

[0034] 3, the light source (120), the camera (130), and the light path control element (140) are disposed inside the first housing (161), and the second housing (162) has an opening (162h) formed therein so that the pattern light output from the light source (120) is irradiated onto the target object (P), but the present disclosure is not limited thereto. In other embodiments, the light source (120), the camera (130), and the light path control element (140) may be disposed in the second housing (162), and the opening (162h) may also be formed therein.

[0035] In one embodiment, a user can use the medical 3D image measurement device 100 while holding the first housing 161 or the second housing 162. The first housing 161 or the second housing 162 can include a structure (e.g., a handle) that allows the user to easily move, carry, and use the medical 3D image measurement device 100.

[0036] In other embodiments, the first housing 161 or the second housing 162 may be supported by another external device (e.g., a stand (not shown) fixed to the operating table 30 or a floor, etc.). The stand may be configured to operate so as to change the position and orientation of the medical 3D image measurement device 100.

[0037] The medical 3D image measuring device (100) may be configured to transmit information to the external electronic device (20) wirelessly or via a wire. The medical 3D image measuring device (100) may include a communication circuit (150). The communication circuit (50) may be configured to transmit information to the external electronic device (20). The communication circuit (150) may establish a communication channel with the external electronic device (20) and transmit and receive various data with the external electronic device (20). According to one embodiment, the communication circuit (150) may include a cellular communication module and be configured to be connected to a cellular network (e.g., 3G, LTE, 5G, Wibro, or Wimax). According to another embodiment, the communication circuit (150) may include a short-range communication module and may transmit and receive data with the external electronic device (20) using short-range communication (e.g., Wi-Fi, Bluetooth (registered trademark), Bluetooth (registered trademark) Low Energy (BLE), UWB). The medical 3D image measuring device (100) may further include wired communication lines (151, 152, 153) for transmitting and receiving information to and from an external electronic device (20) (see FIGS. 4 to 10).

[0038] In one embodiment, the processor (110) can generate a 3D image of the surface of the object (P) using a light field image of the object (P) acquired through the camera (130). For example, the intensity of the irradiated pattern light appearing on the surface of the actual photographed object area may change depending on the curvature of the surface of the photographed object area of ​​the object (P). The processor (110) can measure the intensity of light that changes due to the curvature of the surface of the object (P) using the light field image of the object (P), generate phase data from this, and calculate the height of each point that constitutes the surface. The processor (110) can generate a 3D image of the surface of the object (P) by calculating the height of each point that constitutes the surface of the object (P). The processor (110) can transmit the 3D image of the surface of the object (P) to the external electronic device (20) via the communication circuit (150).

[0039] The external electronic device 20 may include a controller 21. The controller 21 may perform calculations and data processing related to control and / or communication of each component of the external electronic device 20. The controller 21 may process signals received from the components of the external electronic device 20. The processor 110 may process signals received from the medical 3D image measurement device 100 to be sent. The controller 21 may load received commands or data into a memory (not shown), process the commands or data stored in the memory, and store the result data.

[0040] The external electronic device (20) may include an imaging device (23). The imaging device (23) may capture at least a portion of a tracking surface of a marker (180) attached to the medical 3D image measurement device (100) and form a tracking image for at least a portion of the tracking surface. For example, the tracking surface may be a pattern surface, and in this case, the tracking image is a pattern image. The imaging device (23) may include, for example, at least two or more cameras (23a, 23b) capable of forming an image for at least a portion of the marker. The external electronic device (20) may determine the position and / or orientation of the marker (180) using the formed tracking image.

[0041] In an embodiment in which the marker (180) has a pattern surface as the tracking surface, when the external electronic device (20) acquires a pattern image of the marker (180), at least one of the sub-patterns may be extracted from the pattern image as a basic unit constituting the pattern of the marker (180). The position of the extracted at least one sub-pattern within the entire pattern may be determined, and the attitude of the marker (180) may be determined based on the determined position of the sub-pattern within the entire pattern. Here, the attitude of the marker (180) may refer to the three-dimensional direction or orientation of the marker (180) relative to the imaging device (23). For example, the position of the marker (180) or the medical three-dimensional image measurement device (100) may be determined using triangulation based on two images having a stereoscopic relationship among images formed by the imaging device (23) including at least two or more cameras (23a, 23b). Once the position and orientation of the marker (180) is determined as described above, the position and orientation of the medical 3D image measuring device (100) to which the marker (180) is attached can be determined based on the geometric relationship between the marker (180) and the medical 3D image measuring device (100) to which the marker (180) is attached.

[0042] The external electronic device 20 may include a storage 25. The storage 25 may store various data used by at least one component (e.g., the controller 21) of the external electronic device 20. For example, the storage 25 may store a 3D image of the surface of the object P received from the medical 3D image measuring device 100 by the controller 21. For example, the storage 25 may store medical images (e.g., CT images, MRI images, etc.) received from a medical device (not shown) by the controller 21.

[0043] The external electronic device (20) can transmit and receive information to and from the medical 3D image measuring device (100) wirelessly or via wires. The external electronic device (20) can include a communication circuit (27). The communication circuit (27) of the external electronic device (20) can establish a communication channel with the medical 3D image measuring device (100) and transmit and receive information to and from the medical 3D image measuring device (100). According to one embodiment, the communication circuit (27) of the external electronic device (20) can include a cellular communication module and be configured to be connected to a cellular network (e.g., 3G, LTE, 5G, Wibro, or Wimax). According to another embodiment, the communication circuit (27) of the external electronic device (20) can include a short-range communication module and can transmit and receive data to and from the medical 3D image measuring device (100) using short-range communication (e.g., Wi-Fi, Bluetooth (registered trademark), Bluetooth (registered trademark) Low Energy (BLE), UWB). The external electronic device (20) may further include a wired communication line (not shown) for transmitting and receiving information to and from the medical 3D image measuring device (100).

[0044] The controller 21 can perform image matching between the image of the surface of the object P received from the medical 3D image measuring device 100 and the medical image of the object P. The image of the surface of the object P generated by the medical 3D image measuring device 100 can be the external surface of a target or a part thereof that was previously included in the medical image. For example, if the medical image is an image that models the 3D shape of the head of the object P, the 3D image of the surface of the object P can be an image that measures the external shapes of the eyes, nose, mouth, ears, etc. on the surface of the head of the object P.

[0045] In one embodiment, the 3D image of the surface of the object (P) may have a unique coordinate system (e.g., x1y1z1 coordinate system) with respect to the medical 3D imaging measurement device (100). The coordinate system of the 3D image of the surface of the object (P) may be different from the coordinate system of the medical image (e.g., x2y2z2) and different from the coordinate system of the external electronic device (20) (e.g., x0y0z0). The coordinate system of the external electronic device (20) may refer to, for example, the coordinate system of the imaging device (23) of the external electronic device (20).

[0046] 2, a user (e.g., a doctor) (D) can obtain a 3D image of a surface of a target object (P) using a medical 3D image measuring device (100). For example, the user (D) can irradiate a pattern light onto the surface of the target object (P) using the medical 3D image measuring device (100). A pattern (PA) can be formed on the surface of the target object (P) by the irradiated pattern light.

[0047] In one embodiment, the medical 3D image measurement device (100) can receive light reflected from the object (P) and generate a light field image of the object (P). The light field image of the object (P) can be, for example, an image obtained by combining a plurality of sub-images related to the irradiated pattern (PA). The medical 3D image measurement device (100) can generate a 3D image of the surface of the object (P) using the light field image of the object (P). The medical 3D image measurement device (100) can transmit the generated 3D image of the surface of the object (P) to an external electronic device (20).

[0048] The external electronic device 20 can capture at least a portion of the tracking surface of the marker 180 attached to the medical 3D image measuring device 100 through the imaging device, and form a tracking image for at least a portion of the tracking surface. The external electronic device 20 can determine the position and orientation of the medical 3D image measuring device 100 to which the marker 180 is attached based on the formed tracking image.

[0049] The external electronic device 20 can convert the coordinate system of the 3D image for the surface of the object P into the coordinate system of the external electronic device 20. For example, the external electronic device 20 can convert the coordinate system of the 3D image for the surface of the object P into the coordinate system of the external electronic device 20 based on the position and orientation of the medical 3D image measuring device 100 determined through the markers 180.

[0050] The external electronic device 20 can convert the coordinate system of the medical image of the object P received from the medical device into the coordinate system of the external electronic device 20. According to various embodiments, the external electronic device 20 can perform image registration by unifying the coordinate systems between the 3D image of the surface of the object P and the medical image of the object P.

[0051] Referring to FIG. 3, the medical 3D image measuring device (100) may include at least one condensing lens (171, 172) for condensing light. The condensing lenses (171, 172) may be disposed on a path of light. The condensing lenses (171, 172) may be disposed around the light path control element (140). The at least one condensing lens (171, 172) may include a first condensing lens (171) disposed on a path of light from the light source (120) to the light path control element (140). The at least one condensing lens (171, 172) may include a second condensing lens (172) disposed on a path of light from the light path control element (140) to the object (P).

[0052] The medical 3D image measuring device (100) includes a light source (120) that outputs light. The light source (120) may include an LED (121). The light source (120) may include a pattern unit (123) on which a plurality of patterns are formed. The light output from the LED (121) may be irradiated onto the pattern unit (123). The light source (120) may include a condenser lens (125) between the pattern unit (123) and the LED (121) configured to condense the light output from the LED (121) and irradiate it onto the pattern unit (123). The light output from the LED (121) may pass through the pattern unit (123) to reflect the pattern. According to an embodiment, the light output from the light source (120) may be incident on the light path control element (140). The light incident on the light path control element (140) may be reflected in a direction in which the reflecting mirror (176) is disposed so that the light may be irradiated onto the target object (P). In another embodiment (not shown), the medical 3D imaging measuring device does not include the light path control element 140, and the light output from the light source 120 can be immediately incident in the direction of the reflector 176.

[0053] In one embodiment, the light is reflected by the reflector 176 and can be irradiated to the object P through an opening 162h of the second housing 162. In another embodiment (not shown), the medical 3D image measuring device does not include a reflector 176, and the light can be irradiated to the object P through an opening formed on the optical path LA without reflection from the reflector 176.

[0054] The medical 3D image measuring device (100) may be configured such that an optical path (LA) of light output from a light source (120) and irradiated to a target object (P) and an optical path (LB) of reflected light reflected from the target object (P) and reaching a camera (130) are coaxial. The optical paths (LA) and (LB) may be overlapped coaxially in a section between the optical path control element (140) and the target object (P).

[0055] The light irradiated to the object (P) may be reflected by the object (P). The reflected light reflected from the object (P) may again enter the second housing (162) through the opening (162h). ​​In one embodiment, the reflected light may be reflected by a reflector (176) and enter the light path control element (140). In another embodiment (not shown), the medical 3D image measurement device does not include a reflector (176), and the reflected light may directly enter the light path control element (140) without additional reflection.

[0056] The reflected light incident on the optical path control element 140 can pass through the optical path control element 140 and reach the camera 130. In another embodiment (not shown), the medical 3D image measurement device does not include the optical path control element 140, and the reflected light can directly enter the camera 130.

[0057] The medical 3D image measurement device (100) may include a camera (130) configured to receive the reflected light and generate image information. The camera (130) may include a condensing lens (137) through which the reflected light passes. The camera (130) may include a lens array (135) in which a plurality of microlenses are arranged through which the reflected light passes. The camera (130) may include an image sensor (131) that captures the reflected light. The reflected light may pass through the condensing lens (137) and the lens array (135) to reach the image sensor (131).

[0058] For example, the image sensor (131) can capture reflected light to generate a light field image of the object (P). The light field image of the object (P) can be an image of a pattern irradiated on the object (P). The processor (110) can generate a 3D image of the surface of the object (P) using the light field image. The processor (110) can transmit the 3D image of the surface of the object (P) to the external electronic device (20) via the communication circuit (150).

[0059] The medical three-dimensional image measuring device 100 may include a marker 180 having a surface for tracking. The marker 180 may be disposed in the housings 161, 162. The marker 180 may be disposed in the first housing 161.

[0060] The marker (180) is disposed in the housings (161, 162) such that at least one of the position relative to the opening (162h) and the attitude relative to the opening (162h) is changeable. Specifically, the marker (180) may be fixed to the first housing (161), and the second housing (162) may be coupled to the first housing (161) such that at least one of the position relative to the first housing (161) and the attitude relative to the first housing (161) is changeable. An opening (162h) may be formed in the second housing (162).

[0061] In one embodiment, the marker 180 may include a pattern surface (not shown) on which a pattern is formed as the tracking surface. The marker 180 may include a lens (not shown) configured such that at least a portion of a pattern that appears uniquely depending on the direction of viewing the marker 180 from the outside is identifiable from the outside of the marker 180. The lens of the marker 180 may be a ball lens. The pattern surface may have a concave curved shape.

[0062] The external electronic device 20 is configured to receive the image information generated by the camera 130 of the medical 3D imaging measurement device 100. The external electronic device 20 may include a communication circuit 27 for receiving the image information.

[0063] The external electronic device (20) may include an imaging device (23) for imaging at least a portion of the tracking surface of the marker (180) to form a tracking image. The external electronic device (20) may be configured to use the tracking image to determine a position and orientation of the marker (180) and to determine coordinates of the image information.

[0064] In one embodiment, the external electronic device (20) can determine the location (or coordinate) and posture (or orientation) of the medical 3D image measuring device (100) to which the marker (180) is attached based on the captured tracking image. The position of the medical 3D image measuring device (100) can be defined in spatial coordinates such as coordinates on the x, y, and z axes of a Cartesian coordinate system. The posture of the medical 3D image measuring device (100) can be defined in terms of roll, pitch, and yaw. The external electronic device (20) can track the position and posture of the medical 3D image measuring device (100) by capturing an image of the marker (180) attached to the medical 3D image measuring device (100) via the imaging device (23).

[0065] For example, the imaging device 23 of the external electronic device 20 may image a pattern image of at least a portion of the pattern that is visually distinguishable from the outside of the marker 180 through the ball lens of the marker 180. Once the pattern image of at least a portion of the pattern surface is acquired, the external electronic device 20 may process information extracted from the pattern image of at least a portion of the pattern surface to determine the position and orientation of the marker 180. Based on the position and orientation of the marker 180, the external electronic device 20 may determine the position and orientation of the medical 3D image measuring device 100 to which the marker 180 is attached. As a specific method for calculating the position and orientation of the marker 180 using an image of at least a portion of the pattern surface, one of the general optical tracking methods may be applied.

[0066] The medical 3D image measuring device (100) may be configured to change at least one of a relative position and a relative orientation of the marker (180) with respect to an opening (162h) through which the reflected light enters. The medical 3D image measuring device (100) may include a sensor (196) that senses displacement information resulting from a change in at least one of the relative position and the relative orientation of the marker (180) with respect to the opening. The communication circuit (150) may be configured to transmit the displacement information of the marker (180) to an external electronic device (20).

[0067] The external electronic device (20) can receive the displacement information of the marker (180). The external electronic device (20) can be configured to determine the coordinates of the image information generated by the medical 3D image measuring device (100) based on the displacement information. For example, the external electronic device (20) corrects the position or orientation of the medical 3D image measuring device (100) based on the displacement information, and information on the corrected position or orientation of the medical 3D image measuring device (100) is used for image alignment between the image information (image information generated by the medical 3D image measuring device) and a medical image (e.g., a CT image, an MRI image).

[0068] The image information may have a unique coordinate system (e.g., x1y1z1 coordinate system) relative to the medical 3D imaging measurement device (100). The coordinate system of the image information may be different from the coordinate system of the medical image (e.g., x2y2z2) and may be different from the coordinate system of the external electronic device (20) (e.g., x0y0z0).

[0069] The medical image registration system (10) can convert the coordinate system (e.g., x2y2z2) of the medical image and the coordinate system (e.g., x1y1z1) of the image information into the coordinate system (e.g., x0y0z0) of the external electronic device (20). The external electronic device (20) can perform registration between the medical image and the image information, which have different coordinate systems. In order to perform registration between the medical image and the image information, the external electronic device (20) can extract a surface image from the medical image and perform registration between the extracted surface image and the received image information. Here, the surface image extracted from the medical image may be in the same coordinate system (e.g., x2y2z2) as the medical image. In addition, the external electronic device (20) can convert the coordinate system (e.g., x1y1z1) of the image information into the coordinate system (e.g., x0y0z0) of the external electronic device (20) using a marker (180) attached to the medical 3D image measurement device (100) as an intermediary. The medical image and the surface image extracted from the medical image can also be transformed into the coordinate system (e.g., x0y0z0) of the external electronic device 20. The external electronic device 20 can perform the alignment between the image information and the medical image using any one of a variety of image alignment algorithms. For example, the external electronic device 20 can perform the alignment using an ICP (interactive closest point) algorithm.

[0070] The second housing (162) may be configured to be movable relative to the first housing (161) so that its position and / or attitude relative to the first housing (161) can be changed. The relative movement here means that the second housing (162) moves relative to the first housing (161) and that the first housing (161) moves relative to the second housing (161). Through this, any one of the relative position and the relative attitude of the marker (180) fixed to the first housing (161) with respect to the opening (162h) can be changed.

[0071] The marker (180) is configured to be able to change at least one of a relative position and a relative orientation with respect to the opening (162h) by performing at least one of (i) a relative translational motion with respect to the opening (162h) and (ii) a relative rotational motion around a predetermined rotational axis with respect to the opening (162h). ​​The second housing (162) is configured to be able to change at least one of a relative position and a relative orientation of the marker (180) with respect to the opening (162h) by performing at least one of (i) a relative translational motion with respect to the first housing (161) and (ii) a relative rotational motion around a predetermined rotational axis with respect to the first housing (161). The "rotational axis" used in this disclosure is a virtual axis and does not refer to an actual part of the device. In this disclosure, "component A performs a translational motion with respect to component B" means that component A performs a translational motion with respect to component B and component B performs a translational motion with respect to component A. In addition, in the present disclosure, "structure C performs relative rotational movement with respect to structure D around a predetermined rotational axis" encompasses structure C performing rotational movement with respect to structure D around a predetermined rotational axis and structure D performing rotational movement with respect to structure C around a predetermined rotational axis.

[0072] As an example, the marker 180 may be configured to be able to perform a translational movement relative to the opening 162h. Specifically, the marker 180 may perform a translational movement relative to the opening 162h, or the opening 162h may perform a translational movement relative to the marker 180. Here, the second housing 162 may be configured to be able to perform a translational movement relative to the first housing 161. This allows the distance between the marker 180 and the opening 162h to be changed. For example, in the third, fourth, and sixth embodiments described below with reference to Figs. 7, 8, and 10, the marker 180 may perform a translational movement relative to the opening 162h.

[0073] As another example, the marker 180 may rotate relative to the opening 162h around a predetermined rotation axis. Specifically, the marker 180 may rotate relative to the opening 162h around a predetermined rotation axis, or the opening 162h may rotate relative to the marker 180 around a predetermined rotation axis. Here, the second housing 162 may be configured to be able to rotate relative to the first housing 161 around a predetermined rotation axis. This allows the distance and / or the attitude of the marker 180 with respect to the opening 162h to be changed. For example, in the first, second, fifth and sixth embodiments described below with reference to FIGS. 4, 5, 6, 9 and 10, the marker 180 may rotate relative to the opening 162h around a predetermined rotation axis.

[0074] In one embodiment with reference to Fig. 3, the second housing 162 may be rotatably coupled to the first housing 161. The medical 3D image measuring device 100 may include a bearing 191 disposed between the first housing 161 and the second housing 162 such that the second housing 162 may rotate relative to the first housing 161. The second housing 162 may rotate relative to the first housing 161 about the central axis of the bearing 191.

[0075] The sensor 196 can sense displacement information due to a change in at least one of the relative position and the relative attitude of the first housing 161 with respect to the second housing 162. The sensor 196 can be any one of various sensors, or can be embodied with two or more types of sensors.

[0076] In one embodiment, the sensor 196 can sense rotation angle information of the second housing 162 relative to the first housing 161. That is, when the second housing 162 rotates relative to the first housing 161 or the first housing 161 rotates relative to the second housing 162, the sensor 196 can sense the rotation angle information. For example, the sensor 196 can be a gyro sensor or an encoder. Such a sensor can be applied to the first, second, fifth and sixth embodiments described below with reference to Figures 4, 5, 6, 9 and 10.

[0077] In another embodiment, the sensor 196 may sense distance information of the relative movement of the second housing 162 with respect to the first housing 161. That is, when the second housing 162 moves with respect to the first housing 161 or the first housing 161 moves with respect to the second housing 162, the sensor 196 may sense the distance information. For example, the sensor 196 may be an infrared sensor, a 3D sensor, an ultrasonic sensor, an RF sensor, a geomagnetic sensor, an encoder, or the like. Such a sensor may be applied to the third, fourth, and sixth embodiments described below with reference to FIGS. 7, 8, and 10.

[0078] The processor (110) can transmit the displacement information to the external electronic device (20) via the communication circuit (150). In one embodiment, the processor (110) can transmit the rotation angle information to the external electronic device (20) via the communication circuit (150). In another embodiment, the processor (110) can transmit the distance information to the external electronic device (20) via the communication circuit (150).

[0079] FIG 4 is a perspective view of a medical 3D image measuring device (101) according to a first embodiment of the present disclosure. FIG 5 is a cross-sectional view of the medical 3D image measuring device (101) of FIG 4 taken along line S1-S1'. The medical 3D image measuring device (101) according to the first embodiment will be described below, focusing on the differences from the above-mentioned embodiments of the medical 3D image measuring device (100). The drawing shows the direction (PD) in which light is emitted through the opening (162h) and the direction (MD) in which the marker (180) faces forward.

[0080] 4 and 5, the second housing (162) of the medical 3D image measuring device (101) is disposed to be capable of relative rotational movement about a predetermined rotation axis (X1) with respect to the first housing (161), thereby being configured to be able to change its relative attitude with respect to the first housing (161). The direction of rotation (C1) of the rotational movement is shown in the drawings.

[0081] The rotation axis (X1) may extend parallel to the path (LB) of the reflected light just before it reaches the camera (130) (see FIG. 3). For example, the rotation axis (X1) may coincide with the optical path (LB). Through this, the position of the optical path (LB) can be maintained even if the first housing (161) and the second housing (162) rotate relative to each other. A sensor (not shown) of the medical 3D image measuring device (101) may be configured to sense rotation angle information resulting from a change in the relative attitude of the second housing (162).

[0082] The medical 3D image measuring device (101) may include a first communication line (151) configured to transmit the rotation angle information to an external electronic device (20). The medical 3D image measuring device (101) may include a second communication line (152) configured to transmit the image information generated by the camera (130) to the external electronic device (20). The medical 3D image measuring device (101) may include a third communication line (153) configured to transmit trigger information for syncing the image information generated by the camera (130) and an image captured by an imaging device (23) of the external electronic device (20) to the external electronic device (20).

[0083] The medical 3D image measuring device (101) may be configured so that the rotational movement is performed automatically. For example, the medical 3D image measuring device (101) may include a motor (not shown) that generates a driving force, and gears (197a, 162a) to which the driving force is transmitted. A driven gear (162a) may be formed in the second housing (162). For example, the driven gear (162a) may be a ring gear formed on the inner wall surface of the second housing (162) along the rotation direction (C1). The driving gear (197a) may rotate in mesh with the driven gear (162a) and transmit the driving force to the driven gear (162a).

[0084] 6 is an elevation view of a medical 3D image measurement device (102) according to a second embodiment of the present disclosure. The medical 3D image measurement device (102) according to the second embodiment will be described below, focusing on the differences from the medical 3D image measurement device (101) according to the first embodiment described above.

[0085] 6, the medical 3D image measuring device 102 may be configured such that the rotational movement is manually performed by a user. The medical 3D image measuring device 102 includes a bearing 191 disposed between the first housing 161 and the second housing 162 such that the second housing 162 can rotate relative to the first housing 161. For example, the bearing 191 may be a ball bearing.

[0086] The sensor (196) of the medical 3D image measuring device (101) may be configured to sense rotation angle information according to the change in the relative attitude of the second housing (162). For example, the sensor (196) may include an encoder. The encoder may include a target portion (196a) formed on one of the first housing (161) and the second housing, and a sensing portion (196b) formed on the other. The target portion (196a) may extend along a rotation direction (C1) centered on a rotation axis (X1). The sensing portion (196b) may be disposed to face a specific position of the target portion (196a), and the specific position may be configured to change according to the rotational movement. The rotation angle information may be sensed through information about the specific position of the target portion (196a) sensed by the sensing portion (196b).

[0087] 7 is an elevation view of a medical 3D image measurement device (103) according to a third embodiment of the present disclosure. The medical 3D image measurement device (103) according to the third embodiment will be described below, focusing on the differences from the medical 3D image measurement devices (101, 102) according to the first and second embodiments described above.

[0088] 7, the second housing 162 of the medical 3D image measuring device 103 is arranged to be capable of translational movement relative to the first housing 161, and is configured to be able to change its position relative to the first housing 161. The direction of movement L1 of the translational movement is shown in the drawing.

[0089] The medical 3D image measuring device (103) includes a sensor (196) configured to detect distance information of the second housing (162) moving relative to the first housing (161) due to the relative position change. For example, the sensor (196) may be an infrared sensor. The sensor (196) may include a light emitting unit (196g) that emits infrared light (R1) and a light receiving unit (196h) that detects the infrared light (R1) and generates the distance information. The first communication line (151) may be configured to transmit the distance information to an external electronic device (20).

[0090] The medical 3D image measuring device (103) may be configured so that the translational movement is performed automatically. For example, the medical 3D image measuring device (103) may include a motor (not shown) that generates a driving force, and gears (197c, 162c) to which the driving force is transmitted. A driven gear (162c) may be formed in the second housing (162). For example, the driven gear (162c) may be a rack formed on the inner wall surface of the second housing (162) along the moving direction (L1), and the driving gear (197c) may be a pinion. The driving gear (197c) may rotate in mesh with the driven gear (162c) and transmit the driving force to the driven gear (162a).

[0091] 8 is an elevation view of a medical 3D image measuring device (104) according to a fourth embodiment of the present disclosure. The medical 3D image measuring device (104) according to the fourth embodiment will be described below, focusing on the differences from the medical 3D image measuring device (103) according to the third embodiment described above.

[0092] 8, the medical 3D image measuring device (104) may be configured so that the translational movement is manually performed by a user. The medical 3D image measuring device (104) includes a slider (192) formed on one of the first housing (161) and the second housing (162), and a guide (193) formed on the other. The guide (193) is formed to extend along the moving direction (L1). The slider (192) can slide along the guide (193) to move in the moving direction (L1).

[0093] The sensor (196) of the medical 3D image measuring device (104) may include a sensor (196) configured to sense distance information of a movement of the second housing (162) relative to the first housing (161) due to the relative position change. For example, the sensor (196) may include a linear encoder. The linear encoder may include a target portion (196c) formed on one of the first housing (161) and the second housing, and a sensing portion (196d) formed on the other. The target portion (196c) may extend along the moving direction (C1). The sensing portion (196d) may be disposed to face a specific position of the target portion (196c), and the specific position may be configured to change due to the translational motion. The distance information may be sensed through information on the specific position of the target portion (196c) sensed by the sensing portion (196d).

[0094] 9 is an elevation view of a medical three-dimensional image measurement device (105) according to a fifth embodiment of the present disclosure. The medical three-dimensional image measurement device (105) according to the fifth embodiment will be described below, focusing on the differences from the medical three-dimensional image measurement devices (101, 102, 103, 104) according to the first to fourth embodiments described above.

[0095] 9, the second housing (162) of the medical 3D image measuring device (105) is arranged to be capable of relative rotational movement with respect to the first housing (161) around a predetermined rotation axis (X2), thereby being configured to be able to change its relative attitude with respect to the first housing (161). The direction of rotation (C2) of the rotational movement is shown in the drawing. The medical 3D image measuring device (105) includes a hinge (194) that allows the first housing (161) and the second housing (162) to rotate relative to each other. The hinge (194) may be arranged on the rotation axis (X2).

[0096] The rotation axis (X2) may extend in a direction intersecting the path (LB) of the reflected light just before it reaches the camera (130). In the medical three-dimensional image measurement device (105), the light source (120), the camera (130), and the light path control element (140) may be disposed inside the second housing (162).

[0097] The medical 3D image measuring device (105) may be configured so that the rotational movement is performed automatically. For example, the medical 3D image measuring device (105) may include a motor (not shown) that generates a driving force, and gears (197e, 162e) to which the driving force is transmitted. A driven gear (162e) may be formed in the second housing (162). For example, the driven gear (162e) may be formed on the periphery of the hinge (194). The driving gear (197e) may rotate in mesh with the driven gear (162e) and transmit the driving force to the driven gear (162e).

[0098] 10 is an elevation view of a medical three-dimensional image measurement device (106) according to a sixth embodiment of the present disclosure. The medical three-dimensional image measurement device (106) according to the sixth embodiment will be described below, focusing on the differences from the medical three-dimensional image measurement devices (101, 102, 103, 104, 105) according to the first to fifth embodiments described above.

[0099] 10, the second housing (162) of the medical 3D image measuring device (106) is arranged to be capable of relative rotational movement about a rotation axis (X1) with respect to the first housing (161) and relative translational movement with respect to the first housing (161), thereby being configured to be able to change its position and posture relative to the first housing (161). The drawing shows the rotational direction (C1) of the rotational movement and the moving direction (L1) of the translational movement.

[0100] The medical 3D image measuring device (106) may be configured such that the rotational movement and the translational movement are manually performed by a user. The medical 3D image measuring device (106) includes a bearing (191) disposed between the first housing (161) and the second housing (162) so that the second housing (162) can perform the relative rotational movement and the relative translational movement with respect to the first housing (161). For example, the bearing (191) may be a ball bearing. The bearing (191) may be disposed on an inner wall surface of the second housing (162).

[0101] The sensor (196) of the medical 3D image measuring device (106) may be configured to sense distance information of a relative movement of the second housing (162) with respect to the first housing due to the relative position change, and to sense rotation angle information due to the relative attitude change of the second housing (162). For example, the sensor (196) may be an infrared sensor. The sensor (196) may include a light emitting unit (196e) that emits infrared light (R1) and a light receiving unit (196f) that senses the infrared light (R1) and generates the distance information. A plurality of light receiving units (196f) are arranged along a rotation direction (C1), and the rotation angle information may be generated depending on which light receiving unit (196f) senses the infrared light (R1) among the plurality of light receiving units (196f).

[0102] Although the technical idea of ​​the present disclosure has been described above by way of some embodiments and examples shown in the accompanying drawings, it should be understood that various substitutions, modifications and changes can be made within the scope of the technical idea and scope of the present disclosure that can be understood by a person having ordinary skill in the art to which the present disclosure pertains. In addition, such substitutions, modifications and changes should be considered to fall within the scope of the appended claims.

Claims

[Claim 1] A light source that outputs light; A camera configured to receive reflected light from the light reflected on a target object and generate three-dimensional image information; a housing in which the camera is disposed and which defines an opening through which the reflected light enters; a marker disposed on the housing such that at least one of a relative position and a relative orientation with respect to the opening can be changed, the marker having a tracking surface configured to be imaged by an external imaging device for tracking the position and orientation; Medical 3D imaging measurement device.

Images