Medical three-dimensional image measuring device and medical image integration system

Abstract

A medical three-dimensional image measuring apparatus of the disclosed embodiment includes a light source that outputs light, a camera that receives reflected light formed by the light being reflected by an object body to generate three-dimensional image information, a housing that is internally configured with the camera and formed with an opening through which the reflected light flows into the interior, and a marker that is configured to the housing in a manner capable of changing at least one of a relative position and a relative posture with respect to the opening and has a tracking surface that is photographed by an external imaging device so as to track the position and the posture.

Description

Technical Field

[0001] This disclosure relates to a medical three-dimensional imaging measurement device and a medical imaging integration system. Background Technology

[0002] Recently, surgical navigation technology has been applied to support surgeons in performing procedures. Markers are placed on medical 3D imaging devices that capture images of surgical instruments and the surgical site on the patient. An optical tracking system with an imaging device such as a camera tracks these markers, allowing the position and orientation of the surgical instruments to be tracked. This enables the integration of surgical site images captured by the medical 3D imaging device with previously captured patient medical images (e.g., CT images, MRI images). In this way, the position and orientation information of surgical instruments on the patient's medical images can be identified within the system.

[0003] Using images of markers obtained through an optical tracking system, positional and orientation information of surgical instruments or medical 3D imaging measurement devices equipped with markers can be acquired. For example, positional information can be defined using spatial coordinates such as the X, Y, and Z axes of a Cartesian coordinate system, while orientation information can be defined using roll, pitch, and yaw angles.

[0004] Medical 3D imaging measurement devices can measure 3D images of a patient's surgical site to obtain and process image information of the surgical site. For example, the method currently being used in medical 3D imaging measurement devices is to illuminate the surgical site with a predetermined pattern of light and measure the resulting pattern, thereby obtaining a 3D image of the object. Summary of the Invention

[0005] Technical issues

[0006] When a medical 3D imaging measurement device changes position and / or posture to capture surgical site images, a limitation exists: the position and / or posture of the medical 3D imaging measurement device cannot be changed to a position and / or posture that the optical tracking system cannot track the markers on the medical 3D imaging measurement device. In the prior art, due to various surgical postures of the patient (e.g., Parkbench posture, Prone posture, Supine posture), interference from the patient or other objects (e.g., bed, surgical instruments, etc.) can obstruct the field of view of the optical tracking system, making it difficult to capture the markers. The embodiments of this disclosure solve the problems existing in the prior art described above.

[0007] Technical solution

[0008] This disclosure provides one aspect of an embodiment of a medical three-dimensional image measuring device. A representative embodiment of the medical three-dimensional image measuring device includes: a light source that outputs light; a camera that receives reflected light from an object to generate three-dimensional image information; a housing in which the camera is disposed internally and has an opening for the reflected light to flow into the housing; and a marker disposed on the housing in a manner capable of changing at least one of a relative position and a relative pose relative to the opening, and having a tracking surface that is captured by an external imaging device to track position and pose.

[0009] This disclosure provides, in another aspect, embodiments of a medical image integration system. A representative embodiment of the medical image integration system includes: a medical three-dimensional image measuring device, the medical three-dimensional image measuring device including a light source, a camera, a housing, and a marker, wherein the light source outputs light, the camera receives reflected light formed by the light being reflected by an object to generate three-dimensional image information, the housing has the camera disposed internally and has an opening for the reflected light to flow into the housing, the marker is disposed in the housing in a manner capable of changing at least one of its relative position and relative posture relative to the opening, and has a tracking surface, the tracking surface being captured by an external imaging device to track position and posture; and an external electronic device, the external electronic device including an imaging device that captures at least a portion of the tracking surface of the marker to image a tracking image, and the external electronic device receiving the three-dimensional image information and using the tracking image to determine the position and posture of the marker to determine the coordinates of the three-dimensional image information.

[0010] Invention Effects

[0011] According to embodiments of this disclosure, the tracking of the marker by an external electronic device can be achieved by changing the position and / or posture of the marker so that the field of view of the marker by the imaging device is not obstructed by the patient fixation device, surgical instruments, the medical three-dimensional imaging measurement device itself and / or the operator.

[0012] According to embodiments of this disclosure, the operator can reduce the cumbersome process of ensuring the optical tracking system's field of vision over the marker is unobstructed, thus improving convenience and providing conditions that allow the operator to focus more on surgical site imaging and surgical instrument operation. For example, even under difficult surgical conditions where the patient is in a prone position (Prone position) and the surgical site is facing downwards, the operator can easily ensure the optical tracking system's field of vision over the marker by changing the position and / or orientation of the marker on the medical 3D imaging measurement device.

[0013] According to embodiments of this disclosure, the position and / or orientation of the marker can be changed so that the marker of the medical three-dimensional imaging measurement device can be tracked continuously. Attached Figure Description

[0014] Figure 1 This is a block diagram illustrating a medical image integration system 10 according to an embodiment of the present disclosure.

[0015] Figure 2 This is a diagram illustrating the usage configuration of a medical image integration system 10 according to an embodiment of the present disclosure.

[0016] Figure 3 This is a cross-sectional view of a medical three-dimensional imaging measurement device 100 according to an embodiment of the present disclosure.

[0017] Figure 4 This is a perspective view of the medical three-dimensional imaging measurement device 101 according to the first embodiment of this disclosure.

[0018] Figure 5 It is Figure 4 A cross-sectional view of the medical three-dimensional imaging measurement device 101 along the S1-S1' line.

[0019] Figure 6 This is an elevation view of the medical three-dimensional image measuring device 102 according to the second embodiment of this disclosure.

[0020] Figure 7 This is an elevation view of a medical three-dimensional imaging measurement device 103 according to a third embodiment of this disclosure.

[0021] Figure 8 This is an elevation view of a medical three-dimensional imaging measuring device 104 according to the fourth embodiment of this disclosure.

[0022] Figure 9 This is an elevation view of a medical three-dimensional imaging measuring device 105 according to the fifth embodiment of this disclosure.

[0023] Figure 10 This is an elevation view of a medical three-dimensional imaging measuring device 106 according to the sixth embodiment of this disclosure. Detailed Implementation

[0024] The embodiments of the present invention are provided by way of example for the purpose of illustrating the technical concept of the invention. The scope of this disclosure is not limited to the embodiments described below or the specific description of these embodiments.

[0025] Unless otherwise defined, all technical and scientific terms used in this disclosure have the meaning commonly understood by one of ordinary skill in the art to which this disclosure pertains. All terms used in this disclosure have been chosen for the purpose of more clearly describing this disclosure and are not intended to limit the scope of this disclosure.

[0026] The expressions such as “including,” “possessing,” and “having” used in this invention, unless otherwise mentioned in the statements or texts containing the corresponding expressions, should be understood as open-ended terms that may include other embodiments.

[0027] Unless otherwise stated, the singular expressions described in this disclosure may include the meaning of the plural expressions, and this also applies to the singular expressions described in the claims.

[0028] The terms "first" and "second" used in this invention are used to distinguish multiple constituent elements from each other, and are not intended to limit the order or importance of the corresponding constituent elements.

[0029] In this disclosure, when a constituent element is referred to as being "connected to" or "continuing on" another constituent element, it should be understood that the constituent element may be directly connected to or continued on the other constituent element, or it may be connected or continued on the basis of a new constituent element.

[0030] Embodiments of the present disclosure are described below with reference to the accompanying drawings. In the drawings, the same or corresponding constituent elements are given the same reference numerals. Furthermore, in the following description of the embodiments, repeated descriptions of the same or corresponding constituent elements may be omitted. However, even if a description of a constituent element is omitted, it does not mean that such a constituent element is not included in a particular embodiment.

[0031] Figure 1 This is a block diagram illustrating a medical image integration system 10 according to an embodiment of the present disclosure. Figure 2 This is a diagram illustrating the usage configuration of a medical image integration system 10 according to an embodiment of the present disclosure. Figure 3 This is a cross-sectional view of a medical three-dimensional imaging measurement device 100 according to an embodiment of the present disclosure.

[0032] Reference Figures 1 to 3 The medical imaging integration system 10 may include a medical three-dimensional image measurement device 100 and an external electronic device 20. The medical three-dimensional image measurement device 100 and the external electronic device 20 can communicate with each other via wired or wireless means to transmit and receive various types of data (e.g., images). Even Figure 1 The omission or substitution of a portion of the configuration shown will not hinder the implementation of the various embodiments disclosed in this document.

[0033] The medical three-dimensional imaging measurement device 100 may include a processor 110. The processor 110 can perform calculations or data processing related to the control and / or communication of the various components of the medical three-dimensional imaging measurement device 100. The processor 110 can process signals received from the components of the medical three-dimensional imaging measurement device 100. The processor 110 can control the medical three-dimensional imaging measurement device 100 to send signals to an external electronic device 20. The processor 110 can load instructions or data received from other components of the medical three-dimensional imaging measurement device 100 into a memory (not shown), process the instructions or data stored in the memory, and store the result data.

[0034] The medical three-dimensional imaging measurement device 100 may include a light source 120. The light source 120 may output patterned light. The patterned light output by the light source 120 may illuminate an object (e.g., a patient) P. To measure a three-dimensional image of the object P, the patterned light may be light with a specific pattern or light with a predetermined or specific periodic pattern. The patterned light may, for example, include patterned light with random dots, patterned light with a grid pattern, patterned light with sine wave patterned stripe brightness, patterned light with an on-off pattern of alternating bright and dark areas, or triangular wave patterned light with brightness variations in a triangular waveform; however, the form of the patterned light is not limited to these.

[0035] The light source 120 may include a patterned portion having multiple patterns and an LED that illuminates the patterned portion. The light source 120 may include a condensing lens 125 that focuses the light emitted from the LED 121 onto the patterned portion. The light emitted from the LED 121 can pass through the patterned portion 123 to reflect the patterns. The LED 121 may emit ultraviolet light, for example, but is not limited thereto.

[0036] The medical three-dimensional image measurement device 100 may include a camera 130. The camera 130 may be configured to capture images of an object P. The camera 130 can capture images of the object P to obtain three-dimensional image data of the object P, and a three-dimensional image of the object P can be obtained by processing the obtained image data.

[0037] For example, camera 130 can obtain an image of object P by photographing it illuminated with patterned light. Processor 110 can generate a three-dimensional image of object P based on a phase-shifting method using patterned light. For example, when patterned light of a predetermined shape is illuminated onto object P by light source 120, the intensity of the light displayed on the surface will vary depending on the curvature of the surface of object P. Processor 110 can generate phase data from the image generated by camera 130, and then generate a three-dimensional image.

[0038] In one embodiment, camera 130 may be a light field camera 130 that generates a light field image. The light field camera 130 can determine the depth of object P after it has been photographed, and combine images with different object P depths. The image sensor of the light field camera 130 may have a post-hoc and variable object P depth. Camera 130 can generate a light field image reflecting a pattern on object P. Processor 110 can generate phase data from the light field image and calculate the height of points constituting the surface of object P, thereby generating a three-dimensional image of the surface of object P.

[0039] Camera 130 may include a condenser lens 137, a lens array 135, and an image sensor 131. The condenser lens 137 can focus light entering from the object 410. The lens array 135 may be a lens composed of multiple microlenses 431 arranged together. The image sensor 131 can capture light passing through the lens array 135 and use the captured light to generate a light field image. The image sensor 131 may be divided into regions corresponding to each of the multiple microlenses. For example, the image sensor 131 may include a CCD (charge-coupled device) sensor or a CMOS (complementary metal-oxide semiconductor) sensor.

[0040] According to one embodiment, the light field image generated by the camera 130 may include multiple sub-images that store color and direction information of the light together. For example, when patterned light illuminates an object P and the reflected light reflected from the object P is received by the camera 130, the light field image may be an image composed of multiple sub-images including color and direction information of the reflected light. The camera 130 may use the multiple sub-images included in the light field image to perform a refocusing process. For example, during the refocusing process, the camera 130 may combine the desired depth of the object P in the pixels of the light field image with the information of the pixels corresponding to the light path and direction calculated inversely thereafter to generate an image with the desired depth. For example, the camera 130 may also generate a focused image for all areas of the object P during the refocusing process. In order to generate an accurate image of the object area in the camera 130, it is necessary to adjust the distance between the medical three-dimensional imaging measurement device 100 and the object area of ​​the object P. When using the camera 130 that generates the light field image, the depth of the object P can be determined afterward, and a focused light field image can be generated for all areas of the object P, so it is not necessary to adjust the focal length in advance. When using a camera 130 that generates light field images, the depth range that can be measured is wider than that of a camera 130 that uses a regular lens, and an image of the object P can be obtained in a single shot.

[0041] The medical 3D imaging measurement device 100 may include a light path control element 140. The light path control element 140 can cause patterned light to be reflected in a specific direction so that the patterned light output from the light source 120 illuminates the object P. The light path control element 140 can also cause reflected light from the object P to pass through so that the reflected light reaches the camera 130. The light path control element 140 may be, for example, a semi-transparent mirror. As an example, the light source 120 and the camera 130 may be configured perpendicular to each other with respect to the light path control element 140.

[0042] The medical three-dimensional imaging measurement device 100 may include housings 161 and 162 forming the exterior. Housings 161 and 162 may include a first housing 161 and a second housing 162 coupled together. The second housing 162 may be movably coupled to the first housing 161. Markings 180 may be disposed on the first housing 161.

[0043] A camera 130 is disposed inside the housings 161 and 162. An opening 162h is formed on the housings 161 and 162 to allow the reflected light to flow into the interior. A lens (not shown) of a light-transmitting material can be disposed in the opening 162h. A light source 120 can be disposed inside the housings 161 and 162. A light path control element 140 can be disposed inside the housings 161 and 162.

[0044] exist Figure 3 In the disclosed embodiments, a light source 120, a camera 130, and a light path control element 140 are disposed inside the first housing 161, and an opening 162h is formed on the second housing 162 so that patterned light output from the light source 120 illuminates the object P, but this disclosure is not limited thereto. In another embodiment, the light source 120, the camera 130, and the light path control element 140 may also be disposed in the second housing 162 and an opening 162h may be formed therein.

[0045] In one embodiment, a user can use the medical three-dimensional imaging measurement device 100 by gripping the first housing 161 or the second housing 162. The first housing 161 or the second housing 162 may include structures (e.g., handles) that facilitate the user's movement, handling, and use of the medical three-dimensional imaging measurement device 100.

[0046] In another embodiment, the first housing 161 or the second housing 162 may be supported by other external devices (e.g., a frame (not shown) fixed to the operating table 30 or the ground). The frame may operate in a manner that allows for changes in the position and orientation of the medical three-dimensional imaging measurement device 100.

[0047] The medical 3D image measurement device 100 can transmit information to an external electronic device 20 wirelessly or via a wired connection. The medical 3D image measurement device 100 may include a communication circuit 150. The communication circuit 150 can transmit information to the external electronic device 20. The communication circuit 150 can establish a communication channel with the external electronic device 20 and send and receive various types of data with it. According to one embodiment, the communication circuit 150 may include a cellular communication module and be connected to a cellular network (e.g., 3G, LTE, 5G, Wibro, or WiMAX). According to another embodiment, the communication circuit 150 may include a near-field communication module and use near-field communication (e.g., Wi-Fi, Bluetooth, Bluetooth Low Energy (BLE), UWB) to send and receive data with the external electronic device 20. The medical 3D image measurement device 100 may also include wired communication circuits 151, 152, and 153 (see reference) for sending and receiving information with the external electronic device 20. Figures 4 to 10 ).

[0048] In one embodiment, processor 110 can generate a three-dimensional image of the surface of object P using an optical field image of object P obtained by camera 130. For example, the intensity of the patterned light illuminating the actual object area surface will vary depending on the curvature of the object P's surface. Processor 110 can use the optical field image of object P to measure the varying light intensity based on the curvature of the object P's surface and thereby generate phase data to calculate the height of each point constituting the surface. By calculating the height of each point constituting the surface of object P, processor 110 can generate a three-dimensional image of the object P's surface. Processor 110 can transmit the three-dimensional image of the object P's surface to external electronic device 20 via communication circuit 150.

[0049] External electronic device 20 may include controller 21. Controller 21 can perform calculations or data processing related to the control and / or communication of the various components of external electronic device 20. Controller 21 can process signals received from the components of external electronic device 20. Processor 110 can process signals received from medical three-dimensional imaging measurement device 100 to transmit them. Controller 21 can load received instructions or data into memory (not shown), process the instructions or data stored in memory, and store the result data.

[0050] The external electronic device 20 may include an imaging device 23. The imaging device 23 can capture images of at least a portion of the tracking surface of the marker 180 attached to the medical three-dimensional imaging measurement device 100, thereby enabling the imaging of a tracking image of at least a portion of the tracking surface. For example, the tracking surface may be a patterned surface, in which case the tracking image may be a patterned image. The imaging device 23 may, for example, include at least two or more cameras 23a, 23b capable of imaging at least a portion of the marker. The external electronic device 20 can use the imaged tracking image to determine the position and / or orientation of the marker 180.

[0051] In an embodiment where the marker 180 has a patterned surface as the tracking surface, when the external electronic device 20 obtains a patterned image of the marker 180, it can extract at least one of the sub-patterns from the patterned image as a basic unit constituting the pattern of the marker 180. The position of the extracted at least one sub-pattern within the entire pattern can be determined, and the orientation of the marker (10 and / or 180) can be determined based on the determined position of the sub-pattern within the entire pattern. The orientation of the marker 180 can refer to the relative three-dimensional orientation and orientation of the marker 180 relative to the imaging device 23. For example, the position of the marker 180 or the medical three-dimensional imaging measurement device 100 can be determined using triangulation based on two images with a stereoscopic relationship captured by the imaging device 23, which includes at least two cameras 23a and 23b. As described above, after the position and orientation of the marker 180 are determined, the position and orientation of the medical three-dimensional imaging measurement device 100 to which the marker 180 is attached can be determined based on the geometric relationship between the marker 180 and the medical three-dimensional imaging measurement device 100 to which the marker 180 is attached.

[0052] External electronic device 20 may include memory 25. Memory 25 may store various types of data used by at least one component of external electronic device 20 (e.g., controller 21). For example, memory 25 may, under the control of controller 21, store three-dimensional images of the surface of object P received from medical three-dimensional imaging measurement device 100. For example, memory 25 may, under the control of controller 21, store medical images (e.g., CT images, MRI images, etc.) received from medical device (not shown).

[0053] External electronic device 20 can transmit and receive information with medical three-dimensional imaging measurement device 100 via wireless or wired means. External electronic device 20 may include communication circuit 27. Communication circuit 27 of external electronic device 20 can establish a communication channel with medical three-dimensional imaging measurement device 100 and transmit and receive information with it. According to one embodiment, communication circuit 27 of external electronic device 20 may include a cellular communication module and be connected to a cellular network (e.g., 3G, LTE, 5G, Wibro, or WiMAX). According to another embodiment, communication circuit 27 of external electronic device 20 includes a near-field communication module and uses near-field communication (e.g., Wi-Fi, Bluetooth, Bluetooth Low Energy (BLE), UWB) to transmit and receive data with medical three-dimensional imaging measurement device 100. External electronic device 20 may also include a wired communication circuit (not shown) for transmitting and receiving information with medical three-dimensional imaging measurement device 100.

[0054] The controller 21 can perform image integration between the image of the surface of the object P received from the medical three-dimensional imaging measurement device 100 and the medical image of the object P. The image of the surface of the object P generated by the medical three-dimensional imaging measurement device 100 can be the external surface of the target included in the preceding medical image or a portion thereof. For example, when the medical image is an image modeling the three-dimensional shape of the head of the object P, the three-dimensional image of the surface of the object P can be an image measuring the external shape of the eyes, nose, mouth, ears, etc., located on the surface of the head of the object P.

[0055] In one embodiment, the three-dimensional image of the surface of the object P may have an inherent coordinate system (e.g., x1y1z1 coordinate system) with respect to the medical three-dimensional imaging measurement device 100. The coordinate system of the three-dimensional image of the surface of the object P may be different from the coordinate system of the medical image (e.g., x2y2z2) and may be different from the coordinate system of the external electronic device 20 (e.g., x0y0z0). The coordinate system of the external electronic device 20 may, for example, refer to the coordinate system of the imaging device 23 of the external electronic device 20.

[0056] Reference Figure 2 A user (e.g., a doctor) D can use the medical 3D imaging measurement device 100 to obtain a 3D image of the surface of an object P. For example, the user D can use the medical 3D imaging measurement device 100 to illuminate a patterned light onto the surface of the object P. A pattern PA can be formed on the surface of the object P by means of the illuminated patterned light.

[0057] In one embodiment, the medical three-dimensional imaging measurement device 100 can receive reflected light from an object P to generate an optical field image of the object P. The optical field image of the object P can, for example, be an image composed of multiple sub-images relating to an irradiated pattern PA. The medical three-dimensional imaging measurement device 100 can use the optical field image of the object P to generate a three-dimensional image of the surface of the object P. The medical three-dimensional imaging measurement device 100 can transmit the generated three-dimensional image of the surface of the object P to an external electronic device 20.

[0058] External electronic device 20 captures at least a portion of the tracking surface of the marker 180 attached to the medical three-dimensional imaging measurement device 100 via an imaging device to image the tracking surface of at least a portion of the tracking surface. External electronic device 20 can determine the position and orientation of the medical three-dimensional imaging measurement device 100 with the marker 180 attached based on the imaged tracking image.

[0059] External electronic device 20 can transform the coordinate system of a three-dimensional image of the surface of object P into the coordinate system of external electronic device 20. For example, external electronic device 20 can transform the coordinate system of a three-dimensional image of the surface of object P into the coordinate system of external electronic device 20 based on the position and orientation of the medical three-dimensional imaging measuring device 100 determined by marker 180.

[0060] External electronic device 20 can transform the coordinate system of the medical image of object P received from the medical device into the coordinate system of external electronic device 20. In various embodiments, external electronic device 20 can unify the coordinate systems between a three-dimensional image of the surface of object P and the medical image of object P, thereby performing image integration.

[0061] Reference Figure 3 The medical three-dimensional imaging measurement device 100 may include at least one focusing lens 171, 172 for focusing light. The focusing lenses 171, 172 may be disposed in the light path. The focusing lenses 171, 172 may be disposed around the light path control element 140. At least one focusing lens 171, 172 may include a first focusing lens 171 disposed in the light path from the light source 120 toward the light path control element 140. At least one focusing lens 171, 172 may include a second focusing lens 172 disposed in the light path from the light path control element 140 toward the object P.

[0062] A medical three-dimensional imaging measurement device 100 includes a light source 120 that outputs light. The light source 120 may include an LED 121. The light source 120 may include a pattern section 123 with multiple patterns formed on it. Light output from the LED 121 can illuminate the pattern section 123. The light source 120 may include a condenser lens 125 between the pattern section 123 and the LED 121, the condenser lens 125 focusing the light output from the LED 121 onto the pattern section 123. Light output from the LED 121 can pass through the pattern section 123 to reflect the pattern. According to one embodiment, light output from the light source 120 can be incident on a light path control element 140. Light incident on the light path control element 140 can be reflected in a direction configured towards a reflector 176 so that it can illuminate an object P. In another embodiment (not shown), the medical three-dimensional imaging measurement device may not include the light path control element 140, and light output from the light source 120 can be directly incident towards the reflector 176.

[0063] In one embodiment, light can be reflected by mirror 176 and irradiate the object P through opening 162h in the second housing 162. In another embodiment, not shown, the medical three-dimensional imaging measuring device may not include mirror 176, and the light can irradiate the object P through an opening formed in the light path LA without being reflected by mirror 176.

[0064] The medical 3D imaging measurement device 100 can make the light path LA of the light output from the light source 120 and illuminating the object P and the light path LB of the reflected light from the object P and reaching the camera 130 coaxial. The light path LA and the light path LB can be coaxial and overlapped in the interval between the light path control element 140 and the object P.

[0065] Light illuminating the object P can be reflected by the object P. The reflected light from the object P can then re-enter the interior of the second housing 162 through the opening 162h. In one embodiment, the reflected light can be reflected by the mirror 176 and incident on the light path control element 140. In another embodiment (not shown), the medical three-dimensional imaging measuring device may not include the mirror 176, and the reflected light can be directly incident on the light path control element 140 without additional reflection.

[0066] Reflected light incident on the light path control element 140 can pass through the light path control element 140 and reach the camera 130. In another embodiment, not shown, the medical three-dimensional imaging measuring device may not include the light path control element 140, and the reflected light can be directly incident on the camera 130.

[0067] The medical three-dimensional imaging measurement device 100 may include a camera 130 that receives the reflected light to generate image information. The camera 130 may include a condenser lens 137 through which the reflected light passes. The camera 130 may include a lens array 135 composed of a plurality of microlenses through which the reflected light passes. The camera 130 may include an image sensor 131 that captures the reflected light. The reflected light can pass through the condenser lens 137 and the lens array 135 to reach the image sensor 131.

[0068] For example, image sensor 131 can capture reflected light to generate a light field image of object P. The light field image of object P can be an image of a pattern illuminating object P. Processor 110 can use the light field image to generate a three-dimensional image of the surface of object P. Processor 110 can transmit the three-dimensional image of the surface of object P to external electronic device 20 via communication circuit 150.

[0069] The medical three-dimensional imaging measurement device 100 may include a marker 180 having a tracking surface. The marker 180 is disposed in housings 161, 162. The marker 180 may be disposed in the first housing 161.

[0070] The mark 180 is disposed on the housings 161 and 162 in a manner that allows for changing at least one of its relative position and relative orientation with respect to the opening 162h. Specifically, the mark 180 may be fixed to the first housing 161, and the second housing 162 may be coupled to the first housing 161 in a manner that allows for changing at least one of its relative position and relative orientation with respect to the first housing 161. The opening 162h may be formed on the second housing 162.

[0071] In one embodiment, the mark 180 may include a patterned surface (not shown) as the tracking surface. The mark 180 may include a lens (not shown) that enables at least a portion of the pattern inherently displayed along a direction of view from outside the mark 180 to be identifiable from outside the mark 180. The lens of the mark 180 may be a ball lens. The patterned surface may have a concave curved shape.

[0072] The external electronic device 20 can receive the image information generated by the camera 130 of the medical three-dimensional imaging measurement device 100. The external electronic device 20 may include a communication circuit 27 for receiving the image information.

[0073] The external electronic device 20 may include an imaging device 23 that captures at least a portion of the tracking surface of the marker 180 to form a tracking image. The external electronic device 20 can use the tracking image to determine the position and orientation of the marker 180, and thus determine the coordinates of the image information.

[0074] In one embodiment, the external electronic device 20 can determine the location and orientation of the medical 3D imaging measurement device 100 with the attached marker 180 based on the imaging tracking image. The location of the medical 3D imaging measurement device 100 can be defined using spatial coordinates such as coordinates on the X, Y, and Z axes of a Cartesian coordinate system. The orientation of the medical 3D imaging measurement device 100 can be defined using roll, pitch, and yaw angles. The external electronic device 20 can track the location and orientation of the medical 3D imaging measurement device 100 by capturing images of the marker 180 attached to the medical 3D imaging measurement device 100 through the imaging device 23.

[0075] For example, the imaging device 23 of the external electronic device 20 can image a pattern image of at least a portion of a pattern that can be visually recognized outside the mark 180 using a spherical lens of the mark 180. After obtaining the pattern image of at least a portion of the pattern surface, the external electronic device 20 can determine the position and orientation of the mark 180 by processing information extracted from the pattern image of at least a portion of the pattern surface. The external electronic device 20 can determine the position and orientation of the medical three-dimensional imaging measurement device 100 to which the mark 180 is attached based on the position and orientation of the mark 180. A specific method for calculating the position and orientation of the mark 180 using an image of at least a portion of the pattern surface can be applied using one of the known optical tracking methods.

[0076] The medical three-dimensional imaging measurement device 100 is capable of changing at least one of the relative position and relative orientation of the marker 180 relative to the opening 162h into which the reflected light flows. The medical three-dimensional imaging measurement device 100 may include a sensor 196 that senses displacement information based on changes in at least one of the relative position and relative orientation of the marker 180 relative to the opening. A communication circuit 150 can transmit the displacement information of the marker 180 to an external electronic device 20.

[0077] External electronic device 20 can receive the displacement information of marker 180. Based on the displacement information, external electronic device 20 can determine the coordinates of the image information generated by the medical three-dimensional imaging measurement device 100. For example, based on the displacement information, external electronic device 20 can correct the position or orientation of the medical three-dimensional imaging measurement device 100, and the corrected position or orientation information can be used for image integration between the image information (the image information generated by the medical three-dimensional imaging measurement device) and medical images (e.g., CT images, MRI images).

[0078] The image information may have an inherent coordinate system (e.g., x1y1z1 coordinate system) with respect to the medical three-dimensional imaging measurement device 100. The coordinate system of the image information may be different from the coordinate system of the medical image (e.g., x2y2z2) and may be different from the coordinate system of the external electronic device 20 (e.g., x0y0z0).

[0079] The medical image integration system 10 can transform the coordinate system of the medical image (e.g., x2y2z2) and the coordinate system of the image information (e.g., x1y1z1) into the coordinate system of the external electronic device 20 (e.g., x0y0z0). The external electronic device 20 can perform the integration of the medical image and the image information with different coordinate systems. To perform the integration of the medical image and the image information, the external electronic device 20 can extract a surface image from the medical image and perform integration between the extracted surface image and the received image information. The surface image extracted from the medical image can have the same coordinate system as the medical image (e.g., x2y2z2). Alternatively, the external electronic device 20 can use a marker 180 attached to the medical three-dimensional imaging measurement device 100 as a medium to transform the coordinate system of the image information (e.g., x1y1z1) into the coordinate system of the external electronic device 20 (e.g., x0y0z0). Furthermore, the medical image and the surface image extracted from the medical image can also be transformed into the coordinate system of the external electronic device 20 (e.g., x0y0z0). The external electronic device 20 can perform the integration between the image information and the medical image using any of a variety of image integration algorithms. For example, the external electronic device 20 can perform the integration using the ICP (Iterative Closest Point) algorithm.

[0080] The second outer shell 162 is movably disposed relative to the first outer shell 161, thereby enabling changes in its relative position and / or orientation with respect to the first outer shell 161. This relative movement encompasses both movement of the second outer shell 162 relative to the first outer shell 161 and movement of the first outer shell 161 relative to the second outer shell 161. Consequently, either the relative position or the relative orientation of the mark 180 fixed to the first outer shell 161 relative to the opening 162h can be changed.

[0081] Marker 180 can change at least one of its relative position and relative orientation relative to opening 162h by performing at least one of (i) a relative translational movement relative to opening 162h and (ii) a relative rotational movement relative to opening 162h about a predetermined rotation axis. Second housing 162 can change at least one of its relative position and relative orientation relative to opening 162h by performing at least one of (i) a relative translational movement relative to first housing 161 and (ii) a relative rotational movement relative to first housing 161 about a predetermined rotation axis. The term "rotation axis" used in this disclosure is a virtual axis and does not refer to an actual component of the device. The phrase "relative translational movement of component A relative to component B" in this disclosure means encompassing both component A performing a translational movement relative to component B and component B performing a translational movement relative to component A. Furthermore, the phrase "relative rotational movement of component C relative to component D about a predetermined rotation axis" in this disclosure means encompassing both component C rotating relative to component D about a predetermined rotation axis and component D rotating relative to component C about a predetermined rotation axis.

[0082] As an example, marker 180 can undergo a relative translational movement relative to opening 162h. Specifically, either marker 180 can translate relative to opening 162h, or opening 162h can translate relative to marker 180. The second housing 162 can perform a relative translational movement relative to the first housing 161. Therefore, the distance between marker 180 and opening 162h can be varied. For example, see reference later. Figure 7 , Figure 8 and Figure 10 In the third, fourth, and sixth embodiments described, the marker 180 can perform a relative translational movement based on the opening 162h.

[0083] As another example, marker 180 can rotate relative to opening 162h about a predetermined axis of rotation. Specifically, either marker 180 can rotate relative to opening 162h about a predetermined axis of rotation, or opening 162h can rotate relative to marker 180 about a predetermined axis of rotation. The second housing 162 can rotate relative to the first housing 161 about a predetermined axis of rotation. Therefore, the distance and / or orientation of marker 180 relative to opening 162h can be changed. See below for example. Figure 4 , Figure 5 , Figure 6 , Figure 9 and Figure 10In the first, second, fifth, and sixth embodiments described, the numeral 180 can rotate relative to the opening 162h around a predetermined rotation axis.

[0084] In reference Figure 3 In one embodiment, the second housing 162 may be coupled in a manner rotatable relative to the first housing 161. The medical three-dimensional imaging measurement device 100 may include a bearing 191 disposed between the first housing 161 and the second housing 162, so that the second housing 162 can rotate relative to the first housing 161. The second housing 162 can rotate relative to the first housing 161 about the central axis of the bearing 191.

[0085] Sensor 196 can sense displacement information based on at least one change in the relative position and relative orientation of the first housing 161 relative to the second housing 162. Sensor 196 can be any of a variety of sensors, or it can be implemented with two or more sensors.

[0086] In one embodiment, sensor 196 can sense rotation angle information of the second housing 162 relative to the first housing 161. That is, sensor 196 can sense the rotation angle information when the second housing 162 rotates relative to the first housing 161 or when the first housing 161 rotates relative to the second housing 162. For example, sensor 196 can be a gyroscope sensor or an encoder. Such a sensor can be applied to [further details omitted]. Figure 4 , Figure 5 , Figure 6 , Figure 9 and Figure 10 The first, second, fifth, and sixth embodiments are described.

[0087] In another embodiment, sensor 196 can sense distance information related to the relative movement of the second housing 162 with respect to the first housing 161. That is, sensor 196 can sense the distance information when the second housing 162 moves relative to the first housing 161 or when the first housing 161 moves relative to the second housing 162. For example, sensor 196 can be an infrared sensor, a 3D sensor, an ultrasonic sensor, an RF sensor, a geomagnetic sensor, an encoder, etc. Such a sensor can be applied to applications described later. Figure 7 , Figure 8 and Figure 10 The third, fourth, and sixth embodiments are described.

[0088] The processor 110 can transmit the displacement information to the external electronic device 20 via the communication circuit 150. In one embodiment, the processor 110 can transmit the rotation angle information to the external electronic device 20 via the communication circuit 150. In another embodiment, the processor 110 can transmit the distance information to the external electronic device 20 via the communication circuit 150.

[0089] Figure 4 This is a perspective view of the medical three-dimensional imaging measurement device 101 according to the first embodiment of this disclosure. Figure 5 It is Figure 4 A cross-sectional view of the medical three-dimensional imaging measurement device 101 taken along line S1-S1' is shown. The medical three-dimensional imaging measurement device 101 of the first embodiment will be described below, focusing on the differences from the embodiment of the medical three-dimensional imaging measurement device 100 described above. In the accompanying drawings, the direction PD of light emitted through the opening 162h and the direction MD of light viewed from the front by the marker 180 are shown.

[0090] Reference Figure 4 and Figure 5 The second housing 162 of the medical three-dimensional imaging measuring device 101 is configured to rotate relative to the first housing 161 in a manner that allows for relative rotation about a predetermined rotation axis X1, thereby enabling changes in its relative posture with respect to the first housing 161. The rotation direction C1 of the said rotational movement is shown in the accompanying drawings.

[0091] The rotation axis X1 can extend parallel to the path LB of the reflected light approaching the camera 130 (see reference). Figure 3 For example, the rotation axis X1 can be aligned with the light path LB. Thus, even if the first housing 161 and the second housing 162 rotate relative to each other, the position of the light path LB can be maintained. The sensor (not shown) of the medical three-dimensional imaging measurement device 101 can sense rotation angle information based on the relative posture change of the second housing 162.

[0092] The medical three-dimensional image measuring device 101 may include a first communication circuit 151, which transmits the rotation angle information to an external electronic device 20. The medical three-dimensional image measuring device 101 may include a second communication circuit 152, which transmits the image information generated by the camera 130 to the external electronic device 20. The medical three-dimensional image measuring device 101 may include a third communication circuit 153, which transmits the image information generated by the camera 130 and trigger information for synchronizing the image captured by the imaging device 23 of the external electronic device 20 to the external electronic device 20.

[0093] The medical three-dimensional imaging measuring device 101 can automatically perform the rotational motion. For example, the medical three-dimensional imaging measuring device 101 may include a motor (not shown) that generates driving force, and gears 197a and 162a that transmit the driving force. A driven gear 162a may be formed on the second housing 162. For example, the driven gear 162a may be a gear ring formed on the inner wall surface of the second housing 162 along the rotation direction C1. The driving gear 197a can mesh and rotate with the driven gear 162a, and can transmit the driving force to the driven gear 162a.

[0094] Figure 6 This is an elevation view of a medical three-dimensional image measuring device 102 according to a second embodiment of this disclosure. The medical three-dimensional image measuring device 102 of the second embodiment will be described below, focusing on the differences from the medical three-dimensional image measuring device 101 of the first embodiment described above.

[0095] Reference Figure 6 The medical three-dimensional imaging measurement device 102 can be manually rotated by a user. The medical three-dimensional imaging measurement device 102 may include a bearing 191 disposed between a first housing 161 and a second housing 162, so that the second housing 162 can rotate relative to the first housing 161. For example, the bearing 191 may be a ball bearing.

[0096] The sensor 196 of the medical three-dimensional imaging measurement device 101 can sense rotational angle information based on the relative posture change of the second housing 162. For example, the sensor 196 may include an encoder. The encoder may include a target portion 196a formed on either the first housing 161 or the second housing, and a sensing portion 196b formed on the other. The target portion 196a may extend along a rotational direction C1 centered on the rotation axis X1. The sensing portion 196b may be configured facing a specific position of the target portion 196a, which may change with the rotational movement. The rotational angle information can be sensed by the information about the specific position of the target portion 196a sensed by the sensing portion 196b.

[0097] Figure 7 This is an elevation view of a medical three-dimensional image measuring device 103 according to a third embodiment of this disclosure. The medical three-dimensional image measuring device 103 of the third embodiment will be described below with a focus on the differences from the medical three-dimensional image measuring devices 101 and 102 of the first and second embodiments described above.

[0098] Reference Figure 7The second housing 162 of the medical three-dimensional imaging measuring device 103 is configured to perform a relative translational movement relative to the first housing 161, thereby changing its relative position with respect to the first housing 161. The direction of movement L1 of the translational movement is shown in the figure.

[0099] The medical three-dimensional imaging measurement device 103 includes a sensor 196 that senses distance information relative to the first housing 161 based on the relative position change of the second housing 162. For example, the sensor 196 can be an infrared sensor. The sensor 196 may include a light-emitting part 196g that emits infrared light R1 and a light-receiving part 196h that senses the infrared light R1 and generates the distance information. A first communication circuit 151 can transmit the distance information to an external electronic device 20.

[0100] The medical three-dimensional imaging measurement device 103 can automatically perform the translational motion. For example, the medical three-dimensional imaging measurement device 103 may include a motor (not shown) that generates driving force, and gears 197c and 162c that transmit the driving force. A driven gear 162c may be formed on the second housing 162. For example, the driven gear 162c may be a rack formed on the inner wall surface of the second housing 162 along the movement direction L1, and the driving gear 197c may be a pinion. The driving gear 197c can mesh and rotate with the driven gear 162c, and can transmit the driving force to the driven gear 162a.

[0101] Figure 8 This is an elevation view of the medical three-dimensional image measuring device 104 according to the fourth embodiment of this disclosure. The medical three-dimensional image measuring device 104 of the fourth embodiment will be described below, focusing on the differences from the medical three-dimensional image measuring device 103 of the third embodiment described above.

[0102] Reference Figure 8 The medical three-dimensional imaging measuring device 104 can be manually performed by a user to perform the translational movement. The medical three-dimensional imaging measuring device 104 may include a sliding portion 192 formed on either the first housing 161 or the second housing 162, and a guide portion 193 formed on the other. The guide portion 193 extends along the movement direction L1. The sliding portion 192 can slide along the guide portion 193 and move along the movement direction L1.

[0103] The sensor 196 of the medical three-dimensional imaging measurement device 104 may include a sensor 196 that senses distance information relative to the first housing 161 based on the change in the relative position of the second housing 162. For example, the sensor 196 may include a linear encoder. The linear encoder may include a target portion 196c formed on either the first housing 161 or the second housing, and a sensing portion 196d formed on the other. The target portion 196c may extend along the direction of movement C1. The sensing portion 196d is configured facing a specific position of the target portion 196c, which may change with the translational movement. The distance information can be sensed by the information sensed by the sensing portion 196d regarding the specific position of the target portion 196c.

[0104] Figure 9 This is an elevation view of a medical three-dimensional image measuring device 105 according to the fifth embodiment of this disclosure. The medical three-dimensional image measuring device 105 of the fifth embodiment will be described below with a focus on the differences from the medical three-dimensional image measuring devices 101, 102, 103, and 104 of the first to fourth embodiments.

[0105] Reference Figure 9 The second housing 162 of the medical three-dimensional imaging measuring device 105 is configured to rotate relative to the first housing 161 in a manner that allows for relative rotation about a predetermined rotation axis X2, thereby enabling changes in the relative posture with respect to the first housing 161. The rotational direction C2 is shown in the figure. The medical three-dimensional imaging measuring device 105 includes a hinge 194 that allows the first housing 161 and the second housing 162 to rotate relative to each other. The hinge 194 can be configured on the rotation axis X2.

[0106] The rotation axis X2 can extend in a direction intersecting the path LB of the reflected light approaching the camera 130. In the medical three-dimensional imaging measurement device 105, the light source 120, the camera 130, and the light path control element 140 can be arranged inside the second housing 162.

[0107] The medical three-dimensional imaging measuring device 105 can automatically perform the rotational motion. For example, the medical three-dimensional imaging measuring device 105 may include a motor (not shown) that generates driving force, and gears 197e and 162e that transmit the driving force. A driven gear 162e may be formed on the second housing 162. For example, the driven gear 162e may be formed on the outer periphery of the hinge 194. The driving gear 197e can mesh and rotate with the driven gear 162e, transmitting the driving force to the driven gear 162e.

[0108] Figure 10This is an elevation view of the medical three-dimensional image measuring device 106 according to the sixth embodiment of this disclosure. The medical three-dimensional image measuring device 106 of the sixth embodiment will be described below with a focus on the differences from the medical three-dimensional image measuring devices 101, 102, 103, 104, and 105 of the first to fifth embodiments described above.

[0109] Reference Figure 10 The second housing 162 of the medical three-dimensional imaging measuring device 106 is configured to perform relative rotational motion about a rotation axis X1 and relative translational motion relative to the first housing 161, thereby enabling changes in its relative position and orientation relative to the first housing 161. The rotational direction C1 and the translational direction L1 are shown in the figure.

[0110] The medical three-dimensional imaging measurement device 106 allows the user to manually perform the rotational and translational movements. The medical three-dimensional imaging measurement device 106 includes a bearing 191 disposed between a first housing 161 and a second housing 162, enabling the second housing 162 to perform the relative rotational and translational movements relative to the first housing 161. For example, the bearing 191 can be a ball bearing. The bearing 191 can be disposed on the inner wall surface of the second housing 162.

[0111] The sensor 196 of the 110 medical three-dimensional imaging measurement device 106 can sense distance information relative to the first housing based on the relative position change of the second housing 162, and sense rotation angle information based on the relative posture change of the second housing 162. For example, the sensor 196 can be an infrared sensor. The sensor 196 may include a light-emitting part 196e that emits infrared light R1, and a light-receiving part 196f that senses infrared light R1 and generates the distance information. A plurality of light-receiving parts 196f may be arranged along the rotation direction C1, and the rotation angle information is generated based on which of the plurality of light-receiving parts 196f can sense infrared light R1.

[0112] The technical concept of this disclosure has been illustrated above with reference to some embodiments and accompanying drawings. However, it should be understood that various substitutions, modifications, and alterations can be implemented within the limits of the technical concept and scope of this disclosure that can be understood by those skilled in the art. Furthermore, such substitutions, modifications, and alterations should be considered as part of the appended claims.

Claims

1. A medical three-dimensional imaging measurement device, comprising: Light source, which outputs light; A camera that receives reflected light formed by the light being reflected by an object to generate three-dimensional image information; The housing has the camera disposed inside and has an opening for the reflected light to flow into it; and A marker, configured on the housing in a manner capable of changing at least one of its relative position and relative orientation with respect to the opening, and having a tracking surface that is captured by an external imaging device to track position and orientation, The outer casing includes: A first housing, wherein the marking is fixed in place; and A second housing, having the opening, is coupled to the first housing in a manner that allows for changes in at least one of its relative position and relative orientation relative to the first housing. The second housing is configured to rotate relative to the first housing in a manner that allows it to change its relative posture with respect to the first housing. It also includes a sensor that senses rotational angle information based on the relative posture change of the second housing. The axis of rotation extends parallel to the path of the reflected light approaching the camera.

2. The medical three-dimensional imaging measurement device according to claim 1, wherein, The light source outputs patterned light. The camera receives the reflected light formed by the patterned light being reflected by the object to generate three-dimensional image information.

3. The medical three-dimensional imaging measurement device according to claim 1, wherein, include: A sensor that senses displacement information based on a change in at least one of the relative position of the mark relative to the opening and the relative posture; and A communication circuit that transmits the displacement information to an external electronic device.

4. The medical three-dimensional imaging measurement device according to claim 1, wherein, The marker is able to change at least one of its relative position and relative orientation relative to the opening by performing at least one of (i) a relative translational movement relative to the opening and (ii) a relative rotational movement relative to the opening about a predetermined axis of rotation.

5. The medical three-dimensional imaging measurement device according to claim 1, wherein, The second housing is able to change at least one of the relative position and relative orientation of the mark relative to the opening by performing at least one of (i) a relative translational movement relative to the first housing and (ii) a relative rotational movement relative to the first housing about a predetermined axis of rotation.

6. The medical three-dimensional imaging measurement device according to claim 1, wherein, Also includes: A bearing is disposed between the first housing and the second housing so that the second housing can rotate relative to the first housing.

7. The medical three-dimensional imaging measurement device according to claim 1, wherein, The second outer shell is configured to perform a relative translational movement relative to the first outer shell, thereby changing its relative position with respect to the first outer shell. The sensor senses distance information relative to the first housing based on the relative position change of the second housing.

8. The medical three-dimensional imaging measurement device according to claim 7, wherein, Also includes: A bearing is disposed between the first housing and the second housing so that the second housing can perform relative translational and rotational movements relative to the first housing.

9. The medical three-dimensional imaging measurement device according to claim 1, wherein, The second outer shell is configured to perform a relative translational movement relative to the first outer shell, thereby changing its relative position with respect to the first outer shell. It also includes a sensor that senses distance information relative to the first housing based on the change in the relative position of the second housing.

10. The medical three-dimensional imaging measurement device according to claim 9, wherein, Also includes: A bearing is disposed between the first housing and the second housing so that the second housing can perform relative translational movement relative to the first housing.

11. A medical image integration system, comprising: A medical three-dimensional imaging measurement device includes a light source, a camera, a housing, and a marker. The light source outputs light, the camera receives reflected light from an object to generate three-dimensional image information, the housing houses the camera internally and has an opening for the reflected light to flow into the housing, and the marker is configured within the housing in a manner capable of changing at least one of its relative position and relative posture relative to the opening, and has a tracking surface that is captured by an external imaging device to track position and posture. An external electronic device, comprising an imaging device for capturing at least a portion of the tracking surface of the marker to form a tracking image, and the external electronic device receiving the three-dimensional image information and using the tracking image to determine the position and pose of the marker to determine the coordinates of the three-dimensional image information. The outer casing includes: A first housing, wherein the marking is fixed in place; and A second housing, having the opening, is coupled to the first housing in a manner that allows for changes in at least one of its relative position and relative orientation relative to the first housing. The second housing is configured to rotate relative to the first housing in a manner that allows it to change its relative posture with respect to the first housing. It also includes a sensor that senses rotational angle information based on the relative posture change of the second housing. The axis of rotation extends parallel to the path of the reflected light approaching the camera.

12. The medical image integration system according to claim 11, wherein, The medical three-dimensional imaging measurement device also includes: A sensor that senses displacement information based on a change in at least one of the relative position and relative posture of the mark relative to the opening; The external electronic device determines the coordinates of the three-dimensional image information based on the displacement information.

13. The medical image integration system according to claim 11, wherein, The light source outputs patterned light. The camera receives the reflected light formed by the patterned light being reflected by the object to generate a light field image. The medical three-dimensional imaging measurement device also includes a processor, which uses the light field image to generate a three-dimensional image of the object surface.

Images