Portable three-dimensional image measurement device, three-dimensional image measurement method using the same, and medical image integration system

Abstract

A portable three-dimensional image measuring apparatus according to various embodiments of the present disclosure includes a light source that outputs pattern light, a camera that receives reflected light generated by the pattern light reflected from an object body to generate a light field image of the object body, and a light path control element that reflects the pattern light so that the pattern light output from the light source is irradiated to the object body and transmits the reflected light so that the reflected light reflected from the object body reaches the camera, a light path of the pattern light irradiated to the object body from the light source and a light path of the reflected light reaching the camera from the object body are coaxial and overlap in an interval between the light path control element and the object body.

Description

Technical Field

[0001] This disclosure relates to a portable three-dimensional imaging measurement device. More specifically, it relates to a method for performing precise three-dimensional measurements on an object under test using a portable three-dimensional imaging measurement device. Furthermore, it relates to a medical imaging integration system including a portable three-dimensional imaging measurement device.

[0002] This disclosure is derived from research conducted as part of technical development support for the WC300 project. [Project No.: S2482672, Research Topic: Development of a Head and Neck Surgical Robot System Integrating Navigation Fusion with Precision Below 1mm] Background Technology

[0003] Various methods for measuring three-dimensional images of objects are being used in industrial settings. Among these methods, one approach involves measuring the pattern created by shining patterned light onto an object and thereby acquiring a three-dimensional image of that object. For example, there is a moiré 3D image measurement technique that measures the moiré fringe pattern created by shining patterned light onto an object and thereby acquiring a three-dimensional image of that object.

[0004] Recently, surgical navigation technology has been used to support surgeons in surgical procedures. Typical surgical navigation technology provides information by placing markers on surgical instruments and representing the position and orientation of these instruments on the patient's medical images (e.g., CT images, MRI images). In such surgical navigation systems, techniques for measuring three-dimensional images are used because it is necessary to acquire and process three-dimensional image information specific to the patient's affected area. Summary of the Invention

[0005] Technical problems to be solved

[0006] In the case of a 3D image measurement device using a stereo camera, a 3D image of an object can be measured using a fixed-pattern triangulation method. Triangulation is a method of measuring a 3D image of an object using two or more images captured from different positions. For example, a patterned light can be shone onto the object, and two or more cameras positioned at different positions can acquire images of the object illuminated by the patterned light. A 3D image of the object can then be obtained from these acquired images. In the case of the aforementioned 3D image measurement device, two or more cameras are required to use the triangulation method. The device also requires two or more cameras and light sources that illuminate the object with patterned light from different positions.

[0007] In a 3D image measurement device using a chromatic confocal sensor, the depth of an object can be measured by utilizing the chromatic aberration of the lens, thereby measuring a 3D image of the object. When using a confocal sensor for 3D imaging, a scan of the object's side is required to acquire the 3D image.

[0008] Solution to the problem

[0009] A portable three-dimensional image measuring device according to various embodiments of the present disclosure includes: a light source that outputs patterned light; a camera that receives reflected light generated by the patterned light reflecting off an object, thereby generating a light field image of the object; and a light path control element that reflects the patterned light so that the patterned light output from the light source illuminates the object, and transmits the reflected light so that the reflected light reflected off the object reaches the camera, wherein the light path of the patterned light output from the light source illuminating the object and the light path of the reflected light reflected off the object reaching the camera can be coaxial and overlapped in the interval between the light path control element and the object.

[0010] A medical image integration system according to various embodiments of the present disclosure includes: a light source that outputs patterned light; a camera that receives reflected light generated by the patterned light reflecting off an object, thereby generating a light field image of the object; a portable three-dimensional image measuring device, the portable three-dimensional image measuring device including a light path control element, a communication circuit, and a processor, the light path control element reflecting the patterned light so that the patterned light output from the light source illuminates the object, and transmitting the reflected light so that the reflected light reflected off the object reaches the camera; and an external electronic device capable of communicating with the communication circuit, wherein the light path of the patterned light output from the light source illuminating the object and the light path of the reflected light reflected off the object reaching the camera can be coaxially aligned and overlapped in the interval between the light path control element and the object, the processor generating a three-dimensional image of the surface of the object using the light field image of the object acquired by the camera, and transmitting the three-dimensional image of the surface of the object to the external electronic device via the communication circuit.

[0011] A three-dimensional image measurement method using a portable three-dimensional image measurement device according to various embodiments of the present disclosure may include: an operation of illuminating an object with patterned light output from a light source through a light path control element; and an operation of a camera receiving reflected light generated by the patterned light reflected from the object through the light path control element, thereby generating a light field image of the object, wherein the light path of the patterned light output from the light source illuminating the object and the light path of the reflected light reflected from the object and reaching the camera may be coaxial and overlapped in the interval between the light path control element and the object.

[0012] The effects of the invention

[0013] The three-dimensional image measuring apparatus according to various embodiments of the present disclosure can generate a light field image of an object using a camera that generates a light field image, and generate a three-dimensional image of the object's surface using the light field image of the object. Since the three-dimensional image measuring apparatus according to various embodiments of the present disclosure can be implemented using only one camera, it can be miniaturized compared to existing stereoscopic three-dimensional image measuring apparatuses that include two or more cameras. Furthermore, since it can be implemented using only one camera, the production cost of the three-dimensional image measuring apparatus can be reduced, and its weight can be reduced to improve portability.

[0014] According to various embodiments of this disclosure, when photographing an object using a miniaturized 3D image measuring device, the user can easily move the 3D image measuring device and easily change its shooting posture. In this case, the user can use the 3D image measuring device to photograph the object in various postures (e.g., lying down and prone).

[0015] Since the three-dimensional image measuring apparatus according to various embodiments of the present disclosure uses a camera that generates light field images, a three-dimensional image of the object can be generated even with a single measurement.

[0016] In the three-dimensional image measuring apparatus according to various embodiments of the present disclosure, since the optical axis of the light source illuminating the patterned light and the optical axis of the camera receiving the light reflected from the object are coaxial in a certain section, the distortion of the acquired light field image of the object can be minimized, and this can be achieved through miniaturization. When the optical axis of the light source and the optical axis of the camera are coaxial in a certain section, the phenomenon of light field image distortion due to the tilt of the patterned light illuminating from the light source may not occur.

[0017] In the three-dimensional image measuring apparatus according to various embodiments of the present disclosure, since the optical axis of the light source illuminating the pattern light and the optical axis of the camera receiving the light reflected from the object are coaxial in a certain interval, the pattern light can be uniformly illuminating the object and the loss of light can be minimized. Attached Figure Description

[0018] Figure 1 This is a block diagram illustrating a medical image integration system according to various embodiments of the present disclosure.

[0019] Figure 2a and 2b This is a cross-sectional view of a portable three-dimensional image measuring device according to various embodiments of the present disclosure.

[0020] Figure 3 This is a cross-sectional view of a portable three-dimensional image measuring device according to various embodiments of the present disclosure.

[0021] Figure 4 This is an operation flowchart of a medical image integration system according to various embodiments of the present disclosure.

[0022] Figure 5 This is an operation flowchart of a medical image integration system according to various embodiments of the present disclosure.

[0023] Figure 6 This is an operation flowchart of a portable three-dimensional image measuring device according to various embodiments of the present disclosure.

[0024] Figure 7 This is a diagram illustrating examples of medical image integration systems using various embodiments of the present disclosure.

[0025] Figure 8 This is a diagram illustrating the structure of a camera according to various embodiments of the present disclosure.

[0026] Figure 9 This is a diagram illustrating the lens array of a camera according to various embodiments of the present disclosure.

[0027] Figure 10 This is a diagram illustrating the process by which multiple sub-images included in a light field image acquired by a camera according to various embodiments of the present disclosure are formed at different object depths.

[0028] Figure 11 This is a diagram illustrating a light field image comprising multiple sub-images having different depths of the subject compared to each other, according to various embodiments of the present disclosure. Detailed Implementation

[0029] The embodiments disclosed herein are shown for the purpose of illustrating the technical ideas of this disclosure. The scope of this disclosure is not limited to the embodiments described below or the specific descriptions of these embodiments.

[0030] All technical and scientific terms used in this disclosure, unless otherwise defined, shall have the meanings commonly understood by one of ordinary skill in the art to which this disclosure pertains. All terms used in this disclosure have been chosen for the purpose of clearly describing this disclosure and not for limiting the scope of the claims under this disclosure.

[0031] Expressions such as “comprising,” “possessing,” and “having” as used in this disclosure, unless otherwise stated in the phrase or sentence that includes such expression, should be understood as open-ended terms that include the possibility of including other embodiments.

[0032] The singular expressions described in this disclosure, unless otherwise stated, may include the meaning of the plural, and this also applies to the singular expressions described in the claims.

[0033] The terms "first," "second," etc., used in this disclosure are used to distinguish multiple constituent elements from each other, and do not limit the order or importance of the constituent elements.

[0034] As used in this disclosure, the term "component" refers to software or hardware components such as FPGA (field-programmable gate array) and ASIC (application-specific integrated circuit). However, "component" is not limited to hardware and software. A "component" can be configured to reside on addressable storage media or to reproduce one or more processors 110. Thus, as an example, a "component" includes components such as software components, object-oriented software components, class components, and task components, as well as processors 110, functions, attributes, programs, subroutines, fragments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided by the components and "components" can be combined into a smaller number of components and "components" or further separated into additional components and "components".

[0035] As used in this disclosure, the expression “based on” is used to describe one or more factors that influence the determination or judgment of an action or behavior described in a phrase or sentence that includes the expression, and the expression does not exclude additional factors that influence the determination or judgment of an action or behavior.

[0036] In this disclosure, when a constituent element is referred to as being “connected to” or “connected to” another constituent element, it should be understood that the one constituent element may be directly connected to or connected to the other constituent element, or may be connected or connected through a new constituent element.

[0037] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In the drawings, the same or corresponding components are given the same reference numerals. Furthermore, in the following description of the embodiments, repeated descriptions of the same or corresponding components may be omitted. However, even if descriptions of components are omitted, it does not mean that these components are not included in a particular embodiment.

[0038] Although process steps, method steps, algorithms, etc., are illustrated sequentially in the flowcharts shown, these processes, methods, and algorithms can be configured to operate in any suitable order. That is, the steps of the processes, methods, and algorithms described in the various embodiments of this disclosure do not need to be executed in the order described in this disclosure. Furthermore, although some steps are described as being executed asynchronously, in other embodiments, these partial steps can be executed simultaneously. Moreover, the examples of processes described in the accompanying drawings do not imply that the illustrated processes exclude other changes and modifications thereto, nor do they imply that any illustrated process or its steps are necessary for more than one of the various embodiments of this disclosure, nor do they imply that the illustrated processes are preferred.

[0039] Figure 1 This is a block diagram illustrating a medical image integration system 10 according to various embodiments of the present disclosure.

[0040] 37 Reference Figure 1 The medical image integration system 10 according to various embodiments may include a portable three-dimensional image measurement device 100 and an external electronic device 20. The portable three-dimensional image measurement device 100 and the external electronic device 20 are communicatively connected to each other, thereby enabling the transmission / reception of various data (e.g., images). Even if omitted or replaced... Figure 1 The partial configuration shown herein will not prevent the implementation of the various embodiments disclosed herein.

[0041] 38. The portable three-dimensional image measuring device 100 according to various embodiments may include a processor 110, a light source 120, a camera 130, a light path control element 140, or a communication circuit 150. The portable three-dimensional image measuring device 100 may also include a first housing (not shown) and a second housing (not shown).

[0042] 39. The processor 110 according to various embodiments may be a configuration capable of performing calculations or data processing related to the control and / or communication of various components of the portable 3D image measuring device 100. The processor 110 may, for example, be operatively connected to the components of the portable 3D image measuring device 100. The processor 110 may load commands or data received from other components of the portable 3D image measuring device 100 into a memory (not shown), process the commands or data stored in the memory, and store the resulting data.

[0043] According to various embodiments, the light source 120 can output patterned light. The light source 120 can illuminate an object with the patterned light. In order to measure a three-dimensional image of the object, the patterned light can be light with a specific texture or light with a certain or specific periodic pattern. The patterned light may, for example, include patterned light in the form of random dots, patterned light in the form of grids, patterned light with stripes of sine wave brightness, or patterned light in the form of an on-off switch where bright and dark parts are repeatedly displayed, or patterned light with a triangular wave pattern in which the brightness changes. However, this is only for illustrative purposes, and the shape of the patterned light is not limited to these.

[0044] The light source 120 according to various embodiments may include: a patterned portion having multiple patterns formed thereon; and an LED that illuminates the patterned portion. The light source 120 may also include a condensing lens configured to focus the light emitted from the LED onto the patterned portion. The light emitted from the LED can pass through the patterned portion having multiple patterns and reflect the patterns. The LED may emit infrared light, for example, but is not limited thereto.

[0045] According to various embodiments, the camera 130 can be configured to capture images of an object. The camera 130 can capture images of the object to obtain image data, and can process the acquired image data to obtain a three-dimensional image of the object. For example, the camera 130 can capture images of an object illuminated by patterned light. The processor 110 can generate a three-dimensional image of the object based on a phase-shifting method utilizing the patterned light. For example, when patterned light of a certain shape is illuminated onto the object by the light source 120, the light intensity displayed on the surface may vary depending on the curvature of the object's surface. In this case, the camera 130 can generate a light field image reflecting the patterned object, and the processor 110 can thereby generate phase data and calculate the height of various points constituting the surface of the object, thus generating a three-dimensional image of the object's surface.

[0046] According to various embodiments, the camera 130 can be a light field camera 130 that generates a light field image. The light field camera 130 can determine the depth of the object after it has been photographed and combine images having different object depths. The image sensor of the light field camera 130 can have a post-processed, variable object depth.

[0047] Camera 130 according to various embodiments may include a condenser lens, a lens array, and an image sensor. The condenser lens, for example, can focus light entering from an object. The lens array, for example, may be a lens with multiple microlenses arranged in a row. The image sensor, for example, can capture light passing through the lens array and generate a light field image using the captured light. The image sensor may be divided into regions corresponding to each of the multiple microlenses. The image sensor may include, for example, a CCD (charge-coupled device) sensor or a CMOS (complementary metal-oxide semiconductor) sensor. Figures 8 to 11 The text provides a detailed description of each component included in the camera 130.

[0048] The light field image generated by the camera 130 according to various embodiments may include multiple sub-images that store color and direction information of light together. For example, when patterned light illuminates an object and reflected light from the object is received by the camera 130, the light field image may be an image composed of multiple sub-images including color and direction information of the reflected light. The camera 130 may use the multiple sub-images included in the light field image to perform a refocusing process. For example, during refocusing, the camera 130 may combine the desired depth of the object in the pixels of the light field image with the color information of the pixels corresponding to the light path and direction calculated therefrom to generate an image with the desired depth. For example, during refocusing, the camera 130 may also generate an image focused on all areas of the object. In order for the camera 130 to form an accurate image of the object area, it is necessary to appropriately adjust the distance between the portable three-dimensional image measuring device 100 and the object area of ​​the object. When using the camera 130 that generates the light field image, the depth of the object can be determined afterward, and a light field image focused on all areas of the object can be generated, so there is no need to adjust the focus beforehand. In the case of camera 130 that generates light field images, compared with camera 130 using a normal lens, the measurable depth range is wider, and a three-dimensional image of the object can be acquired in a single shot.

[0049] According to various embodiments, the light path control element 140 can reflect patterned light in a specific direction so that the patterned light output from the light source 120 illuminates the object. The light path control element 140 can also transmit reflected light so that the reflected light from the object reaches the camera 130. The light path control element 140 can be, for example, a semi-transparent mirror. According to various embodiments, the light source 120 and the camera 130 can be configured in mutually perpendicular directions with respect to the light path control element 140.

[0050] According to various embodiments, a first housing may internally house a light source 120, a camera 130, and a light path control element 140. According to various embodiments, a second housing may be attached to the first housing and may form an opening to allow patterned light emitted from the light source 120 to illuminate the object. The second housing may be rotatably attached relative to the first housing. The first or second housing may incorporate a structure (e.g., a handle) to facilitate easy movement, handling, and use of the portable 3D imaging measurement device 100 by a user.

[0051] According to various embodiments, the communication circuit 150 can establish a communication channel with the external electronic device 20 and send / receive various data with the external electronic device 20. According to various embodiments, the communication circuit 150 may include a cellular communication module for connecting to a cellular network (e.g., 3G, LTE, 5G, Wibro, or WiMAX). According to various embodiments, the communication circuit 150 may include a near-field communication module to send / receive data with the external electronic device 20 using near-field communication (e.g., Wi-Fi, Bluetooth, Bluetooth Low Energy (BLE), UWB), but is not limited thereto.

[0052] According to various embodiments, the processor 110 can generate a three-dimensional image of the surface of an object using a light field image of the object acquired by the camera 130. For example, depending on the curvature of the surface of the object's shooting area, the intensity of the patterned light irradiated on the actual shooting object area surface may vary. The processor 110 can use the light field image of the object to measure the light intensity varying according to the surface curvature of the object, and thereby generate phase data to calculate the height of each point constituting the surface. The processor 110 can generate a three-dimensional image of the surface of the object by calculating the height of each point constituting the surface of the object.

[0053] According to various embodiments, the processor 110 can transmit a three-dimensional image of the surface of an object to an external electronic device 20 via the communication circuit 150.

[0054] The external electronic device 20, according to various embodiments, may include a controller 21, an imaging device 23, a memory 25, and a communication circuit 27. The controller 21, according to various embodiments, may be a configuration capable of performing control and / or communication-related calculations or data processing of the various components of the external electronic device 20. The controller 21 may, for example, be operatively connected to the components of the external electronic device 20.

[0055] Imaging device 23 according to various embodiments can image a patterned image of at least a portion of the patterned surface of a mark (not shown) attached to a portable three-dimensional image measuring device 100. Imaging device 23 may, for example, include at least two or more cameras capable of image imaging of at least a portion of the mark. External electronic device 20 can use the imaged patterned image to determine the position and / or orientation of the mark or the portable three-dimensional image measuring device 100 with the mark attached.

[0056] For example, when the external electronic device 20 acquires a pattern image of the marker, at least one of the sub-patterns, as the basic unit constituting the pattern of the marker from the pattern image, can be extracted. The position of the extracted at least one sub-pattern within the entire pattern can be determined, and the orientation of the marker can be determined based on the determined position of the sub-pattern within the entire pattern. Here, the orientation of the marker can refer to the relative three-dimensional orientation or orientation of the marker with respect to the imaging device 23. For example, the position of the marker or the portable three-dimensional image measuring device 100 can be determined using triangulation based on two images with a stereoscopic relationship in the images captured by the imaging device 23, which includes at least two cameras. When the position and orientation of the marker are determined as described above, the position and orientation of the portable three-dimensional image measuring device 100 with the marker attached can be determined based on the geometric relationship between the marker and the portable three-dimensional image measuring device 100 with the marker attached.

[0057] According to various embodiments, the memory 25 can store various data used by at least one component of the external electronic device 20 (e.g., controller 21). For example, the memory 25 can store a three-dimensional image of the surface of an object received by the controller 21 from the portable three-dimensional imaging measuring device 100. For example, the memory 25 can store medical images (e.g., CT images, MRI images) received by the controller 21 from a medical device (not shown).

[0058] The communication circuit 27 of the external electronic device 20, according to various embodiments, can establish a communication channel with the portable 3D image measuring device 100 and send / receive various data with the portable 3D image measuring device 100. According to various embodiments, the communication circuit 27 of the external electronic device 20 may include a cellular communication module for connecting to a cellular network (e.g., 3G, LTE, 5G, Wibro, or WiMAX). According to various embodiments, the communication circuit 27 of the external electronic device 20 may include a near-field communication module to send / receive data with the portable 3D image measuring device 100 using near-field communication (e.g., Wi-Fi, Bluetooth, Bluetooth Low Energy (BLE), UWB), but is not limited thereto.

[0059] According to various embodiments, the controller 21 of the external electronic device 20 can perform image integration between a three-dimensional image of the surface of an object received from the portable three-dimensional image measuring device 100 and a medical image of the object. The three-dimensional image of the surface of the object generated by the portable three-dimensional image measuring device 100 may be the external surface of a target or a portion thereof included in the aforementioned medical image. For example, when the medical image is an image modeling the three-dimensional shape of the head of an object, the three-dimensional image of the surface of the object may be an image measuring the external shape of the eyes, nose, mouth, ears, etc., of the object's head surface.

[0060] According to various embodiments, a three-dimensional image of the surface of an object may have an inherent coordinate system (e.g., x1y1z1 coordinate system) associated with the portable three-dimensional imaging measurement device 100. The coordinate system of the three-dimensional image of the object's surface may differ from the coordinate system of the medical image (e.g., x2y2z2) and may also differ from the coordinate system of the external electronic device 20 (e.g., x0y0z0). The coordinate system of the external electronic device 20 may, for example, refer to the coordinate system of the imaging device of the external electronic device 20. Hereinafter, Figure 4 The specific image integration method is explained in the text.

[0061] Figure 2a and 2b This is a cross-sectional view of a portable three-dimensional image measuring device 100 according to various embodiments. Specifically, Figure 2a and 2b This diagram is only schematically shown to illustrate the configuration of the components of the portable 3D image measuring device 100. (The remaining text is incomplete and likely refers to further details about the diagram.) Figure 1 The content described in the text is repeated.

[0062] Reference Figure 2a The portable three-dimensional image measuring device 100, according to various embodiments, may include a light source 120, a camera 130, and a light path control element 140. The portable three-dimensional image measuring device 100 may include: a first housing 210, the light source 120, camera 130, and light path control element 140 disposed inside the first housing 210; and a second housing 220, which is coupled to the first housing 210 and forms an opening 225 to allow patterned light output from the light source 120 to illuminate the object O. According to various embodiments, the light source 120 and camera 130 may be arranged in mutually perpendicular directions with reference to the light path control element 140. According to various embodiments, at least one condenser lens 231, 235 for focusing light may be disposed around the light path control element 140.

[0063] According to various embodiments, the light source 120 may include: a patterned portion 123 having formed a plurality of patterns; and an LED 121 that illuminates the patterned portion. The light source 120 may also include: a condenser lens 125 located between the patterned portion 123 and the LED 121, which focuses the light output from the LED 121 onto the patterned portion 123. The light output from the LED 121 can pass through the patterned portion 123 and reflect the pattern. According to various embodiments, the patterned light output from the light source 120 can be incident on a light path control element 140. The patterned light incident on the light path control element 140 can be reflected toward the second housing 220 to illuminate the object O. The patterned light incident inside the second housing 220 can be reflected by a reflector 240 and illuminate the object O through an opening 225 in the second housing 220.

[0064] According to various embodiments, patterned light illuminating object O can be reflected by object O. The reflected light from object O can again enter the interior of the second housing 220 through opening 225. The reflected light can be reflected by mirror 240 and enter the light path control element 140. The reflected light entering the light path control element 140 can pass through the light path control element 140 to reach camera 130. The reflected light passing through the light path control element 140 can pass through condenser lens 137 and through lens array 135 with multiple microlenses arranged. Image sensor 131 can capture the reflected light passing through lens array 135. Image sensor 131 can capture the reflected light to generate a light field image of object O. The light field image of object O can be an image related to the pattern illuminating object O. Processor 110 can use the light field image of object O to generate a three-dimensional image of the surface of object O, and can transmit the three-dimensional image of the surface of object O to external electronic device 20 through communication circuit 150.

[0065] According to various embodiments, the optical path 250 of the patterned light emitted from the light source 120 and illuminating the object O, and the optical path 260 of the reflected light reflected from the object O and reaching the camera 130, can be coaxial and overlapped in the interval between the optical path control element 140 and the object O. When the optical path 250 of the patterned light illuminating the object O and the optical path 260 of the reflected light reflected from the object O are coaxial, the portable three-dimensional image measuring device 100 can be miniaturized, and an accurate image of the object O can be acquired. For example, when the optical path 250 of the patterned light illuminating the object O and the optical path 260 of the reflected light reflected from the object O are not coaxial, the patterned light may illuminate the object O at an angle, or the light reflected from the object O may reach the camera 130 at an angle. Compared to the pattern formed on the object O by the vertically illuminating patterned light, the pattern formed on the object O by the obliquely illuminating patterned light may have a distorted shape. In the above case, a distorted image of the object O may be acquired. Conversely, when the light path 250 of the patterned light illuminating the object O is coaxial with the light path 260 of the reflected light reflected from the object O, the user can use the three-dimensional image measuring device 100 to obtain an accurate image of the object O without distortion.

[0066] According to various embodiments, when the light path 250 of the patterned light illuminating the object O is coaxial with the light path 260 of the reflected light reflected from the object O, the portable three-dimensional image measuring device 100 can easily photograph the object O. When photographing the object O using the miniaturized three-dimensional image measuring device 100, the user can easily move the three-dimensional image measuring device 100 and easily change the shooting posture of the three-dimensional image measuring device 100. In the above case, the user can use the three-dimensional image measuring device 100 to photograph the object in various postures (e.g., lying down and prone postures).

[0067] According to various embodiments, when the light path 250 of the patterned light illuminating the object O is coaxial with the light path 260 of the reflected light reflected from the object O, the portable three-dimensional image measuring device 100 can acquire an image of the object using a single camera 130 instead of a triangulation method using two or more cameras. Therefore, compared with existing stereoscopic three-dimensional image measuring devices that include two or more cameras, it can be miniaturized, reduce production costs, and improve portability by reducing weight.

[0068] In the three-dimensional image measuring apparatus 100 according to various embodiments, since the light path 250 of the pattern light illuminating the object body O is coaxial with the light path 260 of the reflected light reflected from the object body O, the pattern light can be uniformly illuminating the object body O and the loss of light can be minimized.

[0069] Reference Figure 2b The portable three-dimensional image measuring device 100 according to various embodiments may not include Figure 2a The separate reflector 240 is shown. In the above case, the patterned light reflected from the light path control element 140 can illuminate the object O through the opening 225 formed on the light path 250 of the patterned light without additional reflection. In the portable three-dimensional image measuring device 100, in addition to Figure 2a and Figure 2b In addition to the structure shown, various other structures can be applied so that the light path 250 of the patterned light illuminating the object O and the light path 260 of the reflected light reflected from the object can be coaxial.

[0070] Figure 3 This is a cross-sectional view of a portable three-dimensional image measuring device 100 according to various embodiments. Specifically, Figure 3 The diagram is schematically shown only for illustrating the configuration of the components of the portable three-dimensional image measuring device 100. Content that is repeated in the description in Figure 2 will be omitted.

[0071] The portable three-dimensional image measuring device 100 according to various embodiments may further include a marker 310. For example, the marker 310 may be attached to a first housing 210 of the portable three-dimensional image measuring device 100. The marker 310 may include: a patterned surface on which a pattern is formed; and a lens that can identify from the outside of the marker 310 at least a portion of the pattern that is inherently present according to the direction of viewing from the outside of the marker 310. The lens of the marker 310 may be a ball lens, and the patterned surface may have a curved shape.

[0072] According to various embodiments, the external electronic device 20 can image a patterned image of at least a portion of the patterned surface of the mark 310 by capturing images of at least a portion of the patterned surface of the mark 310 using the imaging device 23. The external electronic device 20 can determine the location and orientation of the portable 3D imaging measuring device 100 with the mark 310 attached based on the imaged patterned image. The location of the portable 3D imaging measuring device 100 can be defined as spatial coordinates such as those on the x, y, and z axes of a Cartesian coordinate system. The orientation of the portable 3D imaging measuring device 100 can be defined as roll, pitch, and yaw. The external electronic device 20 can track the location and orientation of the portable 3D imaging measuring device 100 by capturing images of the mark 310 attached to the portable 3D imaging measuring device 100 using the imaging device 23.

[0073] For example, the external electronic device 20 can image at least a portion of the patterned surface of the mark 310 attached to the portable 3D image measuring device 100 using the imaging device 23. For example, the imaging device 23 of the external electronic device 20 can image a pattern image of at least a portion of a visually recognizable pattern from the outside of the mark 310 using the spherical lens of the mark 310. When the pattern image of at least a portion of the patterned surface is acquired, the external electronic device 20 can process the information extracted from the pattern image of at least a portion of the patterned surface to determine the position and orientation of the mark 310. The external electronic device 20 can determine the position and orientation of the portable 3D image measuring device 100 with the mark 310 attached based on the position and orientation of the mark 310. The specific method for calculating the position and orientation of the mark 310 using the image of at least a portion of the patterned surface can be the same as a general optical tracking method.

[0074] According to various embodiments, the marker 310 can be set in a manner that allows it to move from a predetermined position of the portable 3D imaging measuring device 100. The processor 110 can transmit information indicating the displacement of the marker 310 from the predetermined position of the portable 3D imaging measuring device 100 to an external electronic device 20 via a communication circuit 150. The external electronic device 20 receives the information indicating the displacement of the marker 310 and, based on the received information indicating the displacement of the marker 310, corrects the position or orientation of the portable 3D imaging measuring device 100. The corrected information related to the position or orientation of the portable 3D imaging measuring device 100 can be used for image integration between a 3D image of the surface of the object O and a medical image.

[0075] The three-dimensional image of the surface of object O may have an inherent coordinate system (e.g., x1y1z1 coordinate system) associated with the portable three-dimensional imaging measurement device 100. The coordinate system of the three-dimensional image of the surface of object O may be different from the coordinate system of the medical image (e.g., x2y2z2) or the coordinate system of the external electronic device 20 (e.g., x0y0z0).

[0076] According to various embodiments, the medical image integration system 10 can transform or align the coordinate system of a medical image (e.g., x2y2z2) and the coordinate system of a three-dimensional image of the surface of an object O (e.g., x1y1z1) to the coordinate system of an external electronic device 20 (e.g., x0y0z0). The external electronic device 20 can perform the integration of medical images and three-dimensional images of the surface of the object O with different coordinate systems. To perform the integration of the medical image and the three-dimensional image of the surface of the object O, the external electronic device 20 can extract a surface image from the medical image and perform integration between the extracted surface image and the received three-dimensional image of the surface of the object O. Here, the surface image extracted from the medical image can be the same as the coordinate system of the medical image (e.g., x2y2z2). Furthermore, the external electronic device 20 can use a mark 310 attached to the portable three-dimensional imaging measuring device 100 as a medium to transform the coordinate system of the three-dimensional image of the surface of the object O (e.g., x1y1z1) into the coordinate system of the external electronic device 20 (e.g., x0y0z0). Furthermore, medical images and surface images extracted from medical images can also be converted into the coordinate system of external electronic device 20 (e.g., x0y0z0). External electronic device 20 can utilize various image integration algorithms to perform integration between the three-dimensional image of the surface of object O and the medical image. For example, external electronic device 20 can utilize the ICP (interactive closest point) algorithm to perform integration.

[0077] The portable three-dimensional image measuring device 100 according to various embodiments may further include a bearing 320 and a sensor 330. The second housing 220 according to various embodiments may be rotatably coupled relative to the first housing 210. The bearing 320 may be a mechanical element that rotatably coupled the second housing 220 relative to the first housing 210. The second housing 220 may rotate about the central axis of the bearing 320 or rotate independently of the first housing 210. The sensor 330 may be a sensor that senses the rotation angle of the second housing 220 relative to the first housing 210. The sensor 330 may be, for example, a gyroscope sensor or an encoder. The processor 110 according to various embodiments may transmit information about the rotation angle of the second housing 220 relative to the first housing 210 to an external electronic device 20 via a communication circuit 150.

[0078] Figure 4 This is an operation flowchart of the medical image integration system 10 according to various embodiments of the present disclosure.

[0079] Referring to the operation flowchart 400, in operation 401, the external electronic device 20, according to various embodiments, can receive and store medical images of the subject from the medical device. The medical images may be, for example, CT images or MRI images.

[0080] In operation 403, the portable 3D image measuring device 100, according to various embodiments, can illuminate the object with patterned light. For example, the portable 3D image measuring device 100 can output patterned light through a light source 120. The patterned light output through the light source 120 can be reflected by the light path control element 140 and illuminate the object.

[0081] In operation 405, the portable three-dimensional image measuring device 100, according to various embodiments, can receive reflected light from an object to generate a light field image of the object. For example, reflected light from the object can reach the camera 130 via the light path control element 140. The camera 130 can receive the reflected light to generate a light field image of the object.

[0082] In operation 407, the portable three-dimensional image measuring device 100, according to various embodiments, can generate a three-dimensional image of the object's surface using a light field image of the object. For example, the processor 110 can measure the light intensity of patterned light included in the light field image of the object and generate phase data based on the measured light intensity of the patterned light. The processor 110 can generate a three-dimensional image of the object's surface by calculating the height of various points constituting the object's surface based on the generated phase data.

[0083] In operation 409, the portable 3D image measuring device 100, according to various embodiments, can transmit a 3D image of the surface of an object to an external electronic device 20. For example, the processor 110 of the portable 3D image measuring device 100 can transmit the 3D image of the surface of the object to the external electronic device 20 via a communication circuit 150.

[0084] In operation 411, the external electronic device 20, according to various embodiments, can perform image integration between a three-dimensional image of the surface of the object received from a portable three-dimensional measuring device and a pre-stored medical image of the object.

[0085] Figure 5 This is an operation flowchart of a medical image integration system according to various embodiments of the present disclosure. (The remaining text is omitted.) Figure 4 The content described in the text is repeated.

[0086] Referring to the operation flowchart 500, in operation 501, the external electronic device 20, according to various embodiments, can receive and store medical images of the subject from the medical device.

[0087] In operation 503, the portable three-dimensional image measuring device 100 according to various embodiments can illuminate the object with patterned light.

[0088] In operation 505, the portable three-dimensional image measuring device 100 according to various embodiments can receive reflected light from an object and generate a light field image of the object.

[0089] In operation 507, the portable three-dimensional image measuring device 100 according to various embodiments can generate a three-dimensional image of the surface of the object using the light field image of the object.

[0090] In operation 509, the portable three-dimensional image measuring device 100 according to various embodiments can transmit a three-dimensional image of the surface of an object to an external electronic device 20.

[0091] In operation 511, the external electronic device 20, according to various embodiments, images at least a portion of the patterned surface of the mark 310 attached to the portable three-dimensional image measuring device 100. For example, the external electronic device 20 can image a patterned image of at least a portion of the patterned surface of the mark 310 using an imaging device.

[0092] In operation 513, the external electronic device 20, according to various embodiments, can determine the position and orientation of the portable three-dimensional image measuring device 100 with the attached marker 310 based on the image of the pattern. For example, the external electronic device 20 can determine the position and orientation of the marker 310 by processing information extracted from the pattern image targeting at least a portion of the pattern surface. The external electronic device 20 can determine the position and orientation of the portable three-dimensional image measuring device 100 with the attached marker 310 based on the position and orientation of the marker 310.

[0093] In operation 515, the external electronic device 20, according to various embodiments, can perform image integration between a three-dimensional image of the object's surface and a medical image of the object. For example, the external electronic device 20 can convert the coordinate system of the three-dimensional image of the object's surface into the coordinate system of the external electronic device 20 using a marker 310 attached to the portable three-dimensional imaging measuring device 100 as a medium. The external electronic device 20 can also convert the coordinate system of the medical image into its own coordinate system. After completing the coordinate system transformation, the external electronic device 20 can perform image integration between the three-dimensional image of the object's surface and the medical image of the object.

[0094] Figure 6 This is an operation flowchart of a portable three-dimensional image measuring device 100 according to various embodiments of the present disclosure.

[0095] Referring to the operation flowchart 600, in operation 610, the portable 3D image measuring device 100, according to various embodiments, can illuminate the object with patterned light output from the light source 120. For example, the portable 3D image measuring device 100 can output patterned light from the light source 120. The output patterned light can be reflected by the light path control element 140 and illuminate the object.

[0096] In operation 620, the portable 3D image measuring device 100, according to various embodiments, can receive reflected light generated by patterned light reflecting off an object. For example, patterned light illuminating an object can be reflected off the object and re-enter the portable 3D image measuring device 100. The reflected light can pass through the light path control element 140 to reach the camera 130.

[0097] In operation 630, the portable three-dimensional image measuring device 100 according to various embodiments can generate a light field image of an object by receiving reflected light through a camera 130. In operation 640, the portable three-dimensional image measuring device 100 according to various embodiments can generate a three-dimensional image of the object's surface based on the light field image of the object. In operation 650, the portable three-dimensional image measuring device 100 according to various embodiments can transmit the three-dimensional image of the object's surface to an external electronic device.

[0098] Figure 7 This is a diagram illustrating examples of using a medical image integration system 10 according to various embodiments of the present disclosure.

[0099] Reference Figure 7 A doctor (D) can use a portable 3D imaging measurement device 100 to acquire a 3D image of a patient's (P) surface. For example, the doctor (D) can use the portable 3D imaging measurement device 100 to illuminate the surface of the patient (P) with patterned light. A pattern 710 can be formed on the surface of the patient (P) by the illuminated patterned light. The portable 3D imaging measurement device 100 can receive reflected light reflected from the patient (P) to generate a light field image of the patient (P). The light field image of the patient (P) can, for example, be an image combining multiple sub-images associated with the illuminated pattern 710. The portable 3D imaging measurement device 100 can use the light field image of the patient (P) to generate a 3D image of the surface of the patient (P). The portable 3D imaging measurement device 100 can transmit the generated 3D image of the surface of the patient (P) to an external electronic device 20.

[0100] According to various embodiments, the external electronic device 20 images a patterned image of at least a portion of the patterned surface of the marker 310 attached to the portable three-dimensional image measuring device 100 by capturing the image of at least a portion of the patterned surface using an imaging device. The external electronic device 20 can determine the position and orientation of the portable three-dimensional image measuring device 100 with the marker 310 attached based on the imaged pattern.

[0101] According to various embodiments, the external electronic device 20 can transform or align the coordinate system of a three-dimensional image of the surface of the patient P to the coordinate system of the external electronic device 20. For example, the external electronic device 20 can transform the coordinate system of the three-dimensional image of the surface of the patient P to the coordinate system of the external electronic device 20 based on the position and orientation of the portable three-dimensional imaging measurement device 100 determined by the marker 310.

[0102] According to various embodiments, the external electronic device 20 can transform or align the coordinate system of a medical image of patient P received from a medical device to the coordinate system of the external electronic device 20. According to various embodiments, the external electronic device 20 can perform image integration by unifying the coordinate systems of a three-dimensional image of a surface of patient P with the medical image of patient P.

[0103] Figure 8 This is a diagram illustrating the structure of a camera 130 according to various embodiments of the present disclosure.

[0104] Reference Figure 8 The camera 130 may include a condenser lens 137, a lens array 135, and an image sensor 131 arranged sequentially starting from the object 810. The camera 130 is an exemplary configuration for acquiring a light field image; however, different configurations may be used to acquire light field images.

[0105] According to various embodiments, the condenser lens 137 serves as a structure for focusing reflected light from the object 810. The condenser lens 137 can be a convex lens with a focal length, so that the reflected light from the object 810 is focused at a single point. When the condenser lens 137 is implemented using multiple lenses, etc., according to the well-known thin lens theory, the multiple lenses can be defined as a single thin lens. Therefore, the diameter, focal length, and center of the condenser lens 137 can be expressed as the diameter, focal length, and center of a single thin lens as defined above.

[0106] Lens array 135 according to various embodiments can disperse light entering through condenser lens 137 and focus it to multiple points formed at different locations. The lens array can be composed of multiple microlenses. For example, lens array 135 can be configured closer to condenser lens 137 than the focal length of condenser lens 137. Alternatively, lens array 135 can be configured further away from condenser lens 137 than the focal length of condenser lens 137.

[0107] According to various embodiments, the lens array 135 can be configured at a position corresponding to the focal length of the condenser lens 137. In the above case, the focal point of the light entering from the condenser lens 137 can be formed on one of the plurality of microlenses 135a. Furthermore, the image sensor 131 can be fixed at a position corresponding to the focal length of each microlens 135a included in the lens array 135.

[0108] Image sensor 131, according to various embodiments, can sense light passing through lens array 135. Furthermore, image sensor 131 can acquire a light field image including multiple sub-images corresponding to multiple points. Image sensor 131 can include at least one imaging element of any type to acquire an image of any object, and image sensor 131 can be composed of multiple pixels 131a.

[0109] Image sensor 131 according to various embodiments 106 can output a light field image, for example, having a photo aggregation file format, during a single capture. The photo aggregation file can include multiple sub-images of an object whose focal point is formed at a position corresponding to the focal point of multiple microlenses, thus having different object depths. In each sub-image, color and direction information of the light can be stored together according to the X and Y coordinates.

[0110] The sub-images according to various embodiments have different object depths from each other, but can capture the same object. The object seen in each sub-image can be substantially the same, but the positions of the clearly visible and blurred parts may differ. The clearly visible part can be the part that forms the focal point of the corresponding microlens 135a and has the object depth, while the blurred part can be the part excluding this.

[0111] According to various embodiments, a light field camera can determine the depth of a photographed object after it has been captured, and combine images having different depths. Therefore, the image sensor of the light field camera can have a post-processed, variable depth of view. Furthermore, the light field image generated by the light field camera can include multiple sub-images that store color and direction information of light together.

[0112] In another embodiment, the camera 130 can perform a refocusing process using multiple sub-images. During the refocusing process, the desired depth of the subject in the pixels of the light field image and the color information of the pixels corresponding to the light path and direction calculated therefrom can be combined to re-extract an image of the desired depth. This allows the generation of an image in which the illuminated pattern can be clearly identified.

[0113] Figure 9 This is a diagram illustrating the lens array 135 of a camera 130 according to various embodiments of the present disclosure.

[0114] According to various embodiments, the plurality of microlenses 135a included in the lens array 135 can be provided as N (N is a natural number greater than 1). That is, N can refer to multiple. For example, in the lens array 135, each row can be configured with i microlenses, and each column can be configured with j microlenses. Therefore, N microlenses can be composed of i*j rows and columns. For example, in order to form a denser light field, the lens array 135 can have a shape with approximately 1000*1000 microlenses arranged. The arrangement and number of microlenses can vary depending on various conditions of the condenser lens 137 and the microlenses (e.g., physical properties, shooting environment, resolution required for the sub-image, or number of pixels of the image sensor).

[0115] According to various embodiments, N microlenses can disperse light entering through the condenser lens 137 to N points. Figure 8 The image sensor 131 shown can be divided into N regions corresponding to N points formed by N microlenses. The focal points of each of the N microlenses can be distributed across the N regions of the image sensor 131.

[0116] According to various embodiments, when imaging N sub-images in N regions, the light field image can include N sub-images having different object depths from each other. Furthermore, the processor can select predetermined locations within the N images to form an image representing the object depth.

[0117] Figure 10 This is a diagram illustrating the process by which multiple sub-images included in a light field image acquired by camera 130 according to various embodiments of the present disclosure are formed at different object depths.

[0118] The camera 130 according to various embodiments may include a condenser lens 137, a lens array 135, and an image sensor 131. The first object 1010 may be configured to be closer to the condenser lens 137 than the second object 1020.

[0119] According to various embodiments, light emanating from the upper end of the first object 1010 can be focused by a condenser lens 137 onto a microlens 135c disposed below the lens array 135. The light emanating from the microlens 135c can reach region A1 disposed below the image sensor 131. Thus, the amount of light emanating from the upper end of the first object 1010 can be mainly distributed in the lower region A1, while the amount of light in other regions can be distributed in small amounts. That is, the appearance of the upper end of the first object 1010 can be clearly imaged onto pixels 131e, 131f, and 131g included in the lower region A1.

[0120] 117 According to various embodiments, light emanating from the upper end of the second object 1020 can be focused by the condenser lens 137 onto a microlens 135b in the middle portion of the lens array 135. The light emanating from the microlens 135b can reach region A2, which is disposed in the middle portion of the image sensor 131. Thus, the amount of light emanating from the upper end of the second object 1020 can be mainly distributed in the middle region A2, while the amount of light in other regions can be distributed in small amounts. That is, the appearance of the upper end of the second object 1020 can be clearly imaged onto pixels 131b, 131c, and 131d included in the middle region A2.

[0121] Because a small amount of light emitted from the second object 1020 is distributed in the lower region A1, the image of the second object 1020 may be imaged in a blurred state. Furthermore, because a small amount of light emitted from the first object 1010 is distributed in the middle region A2, the image of the first object 1010 may also be imaged in a blurred state. Therefore, the lower region A1 can output a sub-image showing the depth of the subject for the first object 1010, and the middle region A2 can output a sub-image showing the depth of the subject for the second object 1020.

[0122] As described above, when an object is photographed using a camera according to various embodiments, a light field image comprising multiple sub-images having different object depths from each other can be generated.

[0123] Figure 11 This is a diagram illustrating a light field image 1100 comprising multiple sub-images having different depths of the subject compared to each other, according to various embodiments of the present disclosure.

[0124] Reference Figure 11According to various embodiments, the light field image 1100 can be output as a photo aggregate file comprising multiple sub-images forming the depth of the subject in regions C1, C2, C3, C4 located at different positions on the image sensor 131. The positions of the multiple regions C1, C2, C3, C4 within the sub-images may differ from each other; depending on the situation, at least two regions may be in the same position. The photo aggregate file can be a simple aggregation format that collects multiple physically separated sub-images. Alternatively, the photo aggregate file can be a format that combines multiple sub-images into one file with a new file extension. According to various embodiments, each sub-image may include light color and direction information to have different depths of the subject than the others. Figure 11 The arrows shown in the image indicate the direction in which the distance increases to form the depth of the subject.

[0125] While the method has been described through specific embodiments, it can also be implemented as computer-readable code on a computer-readable recording medium. A computer-readable recording medium includes all types of recording devices that store data readable by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage devices, etc. Furthermore, the computer-readable recording medium can be distributed across computer systems connected via a network, thereby storing and executing computer-readable code in a distributed manner. Moreover, functional programs, code, and code snippets for implementing the above embodiments can be readily deduced by a programmer skilled in the art to which this disclosure pertains.

[0126] The technical concept of this disclosure has been illustrated above with the aid of some embodiments and examples shown in the accompanying drawings. However, it should be understood that various substitutions, modifications, and alterations can be made within the scope of the technical concept and scope of this disclosure that can be understood by those skilled in the art. Furthermore, such substitutions, modifications, and alterations are considered to fall within the scope of the appended claims.

Claims

1. A portable three-dimensional image measuring device, characterized in that, include: Light source, which outputs patterned light; A light field camera receives reflected light generated by the reflection of patterned light from an object, thereby generating a light field image of the object reflecting the pattern, wherein the light field image of the object is an image composed of multiple sub-images including color and direction information of the reflected light; and A light path control element that reflects the patterned light so that the patterned light output from the light source illuminates the object, and transmits the reflected light so that the reflected light reflected from the object reaches the light field camera; Communication circuits; and processor, The light path of the patterned light emitted from the light source and illuminating the object, and the light path of the reflected light reflected from the object and reaching the light field camera, are coaxial and overlap in the interval between the light path control element and the object. The processor, Using the multiple sub-images contained in the light field image of the object acquired by the light field camera, the intensity of reflected light at multiple points constituting the surface of the object is measured. Phase data is generated based on the measured intensity of the reflected light. The height of each of the multiple points is calculated based on the generated phase data to generate a three-dimensional image of the surface of the object. The communication circuit transmits a three-dimensional image of the surface of the object to an external electronic device.

2. The portable three-dimensional image measuring device according to claim 1, characterized in that, The three-dimensional image of the surface of the object is used to integrate with the medical image of the object.

3. The portable three-dimensional image measuring device according to claim 1, characterized in that, Also includes: The marker is set in a manner that allows it to move from a predetermined position of the portable 3D image measuring device. The processor transmits information representing the displacement of the marker from the predetermined position to the external electronic device.

4. The portable three-dimensional image measuring device according to claim 1, characterized in that, The light source and the light field camera are configured in mutually perpendicular directions based on the light path control elements.

5. The portable three-dimensional image measuring device according to claim 1, characterized in that, The light field camera includes: Lens array, wherein multiple microlenses are arranged in the lens array; and An image sensor that captures the reflected light passing through the lens array.

6. The portable three-dimensional image measuring device according to claim 1, characterized in that, The light source includes: Patterned section, wherein multiple patterns are formed; and LED, which illuminates the patterned area.

7. The portable three-dimensional image measuring device according to claim 6, characterized in that, The light output by the LED is infrared light.

8. The portable three-dimensional image measuring device according to claim 1, characterized in that, The optical path control element is a semi-transparent mirror.

9. The portable three-dimensional image measuring device according to claim 1, characterized in that, Also includes: A first housing, wherein the light source, the light field camera, and the light path control elements are disposed inside the first housing; and A second housing is attached to the first housing and has an opening to allow the patterned light emitted from the light source to illuminate the object.

10. The portable three-dimensional image measuring device according to claim 9, characterized in that, The second housing is rotatably coupled relative to the first housing.

11. A three-dimensional image measurement method, as a three-dimensional image measurement method for a portable three-dimensional image measurement device, characterized in that, include: The operation of illuminating the object with patterned light output from the light source by controlling the light path element; and The operation of a light field camera receiving reflected light generated by the pattern light reflected from the object through the light path control element, and generating a light field image of the object reflecting the pattern, wherein the light field image of the object is an image composed of multiple sub-images including the color information and direction information of the reflected light; The operation involves using the multiple sub-images contained in the light field image of the object to measure the intensity of reflected light at multiple points constituting the surface of the object, generating phase data based on the measured intensity of reflected light, and then calculating the height of each of the multiple points based on the generated phase data to generate a three-dimensional image of the surface of the object; and The operation of transmitting a three-dimensional image of the surface of the object to an external electronic device. The light path of the patterned light illuminating the object from the light source and the light path of the reflected light reaching the light field camera from the object can be coaxial and overlapped in the interval between the light path control element and the object.

12. The three-dimensional image measurement method according to claim 11, characterized in that, The three-dimensional image of the surface of the object is used to integrate with the medical image of the object.

Images