Tof camera device and driving method thereof

Abstract

A time-of-flight (TOF) camera device and an operating method thereof are provided. The time-of-flight (TOF) camera device includes a pulse generator configured to generate a pulse signal, a light module configured to emit output light to at least one object in response to the pulse signal, a three-dimensional (3D) sensor configured to receive reflected light when the output light is reflected by the at least one object within a first frame, a distance calculator configured to receive an output of the 3D sensor and generate a distance data signal, and a light density control device configured to receive the distance data signal from the distance calculator and output a light density control signal. The light density control signal can adjust a size of an opening in the light module to change a projection area of the output light to the at least one object.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to Korean Patent Application No. 10-2020-0172917, filed with the Korean Intellectual Property Office on December 11, 2020, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure relates to a time-of-flight (TOF) camera device and a method for driving the same. Background Technology

[0004] Time-of-flight (TOF) cameras detect the distance between an object and the camera by measuring the phase delay between light emitted from the camera and light reflected from the object. For example, some traditional TOF cameras modulate a light source at a predetermined frequency, emit the modulated light onto the scene, and measure the phase shift of the light reflected back to the camera to generate a depth map with a resolution determined by the camera's design. These types of cameras are widely used in topographic surveying and object pose control. Summary of the Invention

[0005] Embodiments of this disclosure provide a time-of-flight (TOF) camera device configured to accurately identify the location of an object by controlling light density.

[0006] Embodiments of this disclosure also provide a method for driving a TOF camera device configured to accurately identify the location of an object by controlling light density.

[0007] The aspects of this disclosure are not necessarily limited to those set forth herein, and other aspects of this disclosure will be apparent to those skilled in the art from the following description.

[0008] According to one aspect of this disclosure, a time-of-flight (TOF) camera device includes: a pulse generator configured to generate a pulse signal; an optical module configured to emit output light toward at least one object in response to the pulse signal; a three-dimensional (3D) sensor configured to receive reflected light when the output light is reflected by at least one object in a first frame; a distance calculator configured to receive the output of the 3D sensor and generate a distance data signal; and a light density control device configured to receive the distance data signal from the distance calculator and output a light density control signal, wherein the light density control signal controls the size of an opening in the optical module for emitting the output light to determine the size of a projection area on at least one object, and wherein the optical module emits output light toward at least one object in the first frame, wherein the output light has a light density corresponding to the light density control signal.

[0009] According to one aspect of this disclosure, a method for driving a time-of-flight (TOF) camera device includes: generating a pulse signal; emitting output light toward at least one object in response to the pulse signal; receiving reflected light when the output light is reflected by the at least one object in a first frame; generating a distance data signal from the received reflected light; and receiving the distance data signal and outputting a light density control signal that allows the size of the area in the at least one object where the output light was emitted to be determined based on the distance data signal.

[0010] According to one aspect of this disclosure, a method for driving a time-of-flight (TOF) camera device includes: in a first frame, emitting a first output light from an optical module toward a first region of a first object; changing a region of the first object in which the first output light was emitted from the first region to a second region based on a first reflected light reflected by the first object in the first frame; in a second frame following the first frame, emitting a second output light from the optical module toward a third region of a second object; and changing a region of the second object in which the second output light was emitted from the third region to a fourth region based on a second reflected light reflected by the second object in the second frame, wherein the size of the second region is different from the size of the fourth region. Attached Figure Description

[0011] The above and other aspects and features of this disclosure will become clearer from the detailed description of exemplary embodiments of the invention with reference to the accompanying drawings, in which:

[0012] Figure 1 This is a block diagram showing a time-of-flight (TOF) camera device.

[0013] Figure 2 This is a diagram showing the optical region of an object according to the aperture size of an optical density controller (LDC) according to some embodiments.

[0014] Figure 3 This is a diagram showing the optical region of an object according to the aperture size of the LDC according to some embodiments.

[0015] Figure 4 This is a diagram showing the optical region of an object according to the aperture size of the LDC according to some embodiments.

[0016] Figure 5 An example process for generating a 3D depth map from a 3D sensor is shown according to some embodiments.

[0017] Figure 6 An example process for generating a 3D depth map from a 3D sensor is shown according to some embodiments.

[0018] Figure 7 This is a flowchart of the method for driving a TOF camera device.

[0019] Figure 8 This is a diagram showing the optical regions of a first object and a second object according to the aperture size of an LDC, based on some embodiments.

[0020] Figure 9 Example depth maps of a first object and a second object according to some embodiments are shown.

[0021] Figure 10 It shows the selection Figure 8 The diagram shows the situation of the first object.

[0022] Figure 11 It shows according to Figure 10 A diagram of the optical regions of the first object and the second object.

[0023] Figure 12 It shows the selection Figure 8 The diagram shows the situation of the second object.

[0024] Figure 13 It shows according to Figure 12 A diagram of the optical regions of the first object and the second object.

[0025] Figure 14 This is a flowchart of a method for driving a TOF camera device according to some embodiments.

[0026] Figure 15 This is a diagram showing the optical regions of a first and a second object according to the aperture size of the LDC, based on some embodiments.

[0027] Figure 16 This is a diagram showing the state of emitting output light to the first object within the first frame.

[0028] Figure 17 It shows according to Figure 16 A diagram of the optical region of the first object.

[0029] Figure 18 This is a diagram showing the state of emitting output light to the second object within the second frame.

[0030] Figure 19 It shows according to Figure 18 The diagram of the optical region of the second object.

[0031] Figure 20 It is shown that, according to some embodiments, it includes Figure 1 The diagram shows the computer system of the TOF camera device.

[0032] Figure 21 It is shown that, according to some embodiments, it includes Figure 1The diagram shows the computer system of the TOF camera device.

[0033] Figure 22 It is shown that, according to some embodiments, it includes Figure 1 The diagram shows the computer system of the TOF camera device. Detailed Implementation

[0034] In the following description, embodiments of the present disclosure will be described with reference to the accompanying drawings. Like reference numerals in the drawings may denote like elements, and to the extent that description of an element has been omitted, it will be understood that the element is at least similar to a corresponding element described elsewhere in the specification.

[0035] In traditional phase-detection Time-of-Flight (TOF) cameras, the accuracy depends on the modulation frequency, dynamic range, sensitivity, and other characteristics of the emitted light. Furthermore, if the phase shift relative to a distant object is large enough, it may be impossible to determine the distance to that object because the measurement range is determined by modulo operations on the carrier wavelength (e.g., the phase shift may be large enough that its displacement exceeds the entire wavelength, resulting in uninterpretable results). In traditional phase-detection TOF cameras with a fixed modulation frequency, accurate results beyond a fixed distance may not be guaranteed.

[0036] The TOF camera devices and systems described herein can accurately track the distances of multiple objects over time. Specifically, a TOF camera device according to embodiments of this disclosure may include an optical density control device that can be configured to determine the distance to one or more objects and adjust the projection angle of light to define a corresponding optical area on the one or more objects, thereby tracking one or more objects with higher accuracy over a wide distance range.

[0037] Figure 1 This is a block diagram showing a time-of-flight (TOF) camera device. Figure 2 This is a diagram showing the optical region of an object according to the aperture size of an optical density controller (LDC) according to some embodiments. Figure 3 This is a diagram showing the optical region of an object according to the aperture size of the LDC according to some embodiments. Figure 4 This is a diagram showing the optical region of an object according to the aperture size of the LDC according to some embodiments.

[0038] Reference Figure 1 The TOF camera device 100 may include an optical module 110, an optical density control device 120, a distance calculator 130, a three-dimensional (3D) sensor 140, a storage device 150, an image signal processor (ISP) 160, and a pulse generator 170.

[0039] The optical module 110 may include a light source 111 and an LDC 112. The light source 111 may emit output light OL toward the object OJ in response to a pulse signal P generated by the pulse generator 170.

[0040] LDC 112 can control the opening used to emit output light OL in response to the optical density control signal LCON generated by the optical density control device 120. For example, LDC 112 can control the size of the opening used to emit output light OL in response to the optical density control signal LCON, based on the position of the object OJ.

[0041] Reference Figure 2 The optical module 110 can emit output light OL towards the object OJ. In this case, the gap of LDC 112 can be a first gap d1. In addition, the optical module 110 can project the output light OL onto the object OJ in the object area; this projection area can be referred to as the optical area, and can be the first area S11.

[0042] Reference Figure 3 This can change the optical density control signal LCON. Figure 3 In the example shown, the distance between the optical module 110 and the object OJ can be less than [missing information]. Figure 2 The distance shown is an example of this. Furthermore, the LDC 112 can increase its aperture size in response to the optical density control signal LCON. The LDC 112 can control the size of the aperture used to emit the output light OL in response to the optical density control signal LCON.

[0043] exist Figure 3 In the context, the gap of LDC 112 can be a second gap d2, and the size of the second gap d2 can be larger than the first gap d1 (see...). Figure 2 ).

[0044] As a result, the optical area of ​​the output light OL emitted onto the object OJ can be increased. For example, the size of the optical area of ​​the output light OL emitted onto the object OJ can be increased from the first region S11 (see... Figure 2 ) is added to the second region S12.

[0045] Conversely, refer to Figure 4 The distance between the optical module 110 and the object OJ can be greater than, for example, Figure 2 The distance between the optical module 110 and the object OJ is shown, and the size of the opening of LDC 112 can be reduced in response to the optical density control signal LCON.

[0046] exist Figure 4In this context, the gap in LDC 112 can be a third gap d3. The size of the third gap d3 can be smaller than the size of the first gap d1. The size of the third gap d3 can be smaller than the size of the second gap d2.

[0047] As a result, the optical region of the output light OL emitted onto the object OJ can be reduced. For example, the size of the optical region of the output light OL emitted onto the object OJ can be reduced from the first region S11 (see... Figure 2 It decreases to the third region S13.

[0048] As described above, the size of the optical region of the output light OL emitted onto the object OJ can be changed according to the aperture size of LDC 112.

[0049] Further reference Figure 1 The output light OL can have a constant frequency. For example, a light source in the infrared wavelength range can be used in the optical module 110, but the embodiments are not necessarily limited to this.

[0050] The output light OL emitted to the object OJ can be reflected and received by the 3D sensor 140. The phase of the reflected light RL reflected by the object OJ can be changed with reference to the phase of the output light OL.

[0051] For example, when the phase of the reflected light RL is compared with the phase of the output light OL emitted by the light source 111, the phase of the reflected light RL can be changed according to the distance to the object OJ.

[0052] The 3D sensor 140 can receive the reflected light RL in the first frame, which is the output light OL reflected by the object OJ. The 3D sensor 140 can store phase difference information about the reflected light RL in the storage device 150.

[0053] The 3D sensor 140 can generate timing information of the received reflected light RL based on the phase difference information between the output light OL of the optical module 110 and the reflected light RL reflected by the object OJ. Based on the timing information, the 3D sensor 140 can generate a 3D depth map DM of the object OJ.

[0054] The following will refer to Figure 5 and Figure 6 Describes a 3D sensor 140 used to generate a 3D depth map DM.

[0055] The distance calculator 130 can receive the output of the 3D sensor 140 to generate a distance data signal DCON. In this case, the output of the 3D sensor 140 can be, for example, a 3D depth map DM.

[0056] The distance data signal DCON can be a signal based at least in part on the time information between the output light OL and the reflected light RL.

[0057] The optical density control device 120 can receive a distance data signal DCON from the distance calculator 130 and output an optical density control signal LCON in response. The optical density control signal LCON can determine the size of the area from which the output light OL is emitted by the optical module 110. For example, the optical density control signal LCON can determine the size of the gap in LDC 112, thereby projecting the output light OL onto an object OJ in the area, wherein the area has a size determined by the gap in LDC 112.

[0058] The optical density control device 120 can receive the distance data signal DCON to generate an optical density control signal LCON. The optical density control signal LCON allows adjustment of the size of the aperture used to emit the output light OL of the LDC 112.

[0059] Storage device 150 can store information received from 3D sensor 140. Storage device 150 can send 3D depth map DM, image, and phase difference information generated by 3D sensor 140 to ISP 160.

[0060] The ISP 160 can use phase difference information to calculate the distance between the object OJ and the TOF camera device 100. The ISP 160 can then send the calculated information or image to the display device 200, which can then display the image.

[0061] Figure 5 An example process for generating a 3D depth map from a 3D sensor is shown according to some embodiments. Figure 6 An example process for generating a 3D depth map from a 3D sensor is shown according to some embodiments.

[0062] Reference Figure 5 The 3D sensor 140 can receive and combine the light rays of reflected light RL reflected by the object to generate a 3D depth map DM.

[0063] For example, 3D sensor 140 can receive rays of reflected light from an object OJ within the first frame. 3D sensor 140 can combine the rays of reflected light to generate a 3D depth map DM.

[0064] Reference Figure 6 ,and Figure 5 Unlike other sensors, the 3D sensor 140 can generate a 3D depth map DM, in which rays of reflected light RL from an object are continuously combined.

[0065] For example, the 3D sensor 140 can receive multiple rays of reflected light from the object OJ within a first frame. The 3D sensor 140 can continuously combine the multiple rays of reflected light to generate single rays SRL1 to SRLm of reflected light in order to generate a 3D depth map DM.

[0066] Figure 7 This is a flowchart illustrating a method for driving a TOF camera device.

[0067] Reference Figure 7 A pulse signal is generated (S100).

[0068] Reference Figure 1 The pulse generator 170 can generate a pulse signal P. Hereinafter, the structure of the TOF camera device 100 described above will be used to describe a method of driving a TOF camera device according to embodiments of the present disclosure. However, the embodiments are not necessarily limited thereto.

[0069] Next, in response to the pulse signal, output light is emitted toward the object (S110).

[0070] Reference Figure 1 The optical module 110 can emit output light OL towards the object OJ in response to the pulse signal P generated by the pulse generator 170. For example, the light source 111 of the optical module 110 can emit output light OL towards the object OJ in response to the pulse signal P.

[0071] Next, the reflected light from the object is received by the 3D sensor 140 (S120).

[0072] Reference Figure 1 The 3D sensor 140 can receive reflected light RL reflected by the object OJ.

[0073] Next, a 3D depth map is generated using the received reflected light (S130).

[0074] Reference Figure 1 The 3D sensor 140 can generate time information of the received reflected light RL based on the phase difference information between the output light OL of the optical module 110 and the reflected light RL reflected by the object OJ. The 3D sensor 140 can then generate a 3D depth map DM of the object OJ based on the time information.

[0075] Reference Figure 6 When the 3D sensor 140 generates a 3D depth map DM, multiple rays of reflected light RL reflected by the object OJ can be continuously combined to generate single rays SRL1 to SRLm of the reflected light in order to generate the 3D depth map DM. For example, the 3D depth map DM can be generated using single rays SRL1 to SRLm of the reflected light for each of frames F1 to Fn.

[0076] Reference Figure 5 When the 3D sensor 140 generates a 3D depth map DM, the rays of reflected light reflected by the object OJ can be combined to generate the 3D depth map DM.

[0077] Next, distance data signals are generated based on the generated 3D depth map (S140).

[0078] Reference Figure 1 The distance calculator 130 can generate a distance data signal DCON. The distance calculator 130 can receive a 3D depth map DM output from the 3D sensor 140 to generate the distance data signal DCON.

[0079] Finally, an optical density control signal can be output, which allows the size of the area on the object to which the output light was emitted to be determined using distance data signals (S150).

[0080] Reference Figure 1 The optical density control device 120 can generate an optical density control signal LCON. The optical density control device 120 can determine the size of the area on the object where the output light OL is emitted by the optical module 110. The optical density control device 120 can receive a distance data signal DCON and output the optical density control signal LCON based on the distance data signal DCON.

[0081] Therefore, the LDC 112 can adjust the size of the aperture through which the output light passes in response to the optical density control signal DCON. It can emit output light OL corresponding to the optical density control signal DCON toward the object OJ.

[0082] exist Figures 1 to 7 The previous section described the case where the first frame contains one object (OJ), and the following section will describe an example where the first frame contains two objects. In the following text, details related to or briefly describing the OJ with one object will be omitted or only briefly described. Figures 1 to 7 The components are the same or similar parts, and the differences between them will be described in detail.

[0083] Figure 8 This is a diagram showing the optical regions of a first object and a second object according to the aperture size of an LDC, based on some embodiments. Figure 9 Example depth maps of a first object and a second object according to some embodiments are shown. Figure 10 It shows the selection Figure 8 The diagram shows the situation of the first object. Figure 11 It shows according to Figure 10 A diagram of the optical regions of the first object and the second object. Figure 12 It shows the selection Figure 8The diagram shows the situation of the second object. 13 is an illustration based on... Figure 12 A diagram of the optical regions of the first object and the second object.

[0084] Reference Figure 8 The first object OJ1 can be separated from the optical module 110 by a first distance D1, and the second object OJ2 can be separated from the optical module 110 by a second distance D2, the second distance D2 being greater than the first distance D1.

[0085] For the first frame, the light source 111 can emit output light OL to the first object OJ1 and the second object OJ2 in response to the pulse signal P. For example, the output light OL emitted by the optical module 110 can be emitted to both the first object OJ1 and the second object OJ2.

[0086] In this case, the area on the first object OJ1 where the output light OL is emitted can be the first area S1, and the area on the second object OJ2 where the output light OL is emitted can be the second area S2.

[0087] When the output light OL is reflected by the first object OJ1, the 3D sensor 140 can receive the first reflected light RL1, and when the output light OL is reflected by the second object OJ2 in the first frame, the 3D sensor 140 can receive the second reflected light RL2. The 3D sensor 140 can combine the depth of the first object OJ1 and the depth of the second object OJ2 to generate a 3D depth map OJS.

[0088] Reference Figure 9 Since the first object OJ1 is located at a first distance D1, which is relatively closer to the optical module 110 than the second object OJ2, a depth map with floodlight depth can be generated. Since the second object OJ is located at a second distance D2, which is relatively farther from the optical module 110 than the first object OJ, a depth map with spot depth can be generated. When the depths of the first object OJ1 and the second object OJ are combined, a more accurate depth map DMS can be generated than a depth map with the spot depth of the second object.

[0089] Refer again Figure 8 The distance calculator 130 can receive a 3D depth map OJS, in which a first object OJ1 and a second object OJ2 are combined to generate a distance data signal DCON. The optical density control device 120 can receive the distance data signal DCON and output an optical density control signal LCON based on the distance data signal DCON.

[0090] For example, when there is a first object OJ1 and a second object OJ2, the optical density control device 120 can select either the first object OJ1 or the second object OJ2, and output an optical density control signal LCON based on the selected object in the first frame.

[0091] In the following text, reference will be made to Figure 10 and Figure 11 This describes an example of an optical density control device 120 selecting a first object OJ1 and outputting a first optical density control signal LCON1 based on the first object OJ1.

[0092] Therefore, an example will be described in which the optical regions emitted by the output light OL onto the first object OJ1 and the second object OJ2 are changed according to the first optical density control signal LCON1.

[0093] Reference Figure 10 and Figure 11 The distance calculator 130 can receive a first 3D depth map DM1 generated by the 3D sensor 140 to generate a first distance data signal DCON1. ​​Since the first object OJ1 is separated from the optical module 110 by a first distance D1, the distance calculator 130 can generate the first distance data signal DCON1.

[0094] The optical density control device 120 can receive a first distance data signal DCON1 generated by the distance calculator 130 and generate a first optical density control signal LCON1. ​​For example, the optical density control device 120 can generate a first optical density control signal LCON1 to correspond to the first distance data signal DCON1.

[0095] The first optical density control signal LCON1 can be used to determine the regions on the first object OJ1 and the second object OJ2 where the output light OL is emitted by the optical module 110, based on the first distance data signal DCON1.

[0096] LDC 112 can adjust the size of the aperture used to emit output light OL in response to the first optical density control signal LCON1.

[0097] The opening gap of LDC112 caused by the first optical density control signal LCON1 can be a second gap d5. The size of the second gap d5 can be larger than the first gap d4.

[0098] As described above, since the opening gap of LDC 112 becomes the second gap d5, the optical region where the output light OL is emitted onto the first object OJ1 can be changed to the third region S3. For example, the optical region where the output light OL is emitted onto the first object OJ1 can be changed from the first region S1 to the third region S3 according to the first optical density control signal LCON1. ​​The size of the third region S3 can be larger than the size of the first region S1.

[0099] Similarly, since the opening gap of LDC 112 becomes the second gap d5, the optical region on which the output light OL is emitted onto the second object OJ2 can be changed to the fourth region S4. For example, the optical region on which the output light OL is emitted onto the second object OJ2 can be changed from the second region S2 to the fourth region S4 according to the first optical density control signal LCON1. ​​The size of the fourth region S4 can be larger than the size of the second region S2.

[0100] Next, we will refer to Figure 12 and Figure 13 This describes an example of an optical density control device 120 selecting a second object OJ2 and outputting a second optical density control signal LCON2 based on the second object OJ2.

[0101] An example will now be described of changing the optical regions emitted by the output light OL onto the first object OJ1 and the second object OJ2 according to the second optical density control signal LCON2.

[0102] The distance calculator 130 can receive a second 3D depth map DM2 generated by the 3D sensor 140 to generate a second distance data signal DCON2. Since the second object OJ2 is separated from the optical module 110 by a second distance D2 greater than the first distance D1, the distance calculator 130 can generate the second distance data signal DCON2.

[0103] The optical density control device 120 can receive the second distance data signal DCON2 generated by the distance calculator 130 to generate a second optical density control signal LCON2. For example, the optical density control device 120 can generate a second optical density control signal LCON2 corresponding to the second distance data signal DCON2.

[0104] The second optical density control signal LCON2 can be used to determine the size of the area on the second object OJ2 and the first object OJ1 where the output light OL is emitted by the optical module 110, based on the second distance data signal DCON2.

[0105] LDC 112 can adjust the size of the aperture used to emit output light OL in response to the second optical density control signal LCON2.

[0106] In response to the second optical density control signal LCON2, the opening gap of LDC 112 can be changed to a third gap d6. The size of the third gap d6 can be smaller than the size of the first gap d4 (see...). Figure 10 ).

[0107] Therefore, since the size of the opening of LDC 112 becomes the third gap d6, the optical region where the output light OL is emitted onto the second object OJ2 can be changed to the sixth region S6. According to the second optical density control signal LCON2, the optical region where the output light OL is emitted onto the second object OJ2 can be changed from the second region S2 to the sixth region S6. The sixth region S6 can be smaller than the second region S2. In other words, the second region S2 can be larger than the sixth region S6.

[0108] Similarly, since the size of the opening of LDC 112 becomes the third gap d6, the optical region on which the output light OL is emitted onto the first object OJ1 can be changed to the fifth region S5. According to the second optical density control signal LCON2, the optical region on which the output light OL is emitted onto the first object OJ1 can be changed from the first region S1 to the fifth region S5. The fifth region S5 can be smaller than the first region S1. In other words, the first region S1 can be larger than the fifth region S5.

[0109] Figure 14 This is a flowchart illustrating a method for driving a TOF camera device according to some embodiments. Figure 15 This is a diagram showing the optical regions of a first and a second object according to the aperture size of the LDC, based on some embodiments. Figure 16 This is a diagram showing the state of emitting output light to the first object within the first frame. Figure 17 It shows according to Figure 16 A diagram of the optical region of the first object. Figure 18 This is a diagram showing the state of emitting output light to the second object within the second frame. Figure 19 It shows according to Figure 18 The diagram of the optical region of the second object.

[0110] Reference Figure 14 For the first frame, the optical module emits the first output light to the first object (S200).

[0111] Reference Figure 15 For the first frame, the first object OJ3 can be set at a position separated from the optical module 110 by a first distance D1.

[0112] The light source 111 of the optical module 110 can emit a first output light OL3 toward the first object OJ3. In this case, the opening gap of the LDC 112 used to emit the first output light OL3 can be a first gap d7. The area on the first object OJ3 where the first output light OL3 is emitted can be a first area S6.

[0113] Next, for the first frame, based on the first reflected light reflected by the first object, the area on the first object where the first output light was emitted is changed from the first region to the second region (S210).

[0114] Reference Figure 16 and Figure 17 The 3D sensor 140 can receive a first reflected light RL3 reflected by the first object OJ3. The 3D sensor 140 can generate a third 3D depth map DM3 based on the first reflected light RL3. (This is based on a reference...) Figure 5 and Figure 6 The third 3D depth map DM3 generated by the 3D sensor 140 is described, so its specific description will be omitted.

[0115] The distance calculator 130 can receive a third 3D depth map DM3 generated by the 3D sensor 140 to generate a third distance data signal DCON3. The optical density control device 120 can receive the third distance data signal DCON3 to output a third optical density control signal LCON3.

[0116] The third optical density control signal LCON3 allows the determination of the size of the area on the first object OJ3 where the first output light OL3 was emitted by the optical module 110.

[0117] The aperture gap of the LDC 112 can be adjusted in response to the third optical density control signal LCON3. For example, the aperture gap of the LDC 112 can be changed from a first gap d7 to a second gap d8. The size of the first gap d7 can be smaller than the size of the second gap d8. The size of the second gap d8 can be larger than the size of the first gap d7.

[0118] Because the opening gap of LDC 112 is changed, the area on the first object OJ3 where the first output light OL3 is emitted can be changed from the first region S6 to the second region S8. The size of the first region S6 can be smaller than the size of the second region S8.

[0119] Next, refer to Figure 14 For the second frame after the first frame, the optical module emits a second output light to the third region of the second object (S220).

[0120] Reference Figure 15 and Figure 18 For the second frame, the second object OJ4 can be positioned at a distance D2 from the optical module 110. The second distance D2 can be different from the first distance D1. For example, the second distance D2 can be greater than the first distance D1.

[0121] The light source 111 of the optical module 110 can emit a second output light OL4 toward the second object OJ4. In this case, the opening gap of the LDC 112 used to emit the second output light OL4 can be a first gap d7. The area on the second object OJ4 where the second output light OL4 is emitted can be a third area S9.

[0122] Finally, for the second frame, based on the second reflected light reflected by the second object, the area on the second object where the second output light was emitted is changed from the third area to the fourth area (S230).

[0123] Reference Figure 18 and Figure 19 For the second frame, the 3D sensor 140 can receive the second reflected light RL4 reflected by the second object OJ4. The 3D sensor 140 can generate a fourth 3D depth map DM4 based on the second reflected light RL4. Since already referenced... Figure 5 and Figure 6 The third 3D depth map DM3 generated by 3D sensor 14 is described, so its specific description will be omitted.

[0124] The distance calculator 130 can receive a fourth 3D depth map DM4 generated by the 3D sensor 140 to generate a fourth distance data signal DCON4. The optical density control device 120 can receive the fourth distance data signal DCON4 to output a fourth optical density control signal LCON4.

[0125] The fourth optical density control signal LCON4 allows the determination of the size of the area on the second object OJ4 where the second output light OL4 is emitted by the optical module 110.

[0126] The aperture gap of the LDC 112 can be adjusted in response to the fourth optical density control signal LCON4. For example, the aperture gap of the LDC 112 can be changed from a first gap d7 to a second gap d9. The size of the first gap d7 can be larger than the size of the second gap d9.

[0127] Because the opening gap of LDC 112 has been changed, the region on the second object OJ4 where the second output light OL4 is emitted can be changed from the third region S9 to the fourth region S10. The size of the fourth region S10 can be smaller than the size of the third region S9.

[0128] Therefore, the size of the second region S8 emitted by the second output light OL4 onto the first object OJ3 in the first frame and the size of the fourth region S10 emitted by the second output light OL4 onto the second object OJ4 in the second frame can be different. The size of the fourth region S10 can be smaller than the size of the second region S8.

[0129] Figure 20 It is shown that, according to some embodiments, it includes Figure 1 The diagram shows the computer system of the TOF camera device.

[0130] Reference Figure 20The computer system 300 can be implemented as a smartphone, a personal digital assistant (PDA), a portable multimedia player (PMP), a Moving Picture Experts Group (MPEG)-3 third audio layer (MP3) player, an MPEG-4 part fourteen (MP4) player, etc.

[0131] Computer system 300 may include storage device 301, application processor (AP) 302 including a storage controller configured to control storage device 301, wireless transceiver 303, antenna 304, input device 305, and display device 306.

[0132] The wireless transceiver 303 can transmit or receive wireless signals via the antenna 304. For example, the wireless transceiver 303 can convert the wireless signals received through the antenna 304 into signals that can be processed by the AP 302.

[0133] Therefore, AP 302 can process the signal output from wireless transceiver 303 and send the processed signal to display device 306. Wireless transceiver 303 can convert the signal output from AP 302 into a wireless signal and output the converted wireless signal to an external device through antenna 304.

[0134] Input device 305 may be a device for inputting control signals for controlling the operation of AP 302 or data to be processed by AP 302. In some embodiments, input device 305 may be implemented as a pointing device such as a touchpad or computer mouse, a keypad, or a keyboard.

[0135] Additionally, the computer system 300 may include a TOF camera device 307 for measuring the distance to an object and an image sensor 308 for capturing still or moving images. The AP 302 can send the still or moving images received from the image sensor 308 and the distance information to the object to the display device 306.

[0136] For example, Figure 1 The TOF camera device 100 shown can be implemented as a TOF camera device 307.

[0137] Figure 21 It is shown that, according to some embodiments, it includes Figure 1 The diagram shows the computer system of the TOF camera device.

[0138] Reference Figure 21 The computer system 400 can be implemented as a personal computer (PC), a network server, a tablet PC, a netbook, or an e-reader.

[0139] Computer system 400 may include storage device 401, AP 402 including a storage controller capable of controlling data processing operations of storage device 401, input device 405, and display device 406. For example, computer system 400 may be similar to computer system 300, but may not include a transceiver for radio communication.

[0140] AP 402 can display data stored in storage device 401 via display device 406 based on data input via input device 405. For example, input device 405 can be implemented as a pointing device such as a touchpad or computer mouse, a keypad, or a keyboard. AP 402 can control the overall operation of computer system 400.

[0141] Additionally, the computer system 400 may include a TOF camera device 407 for measuring the distance to an object and an image sensor 408 for capturing still or moving images. The AP 402 may send the still or moving images received from the image sensor 408 and the distance information to the object to the display device 406.

[0142] For example, the TOF camera device 407 can be implemented as Figure 1 The TOF camera device 100 shown is shown.

[0143] Figure 22 It is shown that, according to some embodiments, it includes Figure 1 The diagram shows the computer system of the TOF camera device.

[0144] Reference Figure 22 The computer system 500 can be implemented as an image processing device, such as a digital camera or mobile phone, smartphone or tablet computer with a digital camera attached.

[0145] Computer system 500 may include storage device 501, an AP 502 including a storage controller capable of controlling data processing operations (e.g., write or read operations) of storage device 501, input device 505, image sensor 508, display device 506, and TOF camera device 507. For example, computer system 500 may be similar to computer system 400; however, computer system 500 may have a display device integrated into the system, rather than necessarily as an external component.

[0146] Image sensor 508 converts optical images into digital signals and sends the converted digital signals to AP 502. Under the control of AP 502, the converted digital signals can be displayed via display device 506 or stored in storage device 501.

[0147] The TOF camera device 507 can measure the distance to an object. The AP 502 can send distance information to the display device 506. In addition, the AP 502 can send image data stored in the storage device 501 to the display device 506.

[0148] For example, the TOF camera device 507 can be implemented as Figure 1 The TOF camera device 100 shown is shown.

[0149] As is common in the field of this invention, embodiments are described and illustrated in the accompanying drawings in terms of functional blocks, units, and / or modules. Those skilled in the art will understand that these blocks, units, and / or modules are physically implemented by electronic (or optical) circuitry such as logic circuits, discrete components, microprocessors, hardwired circuitry, memory elements, wiring connections, etc., which can be formed using semiconductor-based manufacturing techniques or other manufacturing techniques. Where blocks, units, and / or modules are implemented by microprocessors, etc., they can be programmed using software (e.g., microcode) to perform the various functions discussed herein, and may optionally be driven by firmware and / or software. Alternatively, each block, unit, and / or module can be implemented by dedicated hardware or as a combination of dedicated hardware performing some functions and processors performing other functions (e.g., one or more programmed microprocessors and associated circuitry).

[0150] As described above, the TOF camera devices and systems described herein can accurately track the distances of multiple objects over time. Specifically, a TOF camera device according to embodiments of this disclosure may include an optical density control device that can be configured to determine the distance to one or more objects and adjust the projection angle of light, which determines a corresponding optical area on one or more objects, thereby tracking one or more objects with higher accuracy over a wide distance range.

Claims

1. A time-of-flight (TOF) camera device, comprising: A pulse generator is configured to generate pulse signals; An optical module is configured to emit output light toward at least one object in response to the pulse signal; A 3D sensor is configured to receive reflected light when the output light is reflected by the at least one object in a first frame, and to generate time information of the received reflected light based on phase difference information between the output light and the reflected light reflected by the at least one object, and to generate a 3D depth map of the at least one object based on the time information. A distance calculator is configured to receive the output of the 3D sensor and generate a distance data signal, wherein the distance data signal is generated based on the 3D depth map; and A light density control device is configured to receive the distance data signal from the distance calculator and output a light density control signal, wherein the light density control signal controls the size of an opening within the optical module for emitting the output light to determine the size of a projection area on the at least one object. Furthermore, the optical module emits the output light toward the at least one object within the first frame, wherein the output light has an optical density corresponding to the optical density control signal.

2. The TOF camera device according to claim 1, wherein, The phase difference information includes the difference between the phase of the output light of the optical module and the phase of the reflected light reflected by the object.

3. The TOF camera device according to claim 2, wherein: The at least one object includes a first object separated from the optical module by a first distance and a second object separated from the optical module by a second distance, wherein the second distance is greater than the first distance; and The 3D depth map is generated by combining the depth of the first object and the depth of the second object.

4. The TOF camera device according to claim 2, wherein, The 3D sensor generates the 3D depth map by continuously combining rays of reflected light from the at least one object.

5. The TOF camera device according to claim 2, wherein, The 3D sensor generates the 3D depth map by combining the rays of the reflected light reflected by the at least one object.

6. The TOF camera device according to claim 1, wherein: The optical module includes an optical density controller; and The optical density controller responds to the optical density control signal to adjust the size of the aperture used to emit the output light.

7. The TOF camera device according to claim 6, wherein, In response to the optical density control signal, the optical density controller adjusts the size of the opening from a first size to a second size, wherein the second size is smaller than the first size.

8. The TOF camera device according to claim 6, wherein: The at least one object includes a first object separated from the optical module by a first distance and a second object separated from the optical module by a second distance, wherein the second distance is greater than the first distance; and The optical density control device selects the first object or the second object, and outputs the optical density control signal based on the selected object.

9. A method for driving a time-of-flight (TOF) camera device, comprising: Generate pulse signals; In response to the pulse signal, output light is emitted toward at least one object; When the output light is reflected by the at least one object in the first frame, the reflected light is received, and the timing information of the received reflected light is generated based on the phase difference information between the output light and the reflected light reflected by the at least one object, and a 3D depth map of the at least one object is generated based on the timing information. A distance data signal is generated from the received reflected light, wherein the distance data signal is generated based on the 3D depth map; Receive the distance data signal; and An output optical density control signal is provided, which determines the size of the opening used to emit output light onto the at least one object, and the optical density control signal also determines the size of the projection area on the at least one object based on the distance data signal.

10. The method according to claim 9, wherein: The at least one object includes a first object separated from the optical module by a first distance and a second object separated from the optical module by a second distance, wherein the second distance is greater than the first distance; and The 3D depth map is generated by combining the depth of the first object and the depth of the second object.

11. The method according to claim 9, wherein, Generating the 3D depth map of the at least one object includes: sequentially combining rays of reflected light reflected by the at least one object to generate the 3D depth map.

12. The method according to claim 9, wherein, Generating the 3D depth map of the at least one object includes combining rays of reflected light reflected by the at least one object to generate the 3D depth map.

13. The method according to claim 9, wherein, The optical density control signal allows adjustment of the size of the aperture used to emit the output light.

14. The method according to claim 13, wherein, The optical density control signal allows the size of the opening to be adjusted from a first size to a second size, where the second size is smaller than the first size.

15. The method according to claim 13, wherein: The at least one object includes a first object separated from the optical module by a first distance and a second object separated from the optical module by a second distance, wherein the second distance is greater than the first distance; and Outputting the optical density control signal includes: selecting the first object or the second object, and outputting the optical density control signal based on the selected object.

16. A method for driving a time-of-flight (TOF) camera device, comprising: Within the first frame, the optical module emits a first output light toward a first region of the first object; Based on the first reflected light reflected by the first object within the first frame, based on the first time information of the received first reflected light generated according to the phase difference information between the first output light and the first reflected light reflected by the first object, based on the first 3D depth map of the first object generated according to the first time information, and based on the first distance data signal generated according to the first 3D depth map, the region in the first object where the first output light was emitted is changed from the first region to the second region. In the second frame following the first frame, the optical module emits a second output light toward the third region of the second object; as well as Based on the second reflected light reflected by the second object within the second frame, the second time information of the received second reflected light generated according to the phase difference information between the second output light and the second reflected light reflected by the second object, the second 3D depth map of the second object generated according to the second time information, and the second distance data signal generated according to the second 3D depth map, the region in the second object where the second output light was emitted is changed from the third region to the fourth region. The size of the second region is different from the size of the fourth region.

17. The method of claim 16, wherein: The first object is positioned at a distance from the optical module, which is a first distance away; and The second object is positioned at a distance from the optical module that is different from the first distance.

18. The method of claim 16, wherein: The size of the first region is smaller than the size of the second region; and The size of the third region is greater than the size of the fourth region.

19. The method of claim 16, wherein: The first object is positioned at a distance from the optical module, which is a first distance away. The second object is positioned at a greater distance from the optical module than the first distance; and The size of the fourth region is smaller than the size of the second region.

Images