camera
By introducing multiple control modes and super-resolution technologies into TOF cameras, the output and input light are optimized, and the resolution and power consumption problems of TOF cameras are solved when obtaining depth information, and efficient depth information acquisition is achieved to adapt to different application scenarios. .
Patent Information
- Application Number
- CN201980053832.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-08-07
- Filing Date
- 2019-08-02
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2039-08-02
AI Technical Summary
When TOF cameras obtain depth information, especially for moving objects, it is difficult to achieve the desired depth resolution, and consume too much power and resources when generating high resolution or high frame rate depth information.
A camera is designed, including a light emitting module, a light receiving module and a control module. The output and input light are optimized through different control modes, and the exposure time, frame rate and number of activated pixels are dynamically adjusted according to the presence and distance of the object to generate high resolution. and improve resolution through super-resolution method.
It realizes the optimization of the driving of TOF cameras in different application scenarios, reduces power consumption and resource usage, and improves the resolution of depth information, which is suitable for the needs of low resolution and high resolution depth information.
Smart Images

Figure CN112585961B_ABST
Abstract
Description
Technical Field
[0001] Embodiments relate to cameras. Background Art
[0002] The technology of obtaining 3D images using a camera is developing. In order to obtain a 3D image, depth information (depth map) is required. Depth information is information indicating a distance in space, and indicates perspective information of another point relative to one point of a 2D image.
[0003] One of the methods for obtaining depth information is a method of projecting infrared (IR) structured light onto an object and analyzing light reflected from the object to extract depth information. According to the IR structured light method, there is a problem in that it is difficult to obtain a desired level of depth resolution for a moving object.
[0004] As a technology to replace the light method of the IR structure, the time-of-flight (TOF) method has attracted attention. According to the TOF method, the distance to the object is calculated by measuring the flight time (ie, the time of emission and reflection of light).
[0005] Generally, in the case of the TOF method, in order to accurately measure the distance to an object, an amount of light sufficient to illuminate the surface should be ensured even from a long distance, and thus, a lot of power is consumed.
[0006] However, for each application that wants to use the depth information obtained by the TOF camera, different specifications are required. For example, in some applications, low-resolution depth information may be required, or small-frame depth information may be required. In this case, when the TOF camera is driven in a manner to generate high-resolution depth information or high-frame depth information, in addition to consuming unnecessary power, there is also a problem of unnecessarily increasing resource occupation.
[0007] Therefore, a technology capable of optimizing the driving of a TOF camera is needed. Summary of the invention
[0008] [Technical issues]
[0009] An embodiment is directed to a camera, and in particular, a camera capable of providing a TOF camera driving method optimized for an object shooting situation.
[0010] The problems to be solved by the embodiments are not limited thereto, but also include purposes and effects that can be understood from the technical solutions or implementations of the invention described below.
[0011] [Technical solution]
[0012] One aspect of the present invention provides a camera, comprising: a light emitting module, which is configured to output output light according to a set control mode; a light receiving module, which is configured to receive input light corresponding to the output light according to the control mode; and a control module, which is configured to detect at least one of the presence of an object and the distance to the object based on the input light, reset the control mode according to the detection result, control the output of the light emitting module and the input of the light receiving module according to the reset control mode, and generate a depth map of the object based on the input light input according to the reset control mode.
[0013] The control mode may include a first control mode and a second control mode, and at least one of an exposure time of the light emitting module and a frame rate and a number of activated pixels of the light receiving module may be set differently in the first control mode and the second control mode.
[0014] When a camera operation signal is input, the light emitting module may output a first output light according to a preset first control mode, and the light receiving module may receive a first input light corresponding to the first output light according to the first control mode.
[0015] As a result of detecting the presence of the object based on the first input light, when the object is detected, the control module resets to the second control mode, the light emitting module may output second output light according to the second control mode, and the light receiving module may receive the second input light reflected from the object.
[0016] The control mode may include third to fifth control modes, and at least one of an exposure time and a modulation frequency of the light emitting module may be differently set in the third to fifth control modes.
[0017] The modulation frequency may be set to a first frequency in a third control mode, the modulation frequency may be set to a second frequency having a greater value than the first frequency in a fourth control mode, and the modulation frequency may be set to the first frequency and the second frequency in a fifth control mode.
[0018] When the camera operation signal is input, the light emitting module may output third output light according to the third control mode, and the light receiving module may receive third input light corresponding to the third output light according to the third control mode.
[0019] As a result of detecting the presence of an object based on the third input light, when the object is detected, the control module can calculate the distance to the object based on the third input light, and when the object is not detected, the control module can perform a reset to change to a fifth control mode, the light emitting module can output the fifth output light to the object according to the reset fifth control mode, and the light receiving module can receive the fifth input light reflected from the object.
[0020] When the distance to the object is greater than or equal to a threshold value, the control module can perform a reset to maintain the third control mode, the light-emitting module can output a third output light to the object according to the reset third control mode, and the light-receiving module can receive a third input light reflected from the object, and when the distance to the object is less than the threshold value, the control module can perform a reset to change to a fourth control mode, the light-emitting module can output a fourth output light to the object according to the reset fourth control mode, and the light-receiving module can receive a fourth input light reflected from the object.
[0021] The control module may generate a depth map of the object based on one of the third input light to the fifth input light reflected from the object, and when the depth map is generated based on the third input light or the fifth input light reflected from the object, the control module may generate a depth map having a higher resolution than a resolution of the depth map based on the fourth input light through a super-resolution method.
[0022] The control module may calculate the size of the object based on the depth information on the generated depth map, and transmit the calculated size of the object to the connected application. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 is a block diagram of a camera according to an embodiment of the present invention.
[0024] Figure 2 is a flowchart illustrating a first example of a camera control method according to an embodiment of the present invention.
[0025] Figure 3 is a flowchart illustrating a second example of the camera control method according to the embodiment of the present invention.
[0026] Figure 4 is a flowchart illustrating a third example of the camera control method according to the embodiment of the present invention.
[0027] Figure 5 is a diagram for describing a control mode according to an embodiment of the present invention.
[0028] Figure 6 is a diagram for describing a fifth control mode according to the embodiment of the present invention.
[0029] Figure 7 is a diagram for describing an optimized camera operation according to an embodiment of the present invention. DETAILED DESCRIPTION
[0030] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.
[0031] However, the technical idea of the present invention is not limited to some embodiments to be described, but can be implemented in various forms, and within the scope of the technical idea of the present invention, one or more components can be selectively combined and replaced between the embodiments.
[0032] In addition, unless explicitly defined and described, the terms (including technical terms and scientific terms) used in the embodiments of the present invention may have meanings that can be generally understood by ordinary technicians in the field, and common terms such as terms defined in dictionaries may be understood in consideration of their meanings in the context of relevant technologies.
[0033] In addition, the terms used in the embodiments of the present invention are used to describe the embodiments and are not intended to limit the present invention.
[0034] In this specification, unless otherwise explicitly stated in a phrase, a singular form may also include a plural form, and when described as "at least one (or more than one) of A and B or A, B and C", it may include one or more of all combinations that can be combined by A, B and C.
[0035] In addition, terms such as first, second, A, B, (a), and (b) may be used to describe components of embodiments of the present invention.
[0036] These terms are only for distinguishing one component from other components, and the nature, order or sequence of the components are not limited by the terms.
[0037] In addition, when a component is described as being “connected,” “coupled” or “coupled” to another component, the component may be directly connected, coupled or coupled to the other component, and the component may be “connected,” “coupled” or “coupled” to the other component through another component located between the component and the other component.
[0038] Furthermore, the case where a component is described as being formed or disposed “above (upper) or below (lower)” another component includes not only the case where the two components are in direct contact with each other but also the case where one or more other components are formed or disposed between the two components. In addition, the case expressed as “above (upper) or below (lower)” may include not only the meaning based on the upward direction of one component but also the meaning based on the downward direction of one component.
[0039] First, refer to Figure 1 The configuration of a camera according to an embodiment of the present invention is described.
[0040] Figure 1 is a block diagram of a camera according to an embodiment of the present invention.
[0041] like Figure 1As shown, the camera 100 according to the embodiment of the present invention includes a light emitting module 110 , a light receiving module 120 , and a control module 130 .
[0042] First, the light emitting module 110 outputs output light according to a set control mode. The light emitting module 110 may include a light source and a light modulator to output the output light.
[0043] The light source generates light. The light generated by the light source may be infrared light having a wavelength of 770 to 3000 nm, or visible light having a wavelength of 380 to 770 nm. The light source may be implemented by a light emitting diode (LED), and may be implemented in the form of arranging a plurality of light emitting diodes according to a predetermined pattern. In addition, the light source may include an organic light emitting diode (OLED) or a laser diode (LD). The light source repeatedly flashes (on / off) at predetermined time intervals, thereby outputting output light in the form of a pulse wave or a continuous wave. All of the light emitting diodes in the plurality of light emitting diodes may repeatedly flash at the same time interval. In addition, all of the light emitting diodes in the plurality of light emitting diodes may repeatedly flash at different time intervals during a portion of the exposure time. In addition, among the plurality of light emitting diodes, the first group of light emitting diodes and the second group of light emitting diodes may repeatedly flash at different time intervals.
[0044] The light modulator controls the flickering of the light source according to the control mode. The light modulator can control the flickering of the light source so that the output light with the modulation frequency according to the control mode is output through frequency modulation, pulse modulation, etc. In addition, the light modulator can control the flickering of the light source according to the control mode to output the output light during the exposure time.
[0045] Next, the light receiving module 120 receives input light corresponding to the output light according to the control mode. The light receiving module 120 may include a lens unit and an image sensor unit to receive the input light.
[0046] The lens unit condenses input light and transfers the condensed light to the image sensor unit. To this end, the lens unit may include a lens, a lens barrel, a lens holder, and an IR filter.
[0047] A plurality of lenses may be provided or one lens may be provided. When a plurality of lenses are provided, the corresponding lenses may be aligned based on a central axis to form an optical system. Here, the central axis may be the same as the optical axis of the optical system.
[0048] The lens barrel may be coupled to the lens holder and may have a space for accommodating the lens therein. The lens barrel may be rotatably coupled to the lens or lenses. However, this is exemplary, and the lens barrel may be coupled to the lens or lenses in a different manner, for example, using an adhesive (e.g., an adhesive resin such as epoxy resin).
[0049] The lens holder can be coupled to the lens barrel to support the lens barrel, and can be coupled to a printed circuit board on which an image sensor is mounted. The lens holder can have a space below the lens barrel in which an IR filter can be attached. A spiral pattern is formed on the inner circumferential surface of the lens holder, and the lens holder can be rotatably coupled to the lens barrel, which similarly has a spiral pattern formed on the outer circumferential surface. However, this is exemplary, and the lens holder and the lens barrel can be coupled to each other by an adhesive, or the lens holder and the lens barrel can be formed integrally with each other.
[0050] The lens holder may be divided into an upper holder coupled to the lens barrel and a lower holder coupled to a printed circuit board on which an image sensor is mounted, and the upper holder and the lower holder may be formed integrally with each other or may be formed in a separate structure and then fastened or coupled to each other. In this case, the diameter of the upper holder may be formed to be smaller than the diameter of the lower holder. In the present specification, the lens holder may be a housing.
[0051] The image sensor unit absorbs the concentrated input light to generate an electrical signal.
[0052] The image sensor unit may absorb input light in synchronization with the blinking cycle of the light source. Specifically, the image sensor unit may absorb input light in phase and out of phase with the output light. That is, the image sensor unit may repeatedly perform the steps of absorbing input light when the light source is turned on and absorbing input light when the light source is turned off.
[0053] The image sensor unit may use a plurality of reference signals having different phase differences to generate an electrical signal corresponding to each reference signal. The frequency of the reference signal may be set equal to the frequency of the output light. Thus, when the output light is output at a plurality of frequencies, the image sensor unit generates an electrical signal using a plurality of reference signals corresponding to the corresponding frequencies. The electrical signal may include information about the amount of charge or voltage corresponding to each reference signal.
[0054] According to an embodiment of the present invention, there may be four reference signals C1 to C4. Each of the reference signals C1 to C4 may have the same frequency as the output light and may have a phase difference of 90° from each other. One C1 of the four reference signals may have the same phase as the output light. The phase of the input light is delayed by a distance that is the distance that the output light is reflected and returned after the output light is incident on the object. The image sensor unit may mix the input light and each reference signal to generate an electrical signal for each reference signal.
[0055] In another embodiment, when the output light is generated at a plurality of frequencies during the exposure time, the image sensor unit absorbs the input light according to the plurality of frequencies. For example, it is assumed that the output light is generated at frequencies f1 and f2, and the plurality of reference signals have a phase difference of 90°. Then, since the input light also has frequencies f1 and f2, four electrical signals can be generated by the input light having the frequency f1 and the four reference signals corresponding to the input light. In addition, four electrical signals can be generated by the input light having the frequency f2 and the four reference signals corresponding to the input light. Therefore, a total of 8 electrical signals can be generated.
[0056] The image sensor unit may be implemented as an image sensor in which a plurality of photodiodes are arranged in a grid shape. The image sensor may be a complementary metal oxide semiconductor (CMOS) image sensor, or may be a charge coupled device (CCD) image sensor.
[0057] Meanwhile, the light emitting module 110 and the light receiving module 120 may be implemented as a plurality in the camera. For example, when the camera according to an embodiment of the present invention is included in a smart phone, the first light emitting module 110 and the first light receiving module 120 corresponding thereto may be disposed on the front surface of the smart phone, and the second light emitting module 110 and the second light receiving module 120 corresponding thereto may be disposed on the rear surface of the smart phone.
[0058] Next, the control module 130 detects at least one of the presence of the object and the distance to the object based on the input light. Specifically, the control module 130 can detect at least one of the presence of the object and the distance to the object through a depth map, which is generated by the input light. For example, the control module 130 can generate a depth map through an electrical signal corresponding to the input light. When the output light is output in the form of a continuous wave, the distance to the object can be detected using the following equation 1.
[0059] [Equation 1]
[0060]
[0061] Here, f represents the frequency of output light, c represents the speed of light, and φ represents the phase difference between the output light and the corresponding input light.
[0062] In addition, the phase difference between the output light and the corresponding input light can be calculated by the following Equation 2.
[0063] [Equation 2]
[0064]
[0065] Here, τ represents the flight time. Each of Q1 to Q4 is the charge amount of each of the four electrical signals. Q1 is the charge amount of the electrical signal corresponding to the reference signal in phase with the incident light signal. Q2 is the charge amount of the electrical signal corresponding to the reference signal whose phase is 180° slower than the incident light signal. Q3 is the charge amount of the electrical signal corresponding to the reference signal whose phase is 90° slower than the incident light signal. Q4 is the charge amount of the electrical signal corresponding to the reference signal whose phase is 270° slower than the incident light signal.
[0066] The control module 130 resets the control mode according to a detection result of at least one of the presence of the object and the distance to the object.
[0067] Specifically, when the presence of the object is detected, the control module 130 resets the set first control mode to the second control mode. Meanwhile, when the presence of the object is not detected, the control module 130 performs a reset to maintain the set first control mode.
[0068] Alternatively, when the presence of an object is detected and the distance to the object is greater than or equal to a threshold value, a reset is performed so that the set third control mode is maintained. In addition, when the presence of an object is detected and the distance to the object is greater than or equal to a threshold value, the control module 130 may perform a reset so that the set third control mode is changed to a fourth control mode. Meanwhile, if the presence of an object is not detected, the control module 130 performs a reset so that the set third control mode is changed to a fifth control mode.
[0069] The control module 130 controls the output of the light emitting module 110 and the input of the light receiving module 120 according to the reset control mode.
[0070] The control module 130 generates a depth map of the object based on the input light input according to the reset control mode. The process of generating the depth map is the same as the process described by the above equation, so its detailed description will be omitted.
[0071] Meanwhile, when generating a depth map based on the third input light or the fifth input light reflected from the object, the control module 130 may generate a depth map having a higher resolution than a depth map based on the fourth input light through a super-resolution method.
[0072] For example, when the depth map based on the fourth input light has a resolution of a QVGA (320×240) level, the depth map based on the third input light or the fifth input light may have a resolution of a VGA (640×480) level.
[0073] A super-resolution method, ie, a super-resolution (SR) technique, is a technique for obtaining a high-resolution image from a plurality of low-resolution images, and a mathematical model of the SR technique can be expressed as in the following Equation 3.
[0074] [Equation 3]
[0075] y k =D k B k M k x+n k
[0076] Here, 1≤k≤p, p represents the number of low-resolution images, y k Represents a low-resolution image (=[y k,1 ,y k,2 ,…,y k,M ] T , where M = N1*N2), D k represents the downsampling matrix, B k represents the optical fuzzy matrix, M k is the image warping matrix, x represents the high-resolution image (=[x1,x2,…,x N ] T , where N = L1N1*L2N2), n k represents noise. That is, according to the SR technique, it refers to the inverse function of the estimated resolution degradation element applied to y k The SR technology can be roughly divided into statistical methods and multi-frame methods, and the multi-frame method can be roughly divided into space division methods and time division methods.
[0077] The control module 130 may transmit the depth map of the object to the connected application. In addition, the control module 130 may detect the size of the object through the depth map of the object and transmit the detected size information to the connected application.
[0078] Figure 2 is a flowchart illustrating a first example of a camera control method according to an embodiment of the present invention.
[0079] Reference Figure 2 , when a camera operation signal is input, the light emitting module outputs a first output light according to a set first control mode (S205).
[0080] Then, the light receiving module 120 receives the first input light corresponding to the first output light according to the set first control mode (S210).
[0081] Next, the control module 130 detects the presence of the object based on the first input light ( S215 ).
[0082] When the presence of the object is detected, the control module 130 performs resetting so that the set first control mode is changed to the second control mode ( S220 ).
[0083] Then, according to the reset second control mode, the light emitting module 110 outputs the second output light, and the light receiving module 120 receives the second input light reflected from the object (S225).
[0084] Then, the control module 130 generates a depth map of the object based on the second input light (S230).
[0085] Meanwhile, when the presence of the object is not detected, after performing a reset to maintain the first control mode, it proceeds again from step S205 , and when the object is still not detected after repeating a predetermined number of times, termination may be performed.
[0086] Figure 3 is a flowchart illustrating a second example of the camera control method according to the embodiment of the present invention.
[0087] When the camera operation signal is input, the light emitting module 110 outputs the third output light according to the preset third control mode (S305).
[0088] Then, the light receiving module 120 receives the third input light corresponding to the third output light according to the set third control mode (S310).
[0089] Next, the control module 130 detects the presence of the object based on the third input light ( S315 ).
[0090] When the presence of the object is detected, the control module 130 calculates a distance to the object based on the third input light, and compares the calculated distance to the object with a threshold value ( S320 ).
[0091] In this case, when the distance to the object is greater than or equal to the threshold, the control module 130 performs a reset to maintain the third control mode (S325). Then, according to the reset third control mode, the light emitting module 110 outputs the third output light to the object, and the light receiving module 120 receives the third input light reflected from the object (S330).
[0092] When the distance to the object is less than the threshold, the control module 130 performs a reset so that the set third control mode is changed to the fourth control mode (S335). Then, according to the reset fourth control mode, the light emitting module 110 outputs the fourth output light to the object, and the light receiving module 120 receives the fourth input light reflected from the object (S340).
[0093] At the same time, when the object is not detected, the control module 130 performs a reset so that the preset third control mode is changed to the fifth control mode (S345). Then, according to the reset fifth control mode, the light emitting module 110 outputs the fifth output light to the object, and the light receiving module 120 receives the fifth input light reflected from the object (S350).
[0094] Then, the control module 130 generates a depth map of the object based on any one of the third to fifth input lights (S355). The process of generating the depth map is the same as the process described by the above equation, so its detailed description will be omitted. Meanwhile, when generating a depth map based on the third or fifth input light reflected from the object, the control module 130 may generate a depth map having a higher resolution than that of the depth map based on the fourth input light reflected from the object by a super-resolution method.
[0095] In addition, the control module 130 may calculate the size of the object based on the depth information of the generated depth map (S360), and transmit the size of the object to the connected application (S365). In this case, the connected application may be the application to which the camera operation signal of step S305 is input.
[0096] Figure 4 is a flowchart illustrating a third example of the camera control method according to the embodiment of the present invention.
[0097] according to Figure 4 The camera control method shown, Figure 3 The camera control method shown and Figure 4 The camera control methods shown can be implemented together.
[0098] Reference Figure 4 When the control module 130 receives the camera operation signal (S405), the type of the camera operation signal is determined (S410). In this case, the control module 130 may operate the first light emitting module 110 and the first light receiving module 120 according to the type of the camera operation signal, or may operate the second light emitting module 110 and the second light receiving module 120.
[0099] Specifically, when the input camera operation signal is the first camera operation signal, the control module 130 may operate the first light emitting module 110 and the first light receiving module 120. Then, the control module 130 controls the camera operation signal according to the first camera operation signal. Figure 2 The camera control method shown in the figure performs camera control. For example, when the user inputs a first camera operation signal through a button input or a motion input for 3D face recognition, the control module 130 can operate the first light emitting module 110 and the first light receiving module 120 to perform camera control according to the first camera operation signal. Figure 2 Steps S205 to S230 perform camera control.
[0100] Meanwhile, when the input camera operation signal is the second camera operation signal, the control module 130 may operate the second light emitting module 110 and the second light receiving module 120. Then, the control module 130 operates according to Figure 3 The camera control method shown performs camera control. For example, when the user inputs a second camera operation signal through the application to detect the size of an object, the control module 130 can operate the second light emitting module 110 and the second light receiving module 120 to perform camera control according to steps S305 to S365.
[0101] Reference above Figure 2 and Figure 3 Steps S205 to S230 and steps 305 to S355 are described, and thus, detailed descriptions thereof will be omitted.
[0102] Figure 5 is a diagram for describing a control mode according to an embodiment of the present invention.
[0103] refer to Figure 5 , the control mode according to the embodiment of the present invention may include the first control mode to the fifth control mode, the first control mode and the second control mode may be grouped, and the third control mode to the fifth control mode may be grouped.
[0104] Specifically, the camera according to the embodiment of the present invention may be controlled according to one of the first control mode and the second control mode according to the camera operation signal.
[0105] The first control mode may be a control mode for searching for an object, and the second control mode may be a control mode for accurately measuring an object. For example, in the case of facial recognition, the first control mode may be a control mode for detecting the presence of a face (object), and the second control mode may be a control mode for generating a depth map of the face (object).
[0106] Table 1 below is a table showing characteristics of the first control mode and the second control mode.
[0107] [Table 1]
[0108]
[0109] As shown in Table 1, in the first control mode and the second control mode, the exposure time of the light emitting module 110, and at least one of the frame rate and the number of activated pixels of the light receiving module 120 can be set differently. In addition, the effects are also different. Specifically, the exposure time of the light emitting module 110 in the first control mode can be set to be shorter than the exposure time in the second control mode. For example, the exposure time of the first control mode can be set to be shorter than 0.1ms, and the exposure time of the second control mode can be set to be longer than 0.7ms.
[0110] The frame rate of the light receiving module 120 in the first control mode can be set to be less than the frame rate of the second control mode. For example, the frame rate of the first control mode can be set to 1fps, and the frame rate of the second control mode can be set to be greater than 15fps. In particular, since the first control mode is used to detect the presence of an object, the frame rate of the light receiving module 120 can be set to 1fps, thereby generating only one frame.
[0111] The number of activated pixels of the light receiving module 120 in the first control mode may be set to be smaller than the number of activated pixels of the light receiving module 120 in the second control mode. That is, the viewing angle of the light receiving module 120 in the first control mode may be set to be smaller than the viewing angle of the light receiving module 120 in the second control mode. For example, in the first control mode, 112x86 pixels may be activated and the viewing angle of the light receiving module 120 may be set to 40°, while in the second control mode, 224x172 pixels may be activated and the viewing angle of the light receiving module 120 may be set to 80°.
[0112] When the camera according to an embodiment of the present invention is operated according to the first control mode, the depth accuracy is lower than that of the second control mode, but an object located at a longer distance can be measured with less power. That is, after detecting the presence of an object with a small amount of power, when it is determined that the object exists, accurate shooting is performed according to the second control mode, and therefore, the power consumption of the camera can be reduced.
[0113] Next, the camera according to the embodiment of the present invention may be controlled according to any one of the third to fifth control modes according to the camera operation signal.
[0114] Table 2 below is a table showing characteristics of the third to fifth control modes.
[0115] [Table 2]
[0116]
[0117] As shown in Table 2, in the third control mode to the fifth control mode, at least one of the exposure time and the modulation frequency of the light emitting module 110 can be set differently. The exposure time of the light emitting module 110 in the fourth control mode can be set to be less than the exposure time in the third control mode and the fifth control mode. For example, the exposure time of the fourth control mode can be set to be less than 1.5ms, and the exposure time of the third control mode and the fifth control mode can be set to be greater than 1.5ms. The fourth control mode is a control mode for photographing an object at a short distance such as within 1m, and therefore, even when the exposure time is shorter than the exposure time of the third control mode and the fifth control mode, the light receiving module 120 can ensure a sufficient amount of light to generate a depth map.
[0118] In the third control mode, the modulation frequency is set to the first frequency, the fourth control mode is set to the second frequency whose modulation frequency is greater than the first frequency, and in the fifth control mode, the modulation frequency can be set to a combination of the first frequency and the second frequency (i.e., two frequencies). For example, the modulation frequency can be set to 60 MHz in the third control mode, the modulation frequency can be set to 80 MHz in the fourth control mode, and the modulation frequency can be set to 60 MHz and 80 MHz in the fifth control mode.
[0119] As shown in Table 2, the camera according to the embodiment of the present invention controls the light emitting module 110 and the light receiving module 120 differently according to the distance from the object through the third control mode to the fifth control mode. That is, since the camera is operated by the control module 130 optimized according to the distance from the object, the power consumption of the camera can be reduced.
[0120] Figure 6 is a diagram for describing a fifth control mode according to an embodiment of the present invention.
[0121] Figure 6 The process of combining two modulation frequencies is shown. For example, assume that the first frequency is 60 MHz and the second frequency is 80 MHz.
[0122] The maximum distance at which an object can be measured is determined according to the frequency of the output light, and an object at a maximum distance of 1.8657 m can be measured by outputting light at a first frequency of 60 MHz, and an object at a maximum distance of 2.4876 m can be measured by outputting light at a second frequency of 80 MHz. In this way, as the frequency increases, the maximum distance at which an object can be measured also increases. However, in order to increase the frequency, it is necessary to quickly control the flashing cycle of the light emitting module 110, and therefore, the power consumption increases.
[0123] Therefore, in the fifth control mode according to the embodiment of the present invention, the measurement distance of the object can be increased by outputting the first frequency and the second frequency at the same time.
[0124] like Figure 6 As shown, when the output light according to the first frequency and the output light according to the second frequency are outputted simultaneously, the first frequency and the second frequency form waveforms with different periods, and a part of the phases of the two frequencies may overlap with each other. In this way, when the two frequencies are outputted simultaneously, the part where the phases of the two frequencies overlap with each other can be regarded as one period. That is, when the frequencies of 60MHz and 80MHz are outputted simultaneously, it can be regarded as one output light with a frequency of 240MHz. In this case, the power consumption can be greatly reduced compared with outputting the output light with a frequency of 240MHz.
[0125] Figure 7 is a diagram for describing an optimized camera operation according to an embodiment of the present invention.
[0126] Figure 7 The diagram is used to describe the Figure 3 An example of camera operation using the camera control method shown in FIG. Figure 7 As shown in FIG. 1 , when photographing a small-sized object (e.g., the ring, bolt, and food shown in (a) to (c)), the user photographs after placing the camera close to the object. In this case, the control module 130 generates a depth map by photographing the object according to the fourth control mode based on information about the existence of the object and the distance to the object according to the third control mode.
[0127] When photographing a relatively large object (e.g., a sofa or curtain shown in (d) and (e)), the user photographs after placing the camera at a distance equal to or greater than a certain distance from the object. In this case, the control module 130 generates a depth map by photographing the object according to the third control mode.
[0128] Meanwhile, as shown in (f), when performing indoor positioning shooting, the user shoots after placing the camera at a position far away from the object. In this case, the control module 130 generates a depth map by shooting the object according to the fifth control mode based on information about the existence of the object according to the third control mode.
Claims
1. A camera, comprising: a light emitting module configured to output output light according to a set control mode; a light receiving module configured to receive input light corresponding to the output light according to the control mode; as well as a control module configured to detect at least one of the presence of an object and a distance to the object based on the input light, reset the control mode according to the detection result, control the output of the light emitting module and the input of the light receiving module according to the reset control mode, and generate a depth map of the object based on the input light input according to the reset control mode, The control module is further configured to, when the presence of the object is not detected, control the output of the light emitting module according to a set control mode, and repeatedly detect at least one of the presence of the object and the distance to the object based on the input light, The control mode is selected from one of the first control mode to the fifth control mode. Wherein, in the third control mode to the fifth control mode, at least one of the exposure time and the modulation frequency of the light emitting module is set differently, Wherein, in the third control mode, the modulation frequency is set to the first frequency, wherein, in the fourth control mode, the modulation frequency is set to a second frequency having a value greater than that of the first frequency, Wherein, in the fifth control mode, the modulation frequency is set to a combination of the first frequency and the second frequency, wherein when the distance to the object is greater than or equal to a threshold, the control module performs a reset to maintain the third control mode, the light emitting module outputs a third output light to the object according to the reset third control mode, and the light receiving module receives a third input light reflected from the object, and When the distance to the object is less than a threshold, the control module performs a reset to change to the fourth control mode, the light emitting module outputs a fourth output light to the object according to the reset fourth control mode, and the light receiving module receives a fourth input light reflected from the object, When the object is not detected, the control module performs a reset to change to the fifth control mode, the light emitting module outputs a fifth output light to the object according to the reset fifth control mode, and the light receiving module receives a fifth input light reflected from the object, The control module generates a depth map of the object based on any one of the third to fifth input lights, and The exposure time of the light emitting module in the fourth control mode is set to be shorter than the exposure time in the third control mode and the fifth control mode.
2. The camera according to claim 1, wherein: The control mode includes a first control mode and a second control mode, and At least one of an exposure time of the light emitting module and a frame rate and a number of activated pixels of the light receiving module is set differently in the first control mode and the second control mode.
3. The camera according to claim 2, wherein: When a camera operation signal is input, the light emitting module outputs a first output light according to a preset first control mode, and The light receiving module receives first input light corresponding to first output light according to the first control mode.
4. The camera according to claim 3, wherein: As a result of detecting the presence of an object based on the first input light, when the object is detected, the control module resets to the second control mode, the light emitting module outputs second output light according to the second control mode, and the light receiving module receives second input light reflected from the object.
5. The camera according to claim 1, wherein: When a camera operation signal is input, the light emitting module outputs a third output light according to the third control mode, and The light receiving module receives third input light corresponding to third output light according to the third control mode.
6. The camera according to claim 5, wherein: As a result of detecting the presence of the object based on the third input light, while: When the object is detected, the control module calculates a distance to the object based on the third input light.
7. The camera according to claim 1, wherein: When the depth map is generated based on the third input light or the fifth input light reflected from the object, the control module generates a depth map having a higher resolution than a resolution of the depth map based on the fourth input light through a super-resolution method.
