A three-dimensional sensor

Abstract

The present application relates to a camera module having an image sensor and a three-dimensional sensor. The camera module includes an image sensor, a three-dimensional (3D) sensor, a pre-processor, and an actuator controller. The pre-processor can be adapted to receive image information from the image sensor, generate an autofocus control signal based on the image information, and provide the autofocus control signal to the actuator controller. The actuator controller can be adapted to receive the autofocus control signal, generate an actuator drive signal based on the autofocus control signal, and provide the actuator drive signal to the 3D sensor. The 3D sensor can be adapted to adjust a field of view (FOV) based on the actuator drive signal.

Description

[0001] This application is a divisional application of Chinese invention patent application No. 202011078311.1, entitled "Camera Module with Image Sensor and Three-Dimensional Sensor", filed with the State Intellectual Property Office of the People's Republic of China on October 10, 2020. Technical Field

[0002] The various embodiments of this disclosure generally relate to a camera module having an image sensor and a three-dimensional sensor, and a camera system including the camera module. Background Technology

[0003] Recently, portable camera modules in mobile devices have achieved more stereoscopic images by introducing additional three-dimensional (3D) sensors. Time-of-flight (ToF) sensors, as a type of 3D sensor, have attracted attention. ToF sensors emit infrared laser light towards an object, receive the laser beam reflected from the object, and measure the time difference between them to obtain 3D information about the object. The maximum effective distance at which a ToF sensor can sense 3D information depends on the laser power. For example, the higher the laser power, the more accurately the ToF sensor can sense the 3D information of objects located at a distance. However, the laser power of 3D sensors, especially those used in mobile devices, cannot be sufficiently increased due to power consumption limitations (i.e., battery performance and eye safety). When the laser emission angle of a ToF sensor is reduced to increase the laser power per unit area, the field of illumination (FOI) narrows, and therefore the effective field of view (FOV) also narrows. In this case, nearby objects and larger objects are outside the FOI or FOV range, and therefore the entire image cannot be obtained. Conversely, when the laser emission angle of the ToF sensor is increased to ensure a wide effective FOV, it becomes difficult to obtain effective 3D information because the laser power per unit area decreases (i.e., the noise component increases). Summary of the Invention

[0004] Various embodiments of this disclosure relate to a three-dimensional sensor in which the laser emission angle is adjusted according to the distance between the camera module and the object.

[0005] The various embodiments of this disclosure relate to a camera module having an autofocus function and a laser emission angle adjustment function based on the distance between the camera module and the object.

[0006] Various embodiments of this disclosure relate to a camera module and a camera system that include a three-dimensional (3D) sensor and have a laser emission angle adjustment function based on an autofocus control signal from an image sensor, according to the distance between the camera module and an object.

[0007] These technical features of the present disclosure are not limited to the above embodiments, and those skilled in the art to which this disclosure pertains may understand other technical features not described herein from the following detailed description.

[0008] According to one embodiment, a camera module may include a first sensor, a second sensor, a preprocessor, and an actuator controller. The preprocessor may be adapted to receive image information from the first sensor, generate an autofocus control signal based on the image information, and provide the autofocus control signal to the actuator controller. The actuator controller may be adapted to receive the autofocus control signal, generate an actuator drive signal based on the autofocus control signal, and provide the actuator drive signal to the second sensor. The second sensor may be adapted to adjust the field of view (FOV) based on the actuator drive signal.

[0009] The second sensor may include: a light source disposed on a substrate; a diffuser disposed on the light source; and an actuator. The actuator may be adapted to adjust the distance between the light source and the diffuser based on an actuator drive signal.

[0010] The actuator may include a housing and a carrier. The carrier can support and move the diffuser.

[0011] The housing can enclose the light source, diffuser, and carrier.

[0012] The housing may include a drive coil. The carrier may include a magnet.

[0013] The number of drive coils can be equal to or greater than three.

[0014] The diffuser can be a single layer and has a fixed laser emission angle.

[0015] According to one embodiment, a camera system may include: an image sensor configured to capture image information; a three-dimensional (3D) sensor configured to acquire 3D information; a preprocessor configured to generate an autofocus control signal based on the image information received from the image sensor; an actuator controller configured to generate an actuator drive signal based on the autofocus control signal received from the preprocessor; an image signal processor configured to generate image data based on the image information received from the image sensor and the 3D information received from the 3D sensor; a memory unit configured to store the image data; and a display unit configured to display a visual image based on the image data. The 3D sensor may include: a light source; a diffuser; and an actuator configured to adjust the distance between the light source and the diffuser based on the actuator drive signal.

[0016] The actuator controller can generate an autofocus drive signal and provide it to the image sensor. The image sensor can then perform autofocus operations based on the autofocus drive signal.

[0017] The actuator may include a housing and a carrier. The housing may have a drive coil. The carrier may have a magnet. The housing may enclose the light source and the diffuser. The carrier may support and move the diffuser.

[0018] The diffuser can be a single layer and has a fixed laser emission angle.

[0019] According to one embodiment, a three-dimensional (3D) sensor may include: a light source disposed on a substrate; a diffuser disposed on the light source; and an actuator. The actuator may include: a housing; and a carrier configured to support and move the diffuser.

[0020] The carrier can be configured to move the diffuser to adjust the distance between the light source and the diffuser to at least three levels.

[0021] The housing can enclose the light source, diffuser, and carrier.

[0022] A diffuser can be positioned at various distances from the light source by a carrier.

[0023] The substrate may include a printed circuit board (PCB).

[0024] The light source may include a semiconductor laser diode.

[0025] The housing may include a drive coil, and the carrier may include a magnet.

[0026] The drive coil may include a bottom drive coil configured to pull the carrier to a low position and a top drive coil configured to pull the carrier to a high position.

[0027] The diffuser can be a single layer and has a fixed laser emission angle.

[0028] According to one embodiment, the operation method of the camera system may include: generating focused image information of an object via an image sensor; generating a control signal based on the focused information of the object, which is obtained from the focused image information; generating 3D information of the object via a Time-of-Flight (ToF) sensor by adjusting the field of view (FOV) of a ToF sensor according to the control signal; and generating a 3D image based on the focused image information and the 3D information.

[0029] Specific details of other embodiments are included in the following detailed description and accompanying drawings. Attached Figure Description

[0030] Figure 1AThis is a block diagram schematically illustrating a camera system according to an embodiment of the present disclosure.

[0031] Figure 1B This is a block diagram that briefly illustrates a camera module according to one embodiment of the present disclosure.

[0032] Figures 2A to 2E This is a schematic diagram illustrating a camera module according to one embodiment of the present disclosure.

[0033] Figure 3A and Figure 3B These are schematic top views and longitudinal cross-sectional views illustrating the configuration of a three-dimensional sensor according to an embodiment of the present disclosure.

[0034] Figures 4A to 4C This is a diagram illustrating the vertical movement of a carrier of a three-dimensional sensor according to an embodiment of the present disclosure in multiple stages.

[0035] Figure 5A and Figure 5B This is a diagram showing how the field of view (FOV) of a 3D sensor changes depending on the distance between the object and the camera module.

[0036] Figure 6A This is a flowchart illustrating a method for adjusting the field of view (FOV) of a three-dimensional sensor according to an embodiment of the present disclosure.

[0037] Figure 6B This is a flowchart illustrating a method for obtaining a three-dimensional image by adjusting the field of view (FOV) of a three-dimensional sensor according to an embodiment of the present disclosure.

[0038] Figure 7 This is a schematic diagram illustrating various camera systems having a camera module according to one embodiment of the present disclosure. Detailed Implementation

[0039] Figure 1A This is a block diagram schematically illustrating a camera system 1000 according to an embodiment of the present disclosure.

[0040] refer to Figure 1A The camera system 1000 according to this embodiment may include: a first sensor 111, a second sensor 112, a first preprocessor 121, a second preprocessor 122, an actuator controller 130, an image signal processor 140, a central processing unit 150, a memory unit 160, an audio unit 170, and a display unit 180.

[0041] The first sensor 111 may include various types of sensors, such as image sensors. For example, the first sensor 111 may include a CMOS image sensor. The second sensor 112 may include various types of sensors, such as three-dimensional (3D) sensors. For example, the second sensor 112 may include a time-of-flight (ToF) sensor.

[0042] The first preprocessor 121 can receive initial image information from the first sensor 111, remove noise from the initial image information, and generate an autofocus control signal for automatic exposure and autofocus. The autofocus control signal can be provided to the actuator controller 130. After the autofocus operation of the first sensor 111 is fully executed, the first preprocessor 121 can receive a valid image signal from the first sensor 111, remove noise from the valid image signal, and provide the processed valid image information to the image signal processor 140.

[0043] The second preprocessor 122 can receive 3D information, remove noise from the 3D information, and provide the processed 3D information to the image signal processor 140.

[0044] The actuator controller 130 can generate an autofocus drive signal based on an autofocus control signal received from the first preprocessor 121. The autofocus drive signal can be provided to the first sensor 111. The first sensor 111 can perform an autofocus operation based on the received autofocus drive signal. For example, the first sensor 111 can perform an autofocus operation by magnifying or reducing its focus.

[0045] The actuator controller 130 can generate an actuator drive signal based on an autofocus control signal received from the first preprocessor 121. The actuator drive signal can be provided to the second sensor 112. The second sensor 112 can perform a field-of-view (FOV) adjustment operation based on the actuator drive signal. The FOV adjustment operation of the second sensor 112 will be described below.

[0046] The image signal processor 140 can process image information and 3D information received from the first preprocessor 121 and the second preprocessor 122 respectively, and generate image data.

[0047] The central processing unit 150 can control the image signal processor 140, the memory unit 160, the audio unit 170, and the display unit 180. The central processing unit 150 can provide image data to the memory unit 160. The memory unit 160 can store the image data. Additionally, the memory unit 160 can provide the stored image data back to the central processing unit 150. The memory unit 160 may include at least one of a memory controller, DRAM, or flash memory.

[0048] The audio unit 170 may include an audio controller. The audio unit 170 may output speech and sound based on image data.

[0049] The display unit 180 may include a display controller and a display panel. The display unit 180 can display visual images based on image data.

[0050] Figure 1B This is a block diagram that briefly illustrates a camera module 100 according to one embodiment of the present disclosure.

[0051] refer to Figure 1B According to one embodiment, a camera module 100 may include an image sensor 111, a 3D sensor 112, and an actuator controller 130. The image sensor 111 may include a CMOS image sensor, while the 3D sensor 112 may include a Time-of-Flight (ToF) sensor. The actuator controller 130 may receive focus information about an object from the image sensor 111 and transmit the focus information to the 3D sensor 112. The field of view (FOV) of the 3D sensor 112 may be adjusted based on the received focus information. For example, the FOV of the 3D sensor 112 may be expanded or narrowed based on the focus information received from the image sensor 111.

[0052] Figures 2A to 2E The diagrams schematically illustrate camera modules 100A to 100E according to embodiments of the present disclosure.

[0053] refer to Figure 2A According to one embodiment, the camera module 100A may include an image sensor 111 and a 3D sensor 112 arranged side by side.

[0054] refer to Figure 2B According to one embodiment, the camera module 100B may include two image sensors 111a and 111b and a 3D sensor 112, which are arranged side by side in a triangular shape. For example, the two image sensors 111a and 111b may be positioned above the 3D sensor 112. In one embodiment, the two image sensors 111a and 111b may be positioned below the 3D sensor 112.

[0055] refer to Figure 2CAccording to one embodiment, a camera module 100C may include two image sensors 111a and 111b and a 3D sensor 112, which are arranged side by side in the same direction. For example, the 3D sensor 112 may be positioned between the image sensors 111a and 111b. Although the image sensors 111a and 111b and the 3D sensor 112 are arranged side by side in the row direction, they may also be arranged side by side in the column direction. For example, image sensor 111a may be positioned above the 3D sensor 112, while image sensor 111b may be positioned below the 3D sensor 112.

[0056] refer to Figure 2D and Figure 2E The camera modules 100D and 100E according to various embodiments may include a plurality of image sensors 111a, 111b and 111c arranged side by side in the column direction, and a 3D sensor 112 arranged adjacent to any one of the image sensors 111a, 111b and 111c in the row direction. For example, the image sensors 111a, 111b and 111c and the 3D sensor 112 may be arranged in a T-shape or an L-shape.

[0057] exist Figures 2B to 2E In this process, the positions of image sensors 111a, 111b, and 111c, as well as 3D sensor 112, can be varied. Furthermore, according to various embodiments of this disclosure, camera module 100 may include four or more image sensors 111 and / or two or more 3D sensors 112.

[0058] Figure 3A and Figure 3B The diagram schematically illustrates a top view and a longitudinal cross-sectional view of the configuration of a 3D sensor 112 according to an embodiment of the present disclosure.

[0059] refer to Figure 3A and Figure 3B According to one embodiment, the 3D sensor 112 may include a light source 20, a diffuser 30, and an actuator 40 mounted on a substrate 10.

[0060] The substrate 10 may include a PCB (printed circuit board), an FPCB (flexible PCB), glass, ceramic, or a board with circuit wiring such as a silicon layer. In one embodiment, the substrate 10 may include a PCB.

[0061] The light source 20 may include a semiconductor laser diode capable of emitting a vertical cavity surface emitting laser (VCSEL).

[0062] The diffuser 30 can emit laser light transmitted from the light source 20 to the outside. For example, the diffuser 30 can emit light at various angles from 80° to 150°. The diffuser 30 can have a fixed field of view (FOV) θ. To facilitate understanding of this disclosure, although other light angles may be used, it is assumed that the laser light L emitted from the light source 20 to the diffuser 30 is straight light. In one embodiment, the diffuser 30 can have a single layer. For example, when the diffuser 30 has multiple layers, it is difficult to obtain suitable 3D information due to the loss of laser light L. In other words, various embodiments of this disclosure relate to a diffuser 30 with a movable single layer.

[0063] Actuator 40 can move diffuser 30. For example, actuator 40 can adjust the distance D between light source 20 and diffuser 30 by moving diffuser 30 in the vertical direction.

[0064] The actuator 40 may include a housing 41 and a carrier 45. The housing 41 may enclose the light source 20, the diffuser 30, and the carrier 45. The housing 41 may protect the light source 20, the diffuser 30, and the carrier 45 from external physical and chemical damage. In one embodiment, the housing 41 may include one of metal, ceramic, plastic, or polymeric resin. The housing 41 may include a drive coil 42. The carrier 45 may support and move the diffuser 30. The carrier 45 may include a magnet 46. When a voltage is applied to the drive coil 42, the drive coil 42 in the housing 41 may be magnetic. The magnetic drive coil 42 may pull the magnet 46, and the carrier 45 may move up and down.

[0065] Figures 4A to 4C This is a diagram illustrating the vertical movement of a carrier 45 of a 3D sensor 112 according to one embodiment of the present disclosure across multiple levels. In this embodiment, the housing 41 may have three drive coils 42b, 42t, and 42m, thus allowing the carrier 45 to be positioned at three levels. In some embodiments, the actuator 40 may have a larger number of drive coils 42, 42b, 42t, and 42m.

[0066] refer to Figure 4A When the distance between the 3D sensor 112 and the object is closer, the diffuser 30 can be positioned closer to the light source 20. For example, when the distance between the camera module 100 and the object is short, the actuator controller 130 can provide voltage and current to the bottom coil 42b according to the autofocus control signal provided by the image sensor 111. The magnet 46 can be pulled by the magnetic bottom coil 42b, so the diffuser 30 can be positioned at its lowest horizontal position. The first distance D1 between the light source 20 and the diffuser 30 can be the minimum distance at which the diffuser 30 can be spaced from the light source 20. Therefore, the first FOV θ1 of the 3D sensor 112 can have the maximum angle. That is, the 3D sensor 112 can have the widest FOV θ1.

[0067] refer to Figure 4B When the distance between the 3D sensor 112 and the object is large, the diffuser 30 can be positioned away from the light source 20. For example, when the distance between the camera module 100 and the object is large, the actuator controller 130 can provide voltage and current to the top coil 42t according to the autofocus control signal provided by the image sensor 111. The magnet 46 can be pulled by the magnet top coil 42t, so the diffuser 30 can be positioned at its highest horizontal position. The second distance D2 between the light source 20 and the diffuser 30 can be the maximum distance that the diffuser 30 can be spaced from the light source 20. Therefore, the second FOVθ2 of the 3D sensor 112 can have the minimum angle. That is, the 3D sensor 112 can have the narrowest FOVθ2.

[0068] refer to Figure 4C When the distance between the 3D sensor 112 and the object is intermediate, the diffuser 30 can be positioned midway between the minimum distance (D1) and the maximum distance (D2) from the light source 20. For example, when the distance between the camera module 100 and the object is intermediate, the actuator controller 130 can apply voltage and current to the intermediate coil 42m based on the autofocus control signal provided by the image sensor 111. The magnet 46 can be pulled by the magnetic intermediate coil 42m, so the diffuser 30 can be positioned at an intermediate level between a low level and a high level. Therefore, the third FOV θ3 of the 3D sensor 112 can have an intermediate angle. That is, the 3D sensor 112 can have a median FOV θ3.

[0069] Figure 5A and Figure 5B This is a diagram showing how the FOV of the 3D sensor 112 changes according to the distance between the object Ob and the camera module 100.

[0070] refer to Figure 5A and Figure 5B When object Ob is relatively close to camera module 100, i.e., when the distance between object Ob and camera module 100 is relatively short, 3D sensor 112 can have a relatively large FOV. When object Ob is relatively far from camera module 100, i.e., when the distance between object Ob and camera module 100 is relatively long, 3D sensor 112 can have a relatively small FOV. As described, when the distance between object Ob and camera module 100 is relatively short, the density of the laser emitted from 3D sensor 112 can decrease, but 3D sensor 112 can have a wider FOV. Conversely, when the distance between object Ob and camera module 100 is relatively long, the FOV of 3D sensor 112 can narrow, but the density of the laser emitted from 3D sensor 112 can increase. For ease of understanding, Figure 5A and Figure 5B The size of the numerical aperture (NA) of the 3D sensor 112 is shown.

[0071] Figure 6A This is a flowchart illustrating a method for adjusting the FOV of a 3D sensor 112 according to an embodiment of the present disclosure.

[0072] Refer to Figure 1 to Figure 6A The method may include:

[0073] Image information of the object is captured using the first camera (i.e., image sensor 111) (S10).

[0074] The captured image information is provided to the preprocessor 121 (S20).

[0075] Based on image information, the preprocessor 121 generates an autofocus control signal to drive the diffuser 30 of the 3D sensor (S30).

[0076] The autofocus control signal is provided to the actuator controller 130 (S40).

[0077] The actuator controller 130 generates an actuator drive signal to drive the second camera (i.e., the diffuser 30 of the 3D sensor 112) (S50).

[0078] The actuator drive signal is provided to the actuator 40 (S60) of the 3D sensor 112, and

[0079] The FOV (S70) of the 3D sensor 112 is adjusted by moving the diffuser 30 using the actuator 40 based on the actuator drive signal.

[0080] Figure 6B This is a flowchart illustrating a method for obtaining a 3D image by adjusting the FOV of a 3D sensor 112 according to an embodiment of the present disclosure.

[0081] Refer to Figure 1 to Figure 6B The method may include:

[0082] Image information of the object is captured using the first camera (i.e., image sensor 111) (S10).

[0083] The captured image information is provided to the preprocessor 121 (S20).

[0084] Based on image information, the preprocessor 121 generates an autofocus control signal to drive the diffuser 30 of the 3D sensor (S30).

[0085] The autofocus control signal is provided to the actuator controller 130 (S40).

[0086] The actuator controller 130 generates an actuator drive signal to drive the second camera (i.e., the diffuser 30 of the 3D sensor 112) (S50).

[0087] The actuator drive signal is provided to the actuator 40 of the 3D sensor 112 (S60).

[0088] The FOV (S70) of the 3D sensor 112 is adjusted by moving the diffuser 30 using the actuator 40 based on the actuator drive signal.

[0089] The 3D sensor 112 acquires the 3D information of the object and provides the 3D information to the image signal processor 140 (S80).

[0090] 3D image data is generated by processing image information provided by image sensor 111 and 3D information provided by 3D sensor 112 using image signal processor 140 (S90).

[0091] Store and / or display 3D image data (S100).

[0092] Figure 7 This is a schematic diagram illustrating a camera system 1000 having a camera module 100 according to an embodiment of the present disclosure.

[0093] refer to Figure 7 According to one embodiment, the camera system 1000 may include the components shown in Figures 1 to 2. Figure 5B The camera module 100 shown in any one of them. Figure 7 In the illustration, the camera system 1000 is shown as a mobile phone as an example. According to various embodiments, the camera system 1000 may include, for example, a mobile phone, a tablet computer, a digital camera, a closed-circuit television system, or other various image acquisition systems. Figure 7 It shows Figure 2A The camera module 100A is shown. In some embodiments, the camera system 1000 may include... Figures 2B to 2E The various camera modules 100B to 100E are shown. Additionally, camera module 100A can be installed at various locations within the camera system 1000.

[0094] According to embodiments of this disclosure, since the laser emission angle of the 3D sensor is adjusted according to the distance between the camera module and the object, suitable 3D information can be obtained without increasing the laser power.

[0095] According to embodiments of this disclosure, the laser density of the 3D sensor can be kept relatively uniform even if the distance between the camera module and the object is changed.

[0096] According to embodiments of this disclosure, the FOV of a 3D sensor can be rapidly adjusted based on focus information from an image sensor.

[0097] According to embodiments of this disclosure, the structure of the camera module can be simplified because the FOV of the 3D sensor is adjusted based on the focus information of the image sensor.

[0098] Embodiments of the present disclosure have been described above with reference to the accompanying drawings. However, those skilled in the art to which this disclosure pertains will recognize that the invention may be implemented in other specific forms without altering its technical spirit or essential features. Therefore, although the present disclosure has been shown and described with respect to specific embodiments, the disclosed embodiments are not intended to be limiting.

[0099] Examples of embodiments have been disclosed herein, and although specific terminology has been used, such terminology is used and interpreted in a general and descriptive sense only and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise specifically indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of this disclosure as set forth in the appended claims.

Claims

1. A three-dimensional sensor included in a moving camera module, having a laser emission angle adjustment function based on an autofocus control signal from an image sensor included in the moving camera module, according to the distance between the moving camera module and an object, the three-dimensional sensor comprising: A light source is disposed on a substrate, wherein the light source is configured to emit laser light; A diffuser, disposed on the light source, wherein the diffuser is configured to emit laser light from the light source at different angles; and An actuator that selectively positions the diffuser. The actuator includes: Housing, which includes a drive coil; and A carrier, comprising a magnet, is configured to support and move the diffuser. The actuator moves the diffuser to one of a plurality of discrete positions in response to the autofocus control signal.

2. The three-dimensional sensor according to claim 1, wherein, The carrier is configured to move the diffuser to adjust the distance between the light source and the diffuser to at least three field-of-view levels.

3. The three-dimensional sensor according to claim 1, wherein, The housing surrounds the light source, the diffuser, and the carrier.

4. The three-dimensional sensor according to claim 1, wherein, The diffuser is spaced at various distances from the light source by the carrier.

5. The three-dimensional sensor according to claim 1, wherein, The substrate includes a printed circuit board.

6. The three-dimensional sensor according to claim 1, wherein, The light source includes a semiconductor laser diode.

7. The three-dimensional sensor according to claim 1, wherein, The drive coil includes a bottom drive coil configured to pull the carrier to a low position and a top drive coil configured to pull the carrier to a high position.

8. The three-dimensional sensor according to claim 1, wherein, The diffuser is a single layer and has a fixed laser emission angle.

Images