Three-dimensional camera module

Abstract

The invention provides a three-dimensional camera module. The three-dimensional camera module comprises a light emitting module used for emitting first light; the grating is used for transmitting the first light to generate second transmission light or diffracting the first light to generate third diffraction light; the flight time sensor is used for receiving and detecting fourth light reflected by the object after being irradiated by the second transmission light; the structured light sensor is used for receiving and detecting fifth light reflected by the object after being irradiated by the third diffraction light; the control circuit is used for controlling the grating to transmit or diffract the first light, and is used for obtaining a two-dimensional image of the object according to the fourth light and / or the fifth light and then generating a corresponding three-dimensional image. Through combination of the flight time sensor and the structured light sensor, a three-dimensional image which is closer to the three-dimensional camera module and is higher in resolution can be generated, and a three-dimensional image which is farther from the three-dimensional camera module can also be generated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional imaging, in particular to a three-dimensional camera module. BACKGROUND

[0002] With the continuous development and maturity of three-dimensional vision, three-dimensional stereo cameras have been widely used in automatic driving, high-end manufacturing, machine vision and other fields. Three-dimensional stereo cameras determine the position and depth information of objects by directly or indirectly detecting the flight time of light, or calculate the displacement of coded light spot patterns (relative to the light spots on the plane object) to obtain the position and depth information of the object. However, the above methods are difficult to balance image clarity and the shooting distance of the illuminated object limited by light attenuation. SUMMARY

[0003] The present application provides a three-dimensional camera module, comprising:

[0004] a light emitting module for emitting first light;

[0005] a grating located on the light path of the first light, for transmitting the first light to generate second transmitted light or diffracting the first light to generate third diffracted light, the number density of light beams of the third diffracted light being greater than that of the second transmitted light;

[0006] a time-of-flight sensor for receiving and detecting fourth light reflected by an object after being irradiated by the second transmitted light;

[0007] a structured light sensor for receiving and detecting fifth light reflected by the object after being irradiated by the third diffracted light;

[0008] a control circuit electrically connected to the light emitting module, the grating, the time-of-flight sensor and the structured light sensor, respectively, for controlling the grating to transmit or diffract the first light, and for generating a corresponding three-dimensional image after obtaining a two-dimensional image of the object according to the fourth light and / or the fifth light.

[0009] The grating in the three-dimensional camera module transmits the first light to generate second transmitted light, or diffracts the first light to generate third diffracted light. The second transmitted light is reflected by an object to generate fourth light, and the third diffracted light is reflected by the object to generate fifth light. The number density of the third diffracted light is greater than the number density of the second transmitted light, so that the number of light spots formed by the third diffracted light when the third diffracted light irradiates the surface of the object is greater than the number of light spots formed by the second transmitted light when the second transmitted light irradiates the surface of the object, so that the number density of the fifth light is greater than the number density of the fourth light. The third diffracted light is beneficial to obtain a three-dimensional image with higher resolution and higher clarity. The light intensity of the second transmitted light is greater than the light intensity of the third diffracted light, so that the second transmitted light can propagate to a farther distance. The second transmitted light is beneficial to enable the three-dimensional camera module to receive and detect light reflected by a distant object.

[0010] When the object is far away from the three-dimensional camera module, the reflected light intensity of the third diffracted light when the third diffracted light irradiates the object is weak, so that the fifth light is difficult to be received and detected by the structured light sensor to generate a three-dimensional image. At this time, the control circuit controls the grating to transmit the first light to generate the second transmitted light, so that the fourth light is received and detected by the time-of-flight sensor, and a three-dimensional image of the object is obtained according to the fourth light. When the object is close to the three-dimensional camera module, the reflected light intensity of the third diffracted light when the third diffracted light irradiates the object can enable the fifth light to be received and detected by the structured light sensor. Since the number density of the fifth light is greater than the number density of the fourth light, the clarity of the three-dimensional image of the object obtained according to the fifth light is higher than the clarity of the three-dimensional image of the object obtained according to the fourth light. Therefore, the control circuit controls the grating to diffract the first light to generate the third diffracted light, so that the structured light sensor receives and detects the fifth light, and a three-dimensional image of the object is generated according to the fifth light. Therefore, the three-dimensional camera module of the present application can obtain a three-dimensional image of the object when the distance between the three-dimensional camera module and the object is medium-close or far.

[0011] Further, the light emitting module includes an output collimating lens and a plurality of light sources arranged in an array. Each of the light sources is configured to emit laser light. The output collimating lens is configured to collimate the laser light to generate the first light.

[0012] Further, each of the light sources is a vertical cavity surface emitting laser or a light emitting diode.

[0013] Further, the three-dimensional camera module further comprises a first receiving lens and a second receiving lens, the first receiving lens is used for converging the second transmitted light reflected by the object to generate the fourth light, and the second receiving lens is used for converging the third diffracted light reflected by the object to generate the fifth light.

[0014] Further, the time-of-flight sensor is a direct time-of-flight sensor, which is used for measuring the time of flight of the fourth light to obtain a two-dimensional image of the object; or

[0015] the time-of-flight sensor is an indirect time-of-flight sensor, which is used for measuring the phase shift size of the fourth light to obtain a two-dimensional image of the object.

[0016] Further, when the time-of-flight sensor is a direct time-of-flight sensor, the time-of-flight sensor comprises a plurality of single-photon avalanche diodes and a time-to-digital converter.

[0017] Further, when the time-of-flight sensor is an indirect time-of-flight sensor, the time-of-flight sensor comprises an image sensor.

[0018] Further, when the three-dimensional camera module captures an object, if the distance between the object and the three-dimensional camera module ranges from 20 cm to 60 cm, the three-dimensional camera module uses the structured light sensor or the time-of-flight sensor to generate a three-dimensional image of the object.

[0019] Further, when the three-dimensional camera module captures an object, if the distance between the object and the three-dimensional camera module ranges from 60 cm to 8 m, the three-dimensional camera module uses the time-of-flight sensor to generate a three-dimensional image of the object. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 It is a structural diagram of the grating transmitting the first light to generate the second transmitted light in the three-dimensional camera module of the embodiment of the present application.

[0021] Figure 2 It is a structural diagram of the grating diffracting the first light to generate the third diffracted light in the three-dimensional camera module of the embodiment of the present application.

[0022] Figure 3 It is a structural diagram of the grating not receiving the first electrical signal. Figure 1 It is a planar structural diagram of the substrate.

[0023] Figure 4 It is another planar structural diagram of the substrate.

[0024] Figure 5 It is a structural diagram of the grating not receiving the first electrical signal. Figure 1

[0025] ​Figure 6 for Figure 1 A schematic diagram of the structure of the grating receiving the first electrical signal.

[0026] Figure 7 for Figure 1 A schematic diagram of the arrangement of light spots of the second transmitted light on the plane.

[0027] Figure 8 for Figure 2 A schematic diagram of the arrangement of light spots of the third diffracted light on a plane.

[0028] Figure 9 This is a schematic diagram illustrating the workflow of the 3D camera module in an embodiment of this application.

[0029] Explanation of main component symbols

[0030] 3D camera module 100

[0031] Substrate 10, 11, 12, 13

[0032] Optical emission module 20

[0033] Light source 21

[0034] Output collimating lens 22

[0035] First receiving lens 30

[0036] Second receiving lens 40

[0037] grating 50

[0038] Glass substrate 51

[0039] LCD 52

[0040] Transparent conductive layer 53

[0041] Time-of-flight sensor 60

[0042] Structured light sensor 70

[0043] Control circuit 80

[0044] First Light LS1

[0045] Second transmitted light LS2

[0046] First light spot LS21

[0047] Third diffraction light LS3

[0048] Second transmitted light spot LS31

[0049] Fourth Light LS4

[0050] Fifth Light LS5

[0051] The following detailed description, in conjunction with the accompanying drawings, will further illustrate this application. Detailed Implementation

[0052] The mainstream technologies for 3D stereo imaging include time-of-flight (TOF) and structured light. TOF determines the position and depth information of an object by directly or indirectly probing the time of flight of light, while structured light uses specific algorithms to calculate the displacement of an encoded light spot pattern to obtain the object's position and depth information. At this point, the control circuitry in the 3D camera receives the information, processes it, and generates a 3D image.

[0053] Please refer to the following: Figure 1 and Figure 2 The 3D camera module 100 of this application embodiment includes a substrate 10, a light emitting module 20, a time-of-flight sensor 60, a structured light sensor 70, a grating 50, and a control circuit 80. The light emitting module 20, the time-of-flight sensor 60, and the structured light sensor 70 are disposed on the same surface of the substrate 10, attached to the substrate 10, and electrically connected to the control module 80 through conductive terminals of the substrate 10. The grating 50 is located in the light path emitted by the light emitting module 20, and the control circuit 80 is located on the side of the substrate 10 away from the light emitting module 20 and is located within the substrate 10.

[0054] The optical emitting module 20 is used to emit a first light LS1, which is a laser. The optical emitting module 20 includes multiple light sources 21 arranged in an array and an output collimating lens 22. Each light source 21 emits a laser. Each light source 21 is a vertical-cavity surface-emitting laser (VCSEL) or a light-emitting diode (LED). The output collimating lens 22 is located on the side of each light source 21 emitting laser light and is used to receive and collimate the laser light to generate the first light LS1. In this embodiment, the output collimating lens 22 can be a single lens, a combination of lenses, a microlens array, or a Fresnel lens, or any combination thereof, optimized in design while considering the overall size of the module to improve the collimation and modulation effect. The laser light emitted by the light source is collimated by the output collimating lens 22 and emitted as the first light LS1.

[0055] The substrate 10 is obtained by selectively drilling, depositing, etching, and other processes on a silicon substrate to form the required circuit pattern. Multiple light sources 21, time-of-flight sensors 60, and structured light sensors 70 arranged in an array are positioned on the substrate 10 according to the layout of the circuit pattern and are electrically connected to the corresponding conductive terminals of the circuit pattern, thereby establishing an electrical connection between them.

[0056] Please see Figure 3 In this embodiment, the three-dimensional camera module 100 includes a complete substrate 10, and multiple light sources 21, time-of-flight sensors 60 and structured light sensors 70 arranged in an array are spaced apart on the same surface of the substrate 10 and electrically connected to conductive terminals on the substrate 10.

[0057] Please see Figure 4 In other embodiments of this application, the 3D camera module 100 includes three independent substrates, namely substrate 11, substrate 12, and 13. Circuit patterns are formed on substrates 11, 12, and 13 respectively. A plurality of light sources 21 arranged in an array in the light emitting module 20 are located on one surface of substrate 12 and electrically connected to the circuit patterns on substrate 12. A time-of-flight sensor 60 is located on one surface of substrate 11 and electrically connected to the circuit patterns on substrate 11. A structured light sensor 70 is located on one surface of substrate 13 and electrically connected to the circuit patterns on substrate 13.

[0058] This application does not limit the relative positions of the array of multiple light sources 21, time-of-flight sensor 60 and structured light sensor 70 in the light emission module 20. Depending on the application scenario of the 3D camera module 100, the light emission module 20, time-of-flight sensor 60 and structured light sensor 70 can be disposed on the same substrate 10 or disposed on different substrates 11, 12 and 13 respectively.

[0059] Grating 50 is located in the optical path of the first light LS1, and grating 50 is an adjustable voltage liquid crystal grating. Please refer to the following: Figure 5 and Figure 6 The grating 50 includes two parallel and spaced-apart glass substrates 51, and a liquid crystal 52 filled between the two glass substrates 51. The liquid crystal 52 has a molecular orientation that responds to a first electrical signal. A transparent conductive layer 53 is coated on the side of each glass substrate 51 closest to the liquid crystal 52, and the transparent conductive layers 53 are periodically arranged on the glass substrates 51. That is, the grating 50 has a periodic electrode structure. The transparent conductive layers 53 include transparent conductive materials such as indium tin oxide (ITO) or aluminum-doped zinc oxide (AZO). The two transparent conductive layers 53 are electrically connected to the control circuit 80. After receiving the first electrical signal transmitted by the control circuit 80, a voltage difference is generated between the two transparent conductive layers 53 to form a periodic electric field, causing the molecules in the liquid crystal 52 located between the two transparent conductive layers 53 to undergo periodic changes in molecular orientation under the action of the electric field. When the grating 50 does not receive the first electrical signal, the grating 50 transmits the first light LS1 to generate the second transmitted light LS2; when the grating 50 receives the first electrical signal, the grating 50 diffracts the first light LS1 to generate the third diffracted light LS3.

[0060] Please see Figure 7 After the grating 50 transmits the first light LS1, it generates a second transmitted light LS2. When the second transmitted light LS2 illuminates the surface of the planar object, it forms an array of multiple first light spots LS21 on the surface of the object. Each first light spot LS21 corresponds to a beam of second transmitted light LS2 propagating parallel to the same direction. The light spot arrangement pattern formed by all the second transmitted light LS2 illuminating the surface of the planar object is the same as the light spot arrangement pattern of the first light LS1, and the beam number density of the second transmitted light LS2 is the same as the beam number density of the first light LS1.

[0061] Please see Figure 8 After the first light LS1 is diffracted by the grating 50, a third diffracted light LS3 is generated. Due to the diffraction effect, after the first light LS1 is diffracted, multiple third diffracted lights LS3 are generated. This causes a first light spot LS21 to be diffracted and generate multiple second transmitted light spots LS31. At this time, the light spot density in the light spot arrangement pattern of the third diffracted light LS3 is greater than the light spot density in the light spot arrangement pattern of the first light LS1. The number density of the third diffracted light LS3 is greater than the number density of the first light LS1. Therefore, the number density of the third diffracted light LS3 is greater than the number density of the second transmitted light LS2.

[0062] Please refer to the following: Figure 1 The 3D camera module 100 also includes a first receiving lens 30 and a second receiving lens 40. The first receiving lens 30 is used to converge the second transmitted light LS2 reflected from the object to generate a fourth light LS4, so that all the fourth light LS4 forms a light spot pattern corresponding to the object's surface structure. The time-of-flight sensor 60 receives the fourth light LS4 and generates depth information of the object's surface through circuit calculations. The second receiving lens 40 is used to converge the third diffracted light LS3 reflected from the object to generate a fifth light LS5, so that all the fifth light LS5 forms a light spot pattern corresponding to the object's surface. The structured light sensor 70 receives and detects the fifth light LS5 and generates depth information of the object's surface through a triangulation algorithm.

[0063] The light emitting module 20 continuously sends light pulses or modulated light sources to the target. The time-of-flight sensor 60 then receives and detects the light returning from the object. By detecting the flight (round-trip) time of the light pulses or modulated light sources, the distance between the 3D camera module 100 and the object is obtained. The time-of-flight sensor 60 can be a direct time-of-flight (dToF) sensor or an indirect time-of-flight (iToF) sensor. If the time-of-flight sensor 60 is a direct time-of-flight sensor, it measures the flight time of the fourth light LS4 to obtain a two-dimensional image of the object. If the time-of-flight sensor 60 is an indirect time-of-flight sensor, it measures the phase shift of the fourth light LS4 to obtain a two-dimensional image of the object.

[0064] The dToF sensor works by emitting light pulses towards an object and directly calculating the object's depth by measuring the time interval between the reflected and emitted light pulses. A dToF sensor includes multiple single-photon avalanche diodes (SPADs) and a time-to-digital converter (TDC).

[0065] The working principle of an iToF sensor is to modulate the emitted light into a periodic signal of a certain frequency, measure the phase difference between the emitted signal and the one reflected back to the receiver from the object, and indirectly calculate the time of flight. That is, the time of flight of light is indirectly measured by measuring the phase shift difference, rather than directly measuring the time of flight of light.

[0066] The structured light sensor 70 works by calculating the displacement distance of the returned coded light spot pattern using a specific algorithm to determine the position and depth information of the object. After receiving and detecting the fifth light LS5, the structured light sensor 70 calculates the distance between the object and the 3D camera module 100 based on the displacement distance of the coded light spot pattern formed by the fifth light LS5.

[0067] The control circuit 80 includes multiple wires (not shown), each wire passing through the substrate 10 to electrically connect to the light emitting module 20, the grating 50, the time-of-flight sensor 60, and the structured light sensor 70 respectively through the circuit pattern on the substrate 10. That is, the control circuit 80 is electrically connected to the light emitting module 20, the grating 50, the time-of-flight sensor 60, and the structured light sensor 70 respectively. The control circuit 80 controls the transmission of a second electrical signal, causing the light emitting module 20 to emit first light LS1 after receiving the second electrical signal. The wires in the control circuit 80 are electrically connected to the two transparent conductive layers 53 in the grating 50, thereby controlling whether the grating 50 receives the first electrical signal, causing the grating 50 to transmit or diffract the first light LS1. When the grating 50 receives the first electrical signal, a voltage difference is generated between the two transparent conductive layers 53 to form a periodic electric field direction, causing the molecules in the liquid crystal 52 to change orientation under the action of the electric field. When the grating 50 does not receive the first electrical signal, there is no electric field between the two transparent conductive layers 53, and the molecular orientation within the liquid crystal 52 remains unchanged. When the grating 50 transmits the first light LS1 to generate the second transmitted light LS2, the control circuit 80 controls the time-of-flight sensor 60 to receive and detect the fourth light LS4 reflected by the object according to the second transmitted light LS2, and acquires a three-dimensional image of the object based on the fourth light LS4. When the grating diffracts the first light LS1 to generate the third diffracted light LS3, the control circuit 80 controls the structured light sensor 70 or the time-of-flight sensor 60 to receive and detect the fifth light LS5 reflected by the object according to the third diffracted light LS3, and acquires a two-dimensional image of the light spot on the object based on the fifth light LS5 to further generate the corresponding three-dimensional image.

[0068] Please see Figure 9 The operation of the 3D camera module 100 in this embodiment is as follows: When the 3D camera module 100 captures an object, if the distance between the object and the 3D camera module 100 is between 20cm and 60cm, the control circuit 80 controls the grating 50 to receive a first electrical signal, causing the grating 50 to diffract a first light LS1 to generate a third diffracted light LS3. After the object reflects the third diffracted light LS3 to generate a fifth light LS5, the 3D camera module 100 uses the time-of-flight sensor 60 or the structured light sensor 70 to receive and detect the fifth light LS5, and then generates a 3D image of the object based on the light propagation time or a triangulation algorithm. If the distance between the object and the 3D camera module 100 is between 60cm and 8m, the control circuit 80 controls the grating 50 not to receive the first electrical signal, causing the grating 50 to transmit the first light LS1 to generate a second transmitted light LS2. After the object reflects the second transmitted light LS2 to generate a fourth light LS4, the 3D camera module 100 uses the time-of-flight sensor 60 to receive and detect the fourth light LS4, and then generates a 3D image of the object based on the light propagation time.

[0069] In summary, in the 3D camera module 100 of this application, the grating 50 transmits a first light LS1 to generate a second transmitted light LS2, or diffracts the first light LS1 to generate a third diffracted light LS3. The second transmitted light LS2 is reflected by an object to generate a fourth light LS4, and the third diffracted light LS3 is reflected by an object to generate a fifth light LS5. The beam number density of the third diffracted light LS3 is greater than that of the second transmitted light LS2 formed by direct transmission. Therefore, the number of light spots formed when the third diffracted light LS3 illuminates the object surface is greater than that when the second transmitted light LS2 illuminates the object surface, making the beam number density of the fifth light LS5 greater than that of the fourth light LS4. The diffracted third diffracted light LS3 is beneficial for obtaining higher resolution and clearer 3D images. Meanwhile, the light intensity of the second transmitted light LS2 formed by direct transmission is greater than that of the third diffracted light LS3 formed by diffraction. Therefore, the second transmitted light LS2 with higher intensity can propagate to a greater distance, which is beneficial for long-distance imaging.

[0070] When the distance between the object and the 3D camera module 100 is far, the intensity of the third diffracted light LS3 is weak when it illuminates the object, making it difficult for the fifth light LS5 to be received and detected by the structured light sensor 70 to generate a 3D image. In this case, the control circuit 80 controls the grating 50 to transmit the first light LS1 to generate the second transmitted light LS2, allowing the time-of-flight sensor 60 to receive and detect the fourth light LS4, thereby acquiring a 3D image of the object based on the fourth light LS4. When the distance between the object and the 3D camera module 100 is medium to close, the intensity of the third diffracted light LS3 is sufficient for the fifth light LS5 to be received and detected by the structured light sensor 70. Because the beam number density of the fifth light LS5 is greater than that of the fourth light LS4, the clarity of the 3D image of the object acquired by the control circuit 80 based on the fifth light LS5 is higher than that acquired based on the fourth light LS4. Therefore, the control circuit 80 controls the grating 50 to diffract the first light LS1 to generate the third diffracted light LS3, so that the structured light sensor 70 receives and detects the fifth light LS5, thereby acquiring a three-dimensional image of the object based on the fifth light LS5. Thus, it can be seen that the three-dimensional camera module 100 of this application can acquire a three-dimensional image of the object when the distance between the object and the module is at a medium or long distance.

[0071] In this article, "near to medium distance" refers to the distance between the 3D camera module 100 and the object ranging from 20cm to 60cm, including the endpoint values ​​of 20cm and 60cm, and "far distance" refers to the distance between the 3D camera module 100 and the object ranging from 60cm to 8m, including the endpoint values ​​of 60cm and 8m.

[0072] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. A three-dimensional camera module, characterized by, The three-dimensional camera module comprises: a light emitting module configured to emit a first light; a grating disposed in a light path of the first light, and configured to transmit the first light to generate a second transmitted light or diffract the first light to generate a third diffracted light, the third diffracted light having a greater number density of light beams than the second transmitted light; a time-of-flight sensor configured to receive and detect a fourth light reflected by an object under irradiation of the second transmitted light; a structured light sensor configured to receive and detect a fifth light reflected by the object under irradiation of the third diffracted light; a control circuit electrically connected to the light emitting module, the grating, the time-of-flight sensor and the structured light sensor, respectively, and configured to control the grating to transmit or diffract the first light, and configured to generate a three-dimensional image of the object based on a two-dimensional image of the object obtained from the fourth light and / or the fifth light.

2. The 3D camera module of claim 1, wherein, The light emitting module comprises an output collimating lens and a plurality of light sources arranged in an array, each of the light sources being configured to emit a laser light, and the output collimating lens being configured to collimate the laser light to generate the first light.

3. The 3D camera module of claim 4, wherein, Each of the light sources is a vertical cavity surface emitting laser or a light emitting diode.

4. The 3D camera module of claim 1, wherein, The three-dimensional camera module further comprises a first receiving lens and a second receiving lens, the first receiving lens being configured to converge the second transmitted light reflected by the object to generate the fourth light, and the second receiving lens being configured to converge the third diffracted light reflected by the object to generate the fifth light.

5. The three-dimensional camera module of claim 1, wherein: the time-of-flight sensor is a direct time-of-flight sensor configured to measure a time of flight of the fourth light to obtain the two-dimensional image of the object; or the time-of-flight sensor is an indirect time-of-flight sensor configured to measure a phase shift of the fourth light to obtain the two-dimensional image of the object.

6. The 3D camera module of claim 5, wherein, When the time-of-flight sensor is the direct time-of-flight sensor, the time-of-flight sensor comprises a plurality of single photon avalanche diodes and a time-to-digital converter.

7. The 3D camera module of claim 5, wherein, When the time-of-flight sensor is the indirect time-of-flight sensor, the time-of-flight sensor comprises an image sensor.

8. The 3D camera module of claim 1, wherein, When the three-dimensional camera module captures an object, if a distance between the object and the three-dimensional camera module is in a range of 20 cm to 60 cm, the three-dimensional camera module generates a three-dimensional image of the object using the structured light sensor or the time-of-flight sensor.

9. The 3D camera module of claim 1, wherein, When the three-dimensional camera module captures an object, if a distance between the object and the three-dimensional camera module is in a range of 60 cm to 8 m, the three-dimensional camera module generates a three-dimensional image of the object using the time-of-flight sensor.

Images