A TOF depth sensing module and image generation method

Abstract

The present application provides a TOF depth sensing module and an image generation method. The TOF depth sensing module includes: a laser light source, a polarization filter device, a beam shaping device, an optical element, a receiving unit and a control unit. Among them, the laser light source is used to generate a laser beam; the polarization filter device is used to filter the laser beam to obtain a beam of a single polarization state; the beam shaping device is used to adjust the FOV of the beam of a single polarization state to obtain a first beam with a FOV range of [5°×5°, 20°×20°]; the control unit is used to control the optical element to control the direction of the first beam to obtain an outgoing beam; the control unit is also used to control the optical element to deflect the reflected beam obtained by the target object reflecting the outgoing beam to the receiving unit. The embodiment of the present application can improve the spatial resolution of the depth map of the target object finally obtained.

Description

Technical Field

[0001] The present application relates to the field of TOF technology, and more specifically, to a TOF depth sensing module and an image generation method. Background Art

[0002] Time of flight (TOF) technology is a commonly used depth or distance measurement technology. Its basic principle is to emit continuous light or pulsed light through the transmitter. When the continuous light or pulsed light hits the object to be measured, it will be reflected, and then the receiver receives the reflected light of the object to be measured. Next, by determining the flight time of the light from the transmitter to the receiver, the distance or depth of the object to be measured to the TOF system can be calculated.

[0003] In traditional solutions, pulsed TOF technology is often used to measure distance. Pulse-type TOF technology measures distance by measuring the time difference between the emission time of the outgoing light beam (emitted by the transmitting end) and the reception time of the reflected light beam (received by the receiving end). Specifically, in pulsed TOF technology, the light source generally emits a pulse light beam with a very short duration, which is reflected by the target and received by the photoelectric detector at the receiving end. By measuring the time interval between the emission pulse and the reception pulse, the depth or distance of the target can be obtained.

[0004] Pulse TOF technology requires that the sensitivity of the photodetector be high enough to achieve the ability to detect single photons. The commonly used photodetector is a single-photon avalanche diode (SPAD). Due to the complex interface and processing circuit of SPAD, the resolution of commonly used SPAD sensors is low and is insufficient to meet the requirements of high spatial resolution of depth sensing. Summary of the invention

[0005] The present application provides a TOF depth sensing module and an image generation method to improve the spatial resolution of a depth map ultimately generated by the TOF depth sensing module.

[0006] In a first aspect, a TOF depth sensing module is provided, which includes a laser light source, a polarization filter device, a beam shaping device, an optical element, a receiving unit and a control unit, wherein the laser light source can generate light in multiple polarization states, and the polarization filter device is located between the laser light source and the beam shaping device.

[0007] The functions of each module or unit in the above TOF depth sensing module are as follows:

[0008] The laser light source is used to generate a laser beam;

[0009] The polarization filter device is used to filter the laser beam to obtain a beam with a single polarization state;

[0010] The beam shaping device is used to adjust the FOV of the light beam in a single polarization state to obtain a first light beam;

[0011] The control unit is used to control the optical element to control the direction of the first light beam to obtain an outgoing light beam;

[0012] The control unit is also used to control the optical element to deflect the reflected light beam obtained by the target object reflecting the outgoing light beam to the receiving unit.

[0013] The above optical element is shared by the transmitting end and the receiving end of the TOF depth sensing module.

[0014] The multiple polarization states may include linear polarization, left-hand circular polarization and right-hand circular polarization, and the single polarization state may be any one of linear polarization, left-hand circular polarization and right-hand circular polarization.

[0015] The FOV range of the first light beam includes [5°×5°, 20°×20°].

[0016] The above-mentioned receiving unit may include a receiving lens and a sensor. The receiving lens may converge the reflected light beam to the sensor so that the sensor can receive the reflected light beam, and then obtain the moment when the reflected light beam is received by the receiving unit, and obtain the TOF corresponding to the outgoing light beam. Finally, a depth map of the target object may be generated according to the TOF corresponding to the outgoing light beam.

[0017] Optionally, the control unit is used to: control the optical element to control the direction of the first light beam at M different moments respectively, to obtain M outgoing light beams in different directions, wherein the total FOV covered by the M outgoing light beams in different directions includes [50°×50°, 80°×80°]; control the optical element to deflect M reflected light beams obtained by the target object reflecting the light beam from the optical element at M different moments to the receiving unit.

[0018] The light beams from the optical element at the M different moments may be light beams emitted in M ​​different directions obtained by the optical element controlling the directions of the first light beam at M different moments.

[0019] In an embodiment of the present application, the FOV of the light beam is adjusted by a beam shaping device so that the first light beam has a larger FOV. At the same time, scanning is performed in a time-division multiplexing manner (the optical element emits light beams in different directions at different times), which can improve the spatial resolution of the depth map of the target object finally obtained.

[0020] Optionally, the control unit is used to adjust the birefringence parameters of the optical element to obtain adjusted birefringence parameters; and the optical element is used to adjust the direction of the first light beam based on the adjusted birefringence parameters to obtain an emergent light beam.

[0021] Wherein, when the birefringence of the optical element is different, the optical element can adjust the first light beam to different directions.

[0022] In combination with the first aspect, in certain implementations of the first aspect, the control unit is further used to: generate a depth map of the target object according to TOFs corresponding to the outgoing light beams in M ​​different directions.

[0023] The TOF corresponding to the above-mentioned M outgoing light beams in different directions may specifically refer to the time difference information between the moment when the reflected light beam corresponding to the above-mentioned M outgoing light beams in different directions is received by the receiving unit and the moment when the above-mentioned M outgoing light beams in different directions are emitted.

[0024] Assuming that the M outgoing light beams in different directions include the outgoing light beam 1, the reflected light beam corresponding to the outgoing light beam 1 may refer to a light beam generated after the outgoing light beam 1 reaches the target object and is reflected after passing through the target object.

[0025] In combination with the first aspect, in some implementations of the first aspect, the optical element is a rotating mirror device, wherein the rotating mirror device controls the emission direction of the outgoing light beam by rotating.

[0026] Optionally, the rotating mirror device is a micro-electromechanical system galvanometer or a multi-faceted rotating mirror.

[0027] In combination with the first aspect, in some implementations of the first aspect, the optical element is a liquid crystal polarization element.

[0028] In combination with the first aspect, in some implementations of the first aspect, the optical element includes: a transverse polarization control plate, a transverse liquid crystal polarization grating, a longitudinal polarization control plate and a longitudinal liquid crystal polarization grating.

[0029] In combination with the first aspect, in certain implementations of the first aspect, in the above-mentioned optical element, the distances between the transverse polarization control plate, the transverse liquid crystal polarization grating, the longitudinal polarization control plate and the longitudinal liquid crystal polarization grating and the laser light source increase successively, or the distances between the longitudinal polarization control plate, the longitudinal liquid crystal polarization grating, the transverse polarization control plate and the transverse liquid crystal polarization grating and the laser light source increase successively.

[0030] In combination with the first aspect, in certain implementations of the first aspect, the beam shaping device is composed of a diffusion lens and a rectangular aperture.

[0031] In combination with the first aspect, in certain implementations of the first aspect, the TOF depth sensing module further includes: a collimating lens, which is located between the laser light source and the polarization filter device, and is used to collimate the laser light beam; and the polarization filter device is used to filter the light beam collimated by the collimating lens to obtain a light beam with a single polarization state.

[0032] In combination with the first aspect, in certain implementations of the first aspect, the TOF depth sensing module further includes: a collimating lens, which is located between the polarization filter device and the beam shaping device, and is used to collimate a light beam in a single polarization state; and the beam shaping device is used to adjust the FOV after the collimation process by the collimating lens to obtain a first light beam.

[0033] The above-mentioned collimation process of the light beam by the collimating lens can obtain a nearly parallel light beam, which can increase the power density of the light beam and further improve the effect of subsequent scanning using the light beam.

[0034] Optionally, the clear aperture of the collimating lens is less than or equal to 5 mm.

[0035] Since the size of the collimating lens is relatively small, the TOF depth sensing module including the collimating lens can be easily integrated into the terminal device, which can reduce the space occupied in the terminal device to a certain extent.

[0036] In combination with the first aspect, in some implementations of the first aspect, the laser light source is a vertical cavity surface emitting laser (VCSEL).

[0037] Optionally, the laser light source is a Fabry-Perot laser (can be referred to as FP laser for short).

[0038] Compared with a single VCSEL, a single FP laser can achieve greater power, and its electro-optical conversion efficiency is also higher than that of VCSEL, which can improve the scanning effect of the TOF depth sensing module.

[0039] Optionally, the wavelength of the laser light beam emitted by the above laser light source is greater than 900 nm.

[0040] Since the intensity of light greater than 900nm in sunlight is relatively weak, when the wavelength of the laser beam is greater than 900nm, it helps to reduce the interference caused by sunlight, thereby improving the scanning effect of the TOF depth sensing module.

[0041] Optionally, the wavelength of the laser light beam emitted by the above laser light source is 940nm or 1550nm.

[0042] Since the intensity of light near 940nm or 1550nm in sunlight is relatively weak, when the wavelength of the laser beam is 940nm or 1550nm, the interference caused by sunlight can be greatly reduced, thereby improving the scanning effect of the TOF depth sensing module.

[0043] In combination with the first aspect, in certain implementations of the first aspect, the light emitting area of ​​the laser light source is less than or equal to 5×5 mm 2 ; The area of ​​the beam incident end face of the above beam splitter is less than 5×5mm 2 ; The aperture of the above-mentioned collimating lens is less than or equal to 5mm.

[0044] Since the above-mentioned laser light source, beam splitter and collimating lens are small in size, the TOF depth sensing module including the above-mentioned devices (laser light source, beam splitter and collimating lens) is easier to be integrated into the terminal device, which can reduce the space occupied in the terminal device to a certain extent.

[0045] In combination with the first aspect, in certain implementations of the first aspect, the average output optical power of the TOF depth sensing module is less than 800mw.

[0046] When the average output optical power of the TOF depth sensing module is less than or equal to 800mw, the power consumption of the TOF depth sensing module is relatively small, and it is convenient to be installed in devices that are sensitive to power consumption, such as terminal devices.

[0047] In a second aspect, an image generation method is provided. The image generation method is applied to a terminal device including the TOF depth sensing module in the first aspect. The image generation method includes:

[0048] Controlling the laser light source to generate a laser beam;

[0049] The laser beam is filtered by a polarization filter device to obtain a beam with a single polarization state;

[0050] Using a beam shaping device to adjust a light beam in a single polarization state to obtain a first light beam;

[0051] Controlling the optical element to control the direction of the first light beam from the beam shaping device at M different times respectively, to obtain outgoing light beams in M ​​different directions;

[0052] Controlling the optical element to deflect M reflected light beams obtained by the target object reflecting outgoing light beams in M ​​different directions to a receiving unit;

[0053] A depth map of the target object is generated according to the TOFs corresponding to the outgoing light beams in M ​​different directions.

[0054] Wherein, the single polarization state is one of multiple polarization states;

[0055] For example, the multiple polarization states may include linear polarization, left-hand circular polarization and right-hand circular polarization, and the single polarization state may be any one of linear polarization, left-hand circular polarization and right-hand circular polarization.

[0056] The FOV range of the above-mentioned first light beam includes [5°×5°, 20°×20°]; the total FOV range covered by the above-mentioned M outgoing light beams in different directions includes [50°×50°, 80°×80°].

[0057] Optionally, the method further includes: acquiring TOFs corresponding to M outgoing light beams in different directions.

[0058] Optionally, the obtaining of TOFs respectively corresponding to the outgoing light beams in M ​​different directions includes: determining the TOFs respectively corresponding to the outgoing light beams in M ​​different directions according to the moments when the reflected light beams corresponding to the outgoing light beams in M ​​different directions are received by the receiving unit and the moments when the outgoing light beams in M ​​different directions are emitted.

[0059] The TOF corresponding to the above-mentioned M outgoing light beams in different directions may specifically refer to the time difference information between the moment when the reflected light beam corresponding to the above-mentioned M outgoing light beams in different directions is received by the receiving unit and the moment when the above-mentioned M outgoing light beams in different directions are emitted.

[0060] In an embodiment of the present application, the FOV of the light beam is adjusted by a beam shaping device so that the first light beam has a larger FOV. At the same time, scanning is performed in a time-division multiplexing manner (the optical element emits light beams in different directions at different times), which can improve the spatial resolution of the depth map of the target object finally obtained.

[0061] Optionally, the above-mentioned control optical element controls the direction of the first light beam from the beam shaping device at M different moments to obtain M outgoing light beams in different directions, including: adjusting the birefringence parameters of the optical element at M different moments to obtain the adjusted birefringence parameters corresponding to the M different moments, so that the optical element adjusts the direction of the first light beam based on the adjusted birefringence parameters at M different moments to obtain M outgoing light beams in different directions.

[0062] In combination with the second aspect, in certain implementations of the second aspect, the above-mentioned generating a depth map of the target object according to the TOFs corresponding to the outgoing light beams in M ​​different directions includes: determining the distances between M regions of the target object and the TOF depth sensing module according to the TOFs corresponding to the outgoing light beams in M ​​different directions; generating depth maps of the M regions of the target object according to the distances between the M regions of the target object and the TOF depth sensing module; and synthesizing the depth map of the target object according to the depth maps of the M regions of the target object.

[0063] In combination with the second aspect, in certain implementations of the second aspect, the above-mentioned use of a beam shaping device to adjust a light beam in a single polarization state to obtain a first light beam includes: using a beam shaping device to adjust the angular spatial intensity distribution of the light beam in a single polarization state to obtain the first light beam.

[0064] In a third aspect, a terminal device is provided, which includes the TOF depth sensing module in the first aspect.

[0065] The terminal device of the third aspect mentioned above can execute the image generating method of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is a schematic diagram of the principle of laser radar ranging;

[0067] Figure 2 is a schematic diagram of using the TOF depth sensing module of an embodiment of the present application to perform distance measurement;

[0068] Figure 3 is a schematic block diagram of a TOF depth sensing module according to an embodiment of the present application;

[0069] Figure 4 It is a schematic diagram of VCSEL;

[0070] Figure 5 is a schematic diagram of an array light source;

[0071] Figure 6 It is a schematic diagram of using a beam splitter to split a light beam emitted by an array light source;

[0072] Figure 7 It is a schematic diagram of a projection area obtained after a beam of light emitted by an array light source is split by a beam splitter;

[0073] Figure 8 It is a schematic diagram of a projection area obtained after a beam of light emitted by an array light source is split by a beam splitter;

[0074] Fig. 9It is a schematic diagram of a projection area obtained after a beam of light emitted by an array light source is split by a beam splitter;

[0075] Fig.10 It is a schematic diagram of a projection area obtained after a beam of light emitted by an array light source is split by a beam splitter;

[0076] Fig.11 is a schematic structural diagram of a TOF depth sensing module according to an embodiment of the present application;

[0077] Fig.12 is a schematic diagram of a beam splitter performing beam splitting processing;

[0078] Fig.13 is a schematic structural diagram of a TOF depth sensing module according to an embodiment of the present application;

[0079] Fig.14 is a schematic structural diagram of a TOF depth sensing module according to an embodiment of the present application;

[0080] Fig.15 A schematic diagram of the operation of the TOF depth sensing module according to an embodiment of the present application;

[0081] Fig.16 is a schematic diagram of a light emitting area of ​​an array light source;

[0082] Fig.17 A beam splitter is used to Fig.16 Schematic diagram of beam splitting processing of light beams emitted by an array light source shown;

[0083] Fig.18 is a schematic flow chart of an image generation method according to an embodiment of the present application;

[0084] Fig.19 is the depth map of the target object at time t0-t3;

[0085] Fig. 20 is a schematic flow chart of an image generation method according to an embodiment of the present application;

[0086] Fig.21 is a schematic flow chart of an image generation method according to an embodiment of the present application;

[0087] Fig. 22 is a schematic flow chart of obtaining a final depth map of a target object in a first working mode;

[0088] Fig.23 is a schematic flow chart of obtaining a final depth map of a target object in a first working mode;

[0089] Fig.24 is a schematic flow chart of obtaining a final depth map of a target object in a second working mode;

[0090] Fig.25 is a schematic flow chart of obtaining a final depth map of a target object in a second working mode;

[0091] Fig.26 is a schematic diagram of using the TOF depth sensing module of an embodiment of the present application to perform distance measurement;

[0092] Fig. 27 is a schematic block diagram of a TOF depth sensing module according to an embodiment of the present application;

[0093] Fig.28 is a schematic diagram of the spatial angle of the laser beam;

[0094] Fig.29 is a schematic block diagram of a TOF depth sensing module according to an embodiment of the present application;

[0095] Fig.30 is a schematic diagram of a TOF depth sensing module scanning a target object according to an embodiment of the present application;

[0096] Fig.31 is a schematic diagram of a scanning trajectory of a TOF depth sensing module according to an embodiment of the present application;

[0097] Fig.32 is a schematic diagram of a scanning method of a TOF depth sensing module according to an embodiment of the present application;

[0098] Fig.33 is a schematic block diagram of a TOF depth sensing module according to an embodiment of the present application;

[0099] Fig.34 is a schematic block diagram of a TOF depth sensing module according to an embodiment of the present application;

[0100] Fig.35 is a schematic structural diagram of a liquid crystal polarization grating according to an embodiment of the present application;

[0101] Fig.36 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0102] Fig.37 is a schematic diagram of changing the physical properties of a liquid crystal polarization grating by a periodic control signal;

[0103] Fig.38 It is a schematic diagram of a liquid crystal polarization grating controlling the direction of an input light beam;

[0104] Fig.39 is a schematic diagram of the voltage signal applied to the liquid crystal polarization grating;

[0105] Fig.40is a schematic diagram of a scanning trajectory of a TOF depth sensing module according to an embodiment of the present application;

[0106] Fig.41 is a schematic diagram of the area to be scanned;

[0107] Fig.42 is a schematic diagram of the area to be scanned;

[0108] Fig.43 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0109] Fig.44 It is a schematic diagram of the electro-optical crystal controlling the direction of the light beam;

[0110] Fig.45 is a schematic diagram of the voltage signal applied to the electro-optical crystal;

[0111] Fig.46 is a schematic diagram of a scanning trajectory of a TOF depth sensing module according to an embodiment of the present application;

[0112] Fig.47 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0113] Fig.48 It is a schematic diagram of the control of the direction of the light beam by the acousto-optic device;

[0114] Fig.49 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0115] Fig.50 This is a schematic diagram of the OPA device controlling the direction of the light beam;

[0116] Fig.51 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0117] Fig.52 is a schematic flow chart of an image generation method according to an embodiment of the present application;

[0118] Fig.53 is a schematic diagram of using the TOF depth sensing module of an embodiment of the present application to perform distance measurement;

[0119] Fig.54 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0120] Fig.55 is a schematic block diagram of a TOF depth sensing module according to an embodiment of the present application;

[0121] Fig.56 is a schematic block diagram of a TOF depth sensing module according to an embodiment of the present application;

[0122] Fig.57 is a schematic flow chart of an image generation method according to an embodiment of the present application;

[0123] Fig.58 is a schematic block diagram of a TOF depth sensing module according to an embodiment of the present application;

[0124] Fig.59 is a schematic block diagram of a TOF depth sensing module according to an embodiment of the present application;

[0125] Fig.60 is a schematic block diagram of a TOF depth sensing module according to an embodiment of the present application;

[0126] Fig.61 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0127] Fig.62 is a schematic flow chart of an image generation method according to an embodiment of the present application;

[0128] Fig.63 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0129] Fig.64 is a schematic structural diagram of a liquid crystal polarizing device according to an embodiment of the present application;

[0130] Fig.65 It is a schematic diagram of the control timing;

[0131] Fig.66 It is the timing diagram of the voltage drive signal;

[0132] Fig.67 It is a schematic diagram of the scanning area of ​​the TOF depth sensor module at different times;

[0133] Fig.68 It is a schematic diagram of the depth map corresponding to the target object at time t0-t3;

[0134] Fig.69 is a schematic diagram of the final depth map of the target object;

[0135] Fig.70 This is a schematic diagram of the TOF depth sensing module of the embodiment of the present application when it is working;

[0136] Fig.71 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0137] Fig.72 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0138] Fig.73 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0139] Fig.74 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0140] Fig.75 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0141] Fig.76 is a schematic structural diagram of a TOF depth sensing module 500 according to an embodiment of the present application;

[0142] Fig.77 is a schematic diagram of the morphology of a microlens diffuser;

[0143] Fig.78 is a schematic flow chart of an image generation method according to an embodiment of the present application;

[0144] Fig.79 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0145] Fig.80 Schematic diagram of the specific structure of the TOF depth sensing module of the embodiment of the present application;

[0146] Fig.81 Schematic diagram of the specific structure of the TOF depth sensing module of the embodiment of the present application;

[0147] Fig.82 Schematic diagram of the specific structure of the TOF depth sensing module of the embodiment of the present application;

[0148] Fig.83 Schematic diagram of the specific structure of the TOF depth sensing module of the embodiment of the present application;

[0149] Fig.84 is a schematic diagram of the structure of a TOF depth sensing module 600 according to an embodiment of the present application;

[0150] Fig.85 is a schematic diagram of the structure of a TOF depth sensing module 600 according to an embodiment of the present application;

[0151] Fig.86 is a schematic diagram of the structure of a TOF depth sensing module 600 according to an embodiment of the present application;

[0152] Fig.87 is a schematic diagram of a polarization filter receiving a polarized light beam;

[0153] Fig.88 is a schematic flow chart of an image generation method according to an embodiment of the present application;

[0154] Fig.89 Schematic diagram of the specific structure of the TOF depth sensing module of the embodiment of the present application;

[0155] Fig.90 Schematic diagram of the specific structure of the TOF depth sensing module of the embodiment of the present application;

[0156] Fig.91 Schematic diagram of the specific structure of the TOF depth sensing module of the embodiment of the present application;

[0157] Fig.92 Schematic diagram of the specific structure of the TOF depth sensing module of the embodiment of the present application;

[0158] Fig.93 Schematic diagram of the specific structure of the TOF depth sensing module of the embodiment of the present application;

[0159] Fig.94 Schematic diagram of a driving signal and a receiving signal of a TOF depth sensing module according to an embodiment of the present application;

[0160] Fig.95 is a schematic diagram of the angle and state of a light beam emitted by a TOF depth sensing module according to an embodiment of the present application;

[0161] Fig.96 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0162] Fig.97 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0163] Fig.98 Schematic diagram of the structure of the TOF depth sensing module of the embodiment of the present application;

[0164] Fig.99 It is a schematic diagram of the principle of light beam deflection by a flat liquid crystal cell;

[0165] Fig.100 It is a schematic diagram of the principle of light beam deflection by a flat liquid crystal cell;

[0166] Fig.101 is a schematic flow chart of an image generation method according to an embodiment of the present application;

[0167] Fig.102 is a schematic diagram of the FOV of the first beam;

[0168] Fig.103 It is a schematic diagram of the FOV covered by M outgoing light beams in different directions. DETAILED DESCRIPTION

[0169] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0170] Figure 1 This is a schematic diagram of the principle of laser radar ranging.

[0171] like Figure 1 As shown in the figure, the laser radar transmitter emits a laser pulse (the pulse width can be in the order of nanoseconds to picoseconds), and at the same time the timer starts timing. When the laser pulse irradiates the target area, a reflected laser pulse is generated due to the reflection of the surface of the target area. When the laser radar detector receives the reflected laser pulse, the timer stops timing to obtain TOF. Next, the distance between the laser radar and the target area can be calculated based on TOF.

[0172] Specifically, the distance between the laser radar and the target area can be determined according to formula (1).

[0173] L=c*T / 2(1)

[0174] Wherein, in the above formula (1), L is the distance between the laser radar and the target area, c is the speed of light, and T is the time for light propagation.

[0175] It should be understood that in the TOF depth sensing module of the embodiment of the present application, after the laser light source emits a light beam, it must be processed by other components in the TOF depth sensing module (for example, a collimating lens, a beam splitter, etc.) so that the light beam is finally emitted from the transmitting end. In this process, the light beam from a certain component in the TOF depth sensing module can also be called the light beam emitted by the component.

[0176] For example, a laser light source emits a light beam, which is then collimated by a collimating lens and then emitted. The light beam emitted by the collimating lens can actually be called the light beam from the collimating lens. The light beam emitted by the collimating lens here does not refer to the light beam emitted by the collimating lens itself, but is the light beam emitted after processing the light beam transmitted from the previous element.

[0177] In addition, in the present application, the light beam emitted by a laser light source or an array light source may also be referred to as a light beam from the laser light source or the array light source.

[0178] Let's combine Figure 2 A brief introduction is given to the TOF depth sensing module of the embodiment of the present application.

[0179] Figure 2 It is a schematic diagram of using the TOF depth sensing module of an embodiment of the present application to perform distance measurement.

[0180] like Figure 2As shown, the TOF depth sensing module may include a transmitting end (also known as a projection end), a receiving end and a control unit, wherein the transmitting end is used to generate an outgoing light beam, and the receiving end is used to receive a reflected light beam of a target object (the reflected light beam is a light beam obtained by the target object reflecting the outgoing light beam), and the control unit can control the transmitting end and the receiving end to transmit and receive the light beam respectively.

[0181] exist Figure 2 In the invention, the transmitting end may generally include a laser light source, a beam splitter, a collimating lens and a projection lens (optional), and the receiving end may generally include a receiving lens and a sensor, which may be collectively referred to as a receiving unit.

[0182] exist Figure 2 In the process, the TOF corresponding to the outgoing light beam can be recorded by a timing device to calculate the distance from the TOF depth sensing module to the target area, thereby obtaining the final depth map of the target object. The TOF corresponding to the outgoing light beam can refer to the time difference information between the moment when the reflected light beam is received by the receiving unit and the moment when the outgoing light beam is emitted.

[0183] Above Figure 2 The laser light source may specifically be an array light source.

[0184] The TOF depth sensing module of the embodiment of the present application can be used for three-dimensional (3D) image acquisition. The TOF depth sensing module of the embodiment of the present application can be set in an intelligent terminal (for example, a mobile phone, a tablet, a wearable device, etc.) for acquiring depth images or 3D images, and can also provide gesture and limb recognition for 3D games or somatosensory games.

[0185] Combine the following Figure 3 The TOF depth sensing module of the embodiment of the present application is introduced in detail.

[0186] Figure 3 It is a schematic block diagram of the TOF depth sensing module of an embodiment of the present application.

[0187] Figure 3 The TOF depth sensing module 100 shown includes an array light source 110, a collimating lens 120, a beam splitter 130, a receiving unit 140 and a control unit 150. These modules or units in the TOF depth sensing module 100 are described in detail below.

[0188] Array light source 110:

[0189] The array light source 110 is used to generate (emit) a laser beam.

[0190] The array light source 110 includes N light-emitting areas, each of which can generate a laser beam independently, and N is a positive integer greater than 1.

[0191] The control unit 150 is used to control M light emitting areas among the N light emitting areas of the array light source 110 to emit light.

[0192] The collimating lens 120 is used to collimate the light beams emitted by the M light-emitting areas;

[0193] The beam splitter 130 is used to split the light beam after being collimated by the collimating lens;

[0194] The receiving unit 140 is used to receive the reflected light beam from the target object.

[0195] Wherein, the above M is less than or equal to N, M is a positive integer, and N is a positive integer greater than 1; the above beam splitter is specifically used to split each received beam of light into multiple beams of light; the above reflected beam of the target object is a beam obtained by the target object reflecting the beam from the beam splitter. The beams emitted by the above M light-emitting areas can also be called beams from the M light-emitting areas.

[0196] Since M is less than or equal to N, the control unit 150 can control part or all of the light emitting areas in the array light source 110 to emit light.

[0197] The N light-emitting areas may be N independent light-emitting areas, that is, each of the N light-emitting areas may emit light independently or individually without being affected by other light-emitting areas. For each of the N light-emitting areas, each light-emitting area is generally composed of a plurality of light-emitting units. In the N light-emitting areas, different light-emitting areas are composed of different light-emitting units, that is, the same light-emitting unit belongs to only one light-emitting area. For each light-emitting area, when the light-emitting area is controlled to emit light, all the light-emitting units in the light-emitting area may emit light.

[0198] The total number of light-emitting areas of the array light source may be N. When M=N, the control unit may control all light-emitting areas of the array light source to emit light simultaneously or in a time-sharing manner.

[0199] Optionally, the control unit is used to control M light-emitting areas among the N light-emitting areas of the array light source to emit light simultaneously.

[0200] For example, the control unit may control M light-emitting areas among the N light-emitting areas of the array light source to emit light simultaneously at time T0.

[0201] Optionally, the control unit is used to control M light-emitting areas among the N light-emitting areas of the array light source to emit light at M different moments respectively.

[0202] For example, when M=3, the control unit can control the three light-emitting areas of the array light source to emit light at time T0, time T1 and time T2 respectively. That is to say, the first light-emitting area among the three light-emitting areas emits light at time T0, the second light-emitting area emits light at time T1, and the third light-emitting area emits light at time T2.

[0203] Optionally, the control unit is used to control M light-emitting areas among the N light-emitting areas of the array light source to emit light at M0 different moments, respectively, where M0 is a positive integer greater than 1 and less than M.

[0204] For example, M=3, M0=2, the control unit can control one of the three light-emitting areas of the array light source to emit light at time T0, and control the other two of the three light-emitting areas of the array light source to emit light at time T1.

[0205] In the embodiment of the present application, by controlling different light-emitting areas of the array light source to emit light in time and controlling the beam splitter to split the light beams, the number of light beams emitted by the TOF depth sensing module within a period of time can be increased, thereby achieving a higher spatial resolution and a higher frame rate during the scanning process of the target object.

[0206] Optionally, the light emitting area of ​​the array light source 110 is less than or equal to 5×5 mm 2 .

[0207] When the light emitting area of ​​the array light source 110 is less than or equal to 5×5 mm 2 When the area of ​​the array light source 110 is small, the space occupied by the TOF depth sensing module 100 can be reduced, making it easier to install the TOF depth sensing module 100 in a terminal device with relatively limited space.

[0208] Optionally, the array light source 110 may be a semiconductor laser light source.

[0209] The array light source 110 may be a vertical cavity surface emitting laser (VCSEL).

[0210] Figure 5 is a schematic diagram of VCSEL, such as Figure 5 As shown, VCSEL includes many light-emitting points ( Figure 5 Each luminous point can emit light under the control of the control unit.

[0211] Optionally, the laser light source may be a Fabry-Perot laser (abbreviated as FP laser).

[0212] Compared with a single VCSEL, a single FP laser can achieve greater power, and its electro-optical conversion efficiency is also higher than that of VCSEL, which can improve the scanning effect.

[0213] Optionally, the wavelength of the laser light beam emitted by the array light source 110 is greater than 900 nm.

[0214] Since the intensity of light greater than 900nm in sunlight is relatively weak, when the wavelength of the laser beam is greater than 900nm, it helps to reduce the interference caused by sunlight and improve the scanning effect of the TOF depth sensing module.

[0215] Optionally, the wavelength of the laser beam emitted by the array light source 110 is 940 nm or 1550 nm.

[0216] Since the intensity of light near 940nm or 1550nm in sunlight is relatively weak, when the wavelength of the laser beam is 940nm or 1550nm, the interference caused by sunlight can be greatly reduced, and the scanning effect of the TOF depth sensing module can be improved.

[0217] Combine the following Figure 5 The case where the array light source 110 includes a plurality of independent light emitting areas is described in detail.

[0218] like Figure 5 As shown, the array light source 110 is composed of mutually independent light-emitting areas 111, 112, 113 and 114, each area has a plurality of light-emitting units 1001, and the plurality of light-emitting units 1001 in each area are connected by a common electrode 1002. The light-emitting units in different light-emitting areas are connected to different electrodes, so that different areas are independent of each other.

[0219] for Figure 5 For the array light source 110 shown, the control unit 150 can control the independent light-emitting areas 111, 112, 113 and 114 to emit light at different times. For example, the control unit 150 can control the light-emitting areas 111, 112, 113 and 114 to emit light at t0, t1, t2 and t3 respectively.

[0220] Optionally, the light beam collimated by the collimating lens 120 may be a quasi-parallel light with a divergence angle less than 1 degree.

[0221] The collimating lens 120 may be composed of one lens or multiple lenses. When the collimating lens 120 is composed of multiple lenses, the collimating lens 120 can effectively reduce the aberration generated during the collimation process.

[0222] The collimating lens 120 may be made of plastic material, glass material, or both. When the collimating lens 120 is made of glass material, the collimating lens 120 can reduce the effect of temperature on the back focal length of the collimating lens 120 during the process of collimating the light beam.

[0223] Specifically, since the thermal expansion coefficient of glass material is relatively small, when the collimating lens 120 is made of glass material, the influence of temperature on the back focal length of the collimating lens 120 can be reduced.

[0224] Optionally, the clear aperture of the collimating lens 120 is less than or equal to 5 mm.

[0225] When the aperture of the collimating lens 120 is less than or equal to 5 mm, the area of ​​the collimating lens 120 is relatively small, which can reduce the space occupied by the TOF depth sensing module 100 and facilitate the installation of the TOF depth sensing module 100 in a terminal device with relatively limited space.

[0226] like Figure 3 As shown, the receiving unit 140 may include a receiving lens 141 and a sensor 142 , and the receiving lens 141 is used to converge the reflected light beam to the sensor 142 .

[0227] The sensor 142 may also be referred to as a sensor array, which may be a two-dimensional sensor array.

[0228] Optionally, the resolution of the sensor 142 is greater than or equal to P×Q, and the number of beams obtained after the beam splitter splits the beam emitted from a light-emitting area of ​​the array light source 110 is P×Q, where P and Q are both positive integers.

[0229] The resolution of the upload sensor is greater than or equal to the number of light beams after the beam splitter 130 splits the light beam from a light-emitting area of ​​the array light source, so that the sensor 142 can receive the reflected light beam obtained by the target object reflecting the light beam from the beam splitter, and then the TOF depth sensing module can achieve normal reception of the reflected light beam.

[0230] Optionally, the beam splitter 130 may be a one-dimensional beam splitting device or a two-dimensional beam splitting device.

[0231] In practical applications, a one-dimensional beam splitting device or a two-dimensional beam splitting device can be selected as needed.

[0232] Specifically, in practical applications, a one-dimensional beam splitter or a two-dimensional beam splitter can be selected as needed. When the outgoing light beam only needs to be split in one dimension, a one-dimensional beam splitter can be used. When the outgoing light beam needs to be split in two dimensions, a two-dimensional beam splitter is required.

[0233] When the beam splitter 130 is a one-dimensional beam splitting device, the beam splitter 130 may specifically be a cylindrical lens array or a one-dimensional grating.

[0234] When the beam splitter 130 is a two-dimensional beam splitting device, the beam splitter 130 may specifically be a microlens array or a two-dimensional diffractive optical element (DOE).

[0235] The beam splitter 130 may be made of a resin material or a glass material, or may be made of both a resin material and a glass material.

[0236] When the components of the beam splitter 130 include glass material, the influence of temperature on the performance of the beam splitter 130 can be effectively reduced, so that the beam splitter 130 maintains relatively stable performance. Specifically, when the temperature changes, the thermal expansion coefficient of glass is lower than that of resin. Therefore, when the beam splitter 130 is made of glass material, the performance of the beam splitter is relatively stable.

[0237] Optionally, the area of ​​the beam incident end face of the beam splitter 130 is less than 5×5 mm 2 .

[0238] When the area of ​​the beam incident end face of the beam splitter 130 is less than 5×5 mm 2 When the area of ​​the beam splitter 130 is small, the space occupied by the TOF depth sensing module 100 can be reduced, making it easier to install the TOF depth sensing module 100 in a terminal device with relatively limited space.

[0239] Optionally, the beam receiving surface of the beam splitter 130 is parallel to the beam emitting surface of the array light source 110 .

[0240] When the beam receiving surface of the beam splitter 130 is parallel to the beam emitting surface of the array light source 110 , the beam splitter 130 can more efficiently receive the beam emitted by the array light source 110 , thereby improving the efficiency of the beam splitter 130 in receiving the beam.

[0241] like Figure 3 As shown, the receiving unit 140 may include a receiving lens 141 and a sensor 142. The manner in which the receiving unit receives the light beam is described below with reference to a specific example.

[0242] For example, the above-mentioned array light source 110 includes four light-emitting areas, then the receiving lens 141 can be used to receive the reflected beam 1, reflected beam 2, reflected beam 3 and reflected beam 4 obtained by the target object reflecting the light beams generated by the beam splitter 130 at four different times (t4, t5, t6 and t7), and transmit the reflected beam 1, reflected beam 2, reflected beam 3 and reflected beam 4 to the sensor 142.

[0243] Optionally, the receiving lens 141 may be composed of one or more lenses.

[0244] When the receiving lens 141 is composed of multiple lenses, the aberration generated when the receiving lens 141 receives the light beam can be effectively reduced.

[0245] In addition, the receiving lens 141 may be made of a resin material or a glass material, or may be made of a resin material and a glass material.

[0246] When the receiving lens 141 includes a glass material, the influence of temperature on the back focal length of the receiving lens 141 can be effectively reduced.

[0247] The above-mentioned sensor 142 can be used to receive the light beam transmitted from the lens 141, and perform photoelectric conversion on the light beam transmitted from the receiving lens 141, and convert the optical signal into an electrical signal, so as to facilitate the subsequent calculation of the time difference between the light beam being emitted from the transmitting end and the light beam being received by the receiving end (this time difference can be called the flight time of the light beam), and calculate the distance between the target object and the TOF depth sensing module based on the time difference, thereby obtaining a depth image of the target object.

[0248] The sensor 142 may be a single-photon avalanche diode array (SPAD).

[0249] Among them, SPAD is an avalanche photodiode working in Geiger mode (bias voltage is higher than breakdown voltage). After receiving a single photon, there is a certain probability of avalanche effect, and a pulse current signal is instantly generated to detect the time of arrival of the photon. Since the SPAD array used in the above-mentioned TOF depth sensing module requires complex quenching circuits, timing circuits, and storage and reading units, the existing SPAD arrays used for TOF depth sensing have limited resolution.

[0250] When the target object is far away from the TOF depth sensing module, the intensity of the reflected light of the target object transmitted from the receiving lens to the sensor is generally very weak, and the sensor needs to have a very high detection sensitivity. SPAD has single-photon detection sensitivity and a response time of picoseconds. Therefore, using SPAD as sensor 142 in this application can improve the sensitivity of the TOF depth sensing module.

[0251] In addition to controlling the array light source 110 , the control unit 150 may also control the sensor 142 .

[0252] The control unit 150 may be electrically connected to the array light source 110 and the sensor 142 to control the array light source 110 and the sensor 142 .

[0253] Specifically, the control unit 150 can control the working mode of the sensor 142 so that at M different moments, the corresponding area of ​​the sensor can respectively receive the reflected light beams reflected by the target object from the corresponding light emitting area of ​​the array light source 110.

[0254] Specifically, the portion of the reflected light beam of the target object within the numerical aperture of the receiving lens will be received by the receiving lens and propagated to the sensor. Through the design of the receiving lens, each pixel of the sensor can receive reflected light beams from different areas of the target object.

[0255] In the present application, by controlling the light emission of the array light source in a partitioned manner and using a beam splitter for beam splitting, the number of light beams emitted by the TOF depth sensing module at the same time can be increased, and the spatial resolution and high frame rate of the depth map of the target object finally obtained can be improved.

[0256] It should be understood that Figure 2 As shown, for the TOF depth sensing module of the embodiment of the present application, the projection end and the receiving end in the TOF depth sensing module can be located on the same side of the target object.

[0257] Optionally, the output optical power of the TOF depth sensing module 100 is less than or equal to 800mw.

[0258] Specifically, the maximum output optical power or the average output power of the TOF depth sensing module 100 is less than or equal to 800mw.

[0259] When the output optical power of the TOF depth sensing module 100 is less than or equal to 800 mw, the power consumption of the TOF depth sensing module 100 is relatively low, and it is convenient to be installed in a terminal device or other device that is sensitive to power consumption.

[0260] Combine the following Figures 6 to 10The process of obtaining the depth map of the target object by the TOF depth sensing module 100 in the embodiment of the present application is described in detail.

[0261] like Figure 6 As shown, the left figure is a schematic diagram of the light-emitting area of ​​the array light source 110. The array light source 110 includes four light-emitting areas (also called light-emitting areas) A, B, C and D. These four light-emitting areas are lit at t0, t1, t2 and t3 respectively. The right figure is a schematic diagram of the light beam generated by the array light source 110 after being split by the beam splitter 130 and projected onto the surface of the target object, where each point represents a projected light spot, and the area surrounded by each black solid line frame is the target area corresponding to a pixel in the sensor 142. Figure 6 In the figure, the replication order of the corresponding beam splitter 130 is 4×4, that is, at each moment, the light spots generated by one area of ​​the array light source will become 4×4 spots after being replicated by the beam splitter 130. Therefore, the number of light spots projected at the same moment can be greatly increased through the beam splitter 130.

[0262] exist Figure 6 In the example, by lighting up the four light-emitting areas of the array light source 110 at times t0, t1, t2 and t3 respectively, depth maps of different positions of the target object can be obtained.

[0263] Specifically, the schematic diagrams of the light beam emitted by the light emitting area a of the array light source 110 at time t0 and projected onto the surface of the target object after being split by the beam splitter 130 are as follows: Figure 7 shown.

[0264] The schematic diagrams of the light beam emitted by the light emitting area b of the array light source 110 at time t1 and projected onto the surface of the target object after being split by the beam splitter 130 are shown in FIG. Figure 8 shown.

[0265] The schematic diagrams of the light beam emitted by the light emitting area c of the array light source 110 at time t2 and projected onto the surface of the target object after being split by the beam splitter 130 are shown in FIG. Fig. 9 shown.

[0266] The schematic diagrams of the light beam emitted by the light emitting area d of the array light source 110 at time t3 and projected onto the surface of the target object after being split by the beam splitter 130 are shown in FIG. Fig.10 shown.

[0267] according to Figures 7 to 10The light beam projection situation shown can obtain the depth maps corresponding to the target object at t0, t1, t2 and t3, and then superimpose the depth maps corresponding to the target object at t0, t1, t2 and t3 to obtain a depth map of the target object with higher resolution.

[0268] exist Figure 3 In the TOF depth sensing module 100 shown, the collimating lens 120 can be located between the array light source 110 and the beam splitter 130. The light beam emitted by the array light source 110 must first be collimated by the collimating lens 120, and then processed by the beam splitter.

[0269] Optionally, for the TOF depth sensing module 100 , the beam splitter 130 may first directly split the light beam generated by the array light source 110 , and then the collimating lens 120 may collimate the split light beam.

[0270] Combine the following Fig.11 Detailed description is given in Fig.11 In the TOF depth sensing module 100 shown, the specific functions of each module or unit are as follows:

[0271] The control unit 150 is used to control M light-emitting areas among the N light-emitting areas in the array light source 110 to emit light;

[0272] The beam splitter 130 is used to split the light beams emitted by the M light-emitting areas;

[0273] The collimating lens 120 is used to collimate the light beam emitted by the beam splitter 130;

[0274] The receiving unit 140 is used to receive the reflected light beam from the target object.

[0275] Wherein, the above M is less than or equal to N, M is a positive integer, and N is a positive integer greater than 1; the beam splitter 130 is specifically used to split each received beam of light into multiple beams of light; the reflected beam of the target object is a beam obtained by the target object reflecting the beam emitted by the collimating lens 120. The beams emitted by the above M light-emitting areas can also be called beams from the M light-emitting areas.

[0276] Fig.11 The TOF depth sensor module shown is Figure 3 The main difference between the TOF depth sensor modules shown is the position of the collimating lens. Figure 3 In the TOF depth sensing module shown, the collimating lens is located between the array light source and the beam splitter, and Fig.11In the TOF depth sensing module shown, the beam splitter is located between the array light source and the collimating lens (equivalent to the collimating lens being located in the direction of the light beam emitted by the beam splitter).

[0277] Fig.11 The TOF depth sensing module 100 and Figure 3 The TOF depth sensing module 100 shown in the figure processes the light beam emitted by the array light source 110 in a slightly different manner. Figure 3 In the TOF depth sensing module 100 shown in FIG. 1 , after the array light source 110 emits a light beam, the collimating lens 120 and the beam splitter 130 perform collimation and beam splitting processing in sequence. Fig.11 In the TOF depth sensing module 100 shown, after the array light source 110 emits a light beam, the beam splitter 130 and the collimating lens 120 perform beam splitting and collimation processing in sequence.

[0278] The following describes the beam splitting process of the beam splitter 130 on the light beam emitted by the array light source in conjunction with the accompanying drawings.

[0279] like Fig.12 As shown, after the multiple light beams generated by the array light source 110 are split by the beam splitter 130, each light beam generated by the array light source 110 can be split into multiple light beams, and ultimately more light beams can be obtained after beam splitting.

[0280] exist Fig.11 On the basis of the TOF depth sensing module shown, the TOF depth sensing module 100 of the embodiment of the present application may further include an optical element having a controllable refractive index. When the refractive index of the optical element is different, the optical element can adjust a light beam in a single polarization state to different directions. Different light beams can be irradiated to different directions without mechanical rotation and vibration, and the scanning area of ​​interest can be quickly located.

[0281] Fig.13 It is a schematic structural diagram of the TOF depth sensing module of an embodiment of the present application.

[0282] exist Fig.13 In the TOF depth sensing module 100 shown, the specific functions of each module or unit are as follows:

[0283] The control unit 150 is used to control M light-emitting areas among the N light-emitting areas of the array light source 110 to emit light;

[0284] The control unit 150 is further used to control the birefringence parameters of the optical element 160 to change the propagation directions of the light beams emitted by the M light emitting regions.

[0285] The beam splitter 130 is used to receive the light beam emitted by the optical element 160 and perform beam splitting processing on the light beam emitted by the optical element 160;

[0286] Optionally, the beam splitter 130 is specifically used to split each received light beam into multiple light beams. The beam splitter 130 can split the light beam emitted from a light-emitting area of ​​the array light source 110 so that the number of light beams obtained is P×Q.

[0287] The collimating lens 120 is used to collimate the light beam emitted by the beam splitter 130;

[0288] The receiving unit 140 is used to receive the reflected light beam from the target object.

[0289] The reflected light beam of the target object is a light beam obtained by the target object reflecting the light beam emitted by the beam splitter 130. The light beams emitted by the M light-emitting regions may also be referred to as light beams from the M light-emitting regions.

[0290] exist Fig.13 In the embodiment, the optical element 160 is located between the array light source 110 and the beam splitter 130. In fact, the optical element 160 can also be located between the collimating lens 120 and the beam splitter 130. Fig.14 Provide explanation.

[0291] Fig.14 It is a schematic structural diagram of the TOF depth sensing module of an embodiment of the present application.

[0292] exist Fig.14 In the TOF depth sensing module 100 shown, the specific functions of each module or unit are as follows:

[0293] The control unit 150 is used to control M light-emitting areas among the N light-emitting areas of the array light source 110 to emit light;

[0294] The collimating lens 120 is used to collimate the light beams emitted by the M light-emitting areas;

[0295] The control unit 150 is also used to control the birefringence parameters of the optical element 160 to change the propagation direction of the light beam after the collimation process by the collimating lens 120;

[0296] The beam splitter 130 is used to receive the light beam emitted by the optical element 160 and perform beam splitting processing on the light beam emitted by the optical element 160;

[0297] Optionally, the beam splitter 130 is specifically used to split each received light beam into multiple light beams. The beam splitter 130 can split the light beam emitted from a light-emitting area of ​​the array light source 110 so that the number of light beams obtained is P×Q.

[0298] The collimating lens 120 is used to collimate the light beam emitted by the beam splitter 130;

[0299] The receiving unit 140 is used to receive the reflected light beam from the target object.

[0300] The reflected light beam of the target object is a light beam obtained by the target object reflecting the light beam emitted by the beam splitter 130. The light beams emitted by the M light-emitting regions may also be referred to as light beams from the M light-emitting regions.

[0301] Combine the following Fig.15 The working process of the TOF depth sensing module in the embodiment of the present application is described in detail.

[0302] Fig.15 Schematic diagram of the operation of the TOF depth sensing module in an embodiment of the present application.

[0303] like Fig.15 As shown, the TOF depth sensing module includes a projection end, a receiving end and a control unit, wherein the control unit is used to control the projection end to emit an outgoing light beam to scan the target area, and the control unit is also used to control the receiving end to receive a reflected light beam reflected from the target scanning area.

[0304] The projection end includes an array light source 110, a collimating lens 120, an optical element 160, a beam splitter 130 and a projection lens (optional). The receiving end includes a receiving lens 141 and a sensor 142. The control unit 150 is also used to control the timing synchronization of the array light source 110, the optical element 160 and the sensor 142.

[0305] Fig.15 The collimating lens 120 in the TOF depth sensing module shown may include 1-4 lenses, and the collimating lens 120 is used to convert the first light beam generated by the array light source 110 into approximately parallel light.

[0306] Above Fig.15 The workflow of the TOF depth sensing module shown is as follows:

[0307] (1) The light beam emitted by the array light source 110 is collimated by the collimating lens 120 to form a collimated light beam and reaches the optical element 160;

[0308] (2) The optical element 160 realizes orderly deflection of the light beam according to the timing control of the control unit, so that the angle of the deflected light beam after being emitted realizes two-dimensional scanning;

[0309] (3) The deflected light beam emitted from the optical element 160 reaches the beam splitter 130;

[0310] (4) The beam splitter 130 replicates the deflected light beam at each angle to obtain output light beams at multiple angles, thereby achieving two-dimensional replication of the light beam;

[0311] (5) In each scanning cycle, the receiving end can only image the target area illuminated by the spot;

[0312] (6) After the optical element completes all S×T scans, the two-dimensional array sensor in the receiving end will generate S×T images, and finally these images will be stitched into a higher resolution image in the processor.

[0313] The array light source in the TOF depth sensing module of the embodiment of the present application can have multiple light-emitting areas, each of which can emit light independently. Fig.16 The working process of the TOF depth sensing module in the embodiment of the present application is described in detail when the array light source of the TOF depth sensing module includes multiple light-emitting areas.

[0314] Fig.16 It is a schematic diagram of the light emitting area of ​​the array light source.

[0315] When the array light source 110 includes multiple light-emitting areas, the working process of the TOF depth sensing module of the embodiment of the present application is as follows:

[0316] (1) The light beams emitted from different light-emitting areas of the array light source 110 are formed into collimated light beams through the collimating lens 120 and reach the beam splitter 130. The beam splitter 130 is controlled by the timing signal of the control unit to realize orderly deflection of the light beam, so that the angle of the output light beam can realize two-dimensional scanning;

[0317] (2) The light beam collimated by the collimating lens 120 reaches the beam splitter 130, and the beam splitter 130 replicates the incident light beam at each angle, and simultaneously generates outgoing light beams at multiple angles, thereby realizing two-dimensional replication of the light beam;

[0318] (3) In each scanning cycle, the receiving end only images the target area illuminated by the spot;

[0319] (4) After the optical element completes all S×T scans, the two-dimensional array sensor in the receiving end will generate S×T images, and finally these images will be stitched into a higher resolution image in the processor.

[0320] Combine the following Fig.16 and Fig.17 The scanning working principle of the TOF depth sensing module of the embodiment of the present application is described.

[0321] like Fig.16As shown, 111, 112, 113, and 114 are independent light-emitting areas of the array light source, which can be lit up in different time periods, and 115, 116, 117, and 118 are light-emitting holes in different independent working areas of the array light source.

[0322] Fig.17 A beam splitter is used to Fig.16 Schematic diagram of beam splitting processing of light beams emitted by an array light source shown.

[0323] like Fig.17 As shown, 120 is a replica level generated by the beam splitter ( Fig.17 121 is the target area corresponding to a pixel of the two-dimensional array sensor (121 includes 122, 123, 124 and 125), 122 is the spot generated by the light beam scanning of the light-emitting hole 115 through the beam splitter, 123 is the spot generated by the light beam scanning of the light-emitting hole 116 using the optical element, 124 is the spot generated by the light beam scanning of the light-emitting hole 117 using the optical element, and 125 is the spot generated by the light beam scanning of the light-emitting hole 118 using the optical element.

[0324] have Fig.16 The specific scanning process of the TOF depth sensing module of the array light source shown is as follows:

[0325] Only 115 is lit, and the optical elements perform beam scanning respectively to achieve the spot of 122;

[0326] Turn off 115, light up 116, and the optical elements respectively perform beam scanning to achieve spot 123;

[0327] Turn off 116, turn on 117, and the optical elements respectively perform beam scanning to achieve a spot of 124;

[0328] Turn off 117, turn on 118, and the optical elements respectively perform beam scanning to achieve a spot of 125;

[0329] Through the above four steps, spot scanning of the target area corresponding to one pixel of the two-dimensional array sensor can be completed.

[0330] Above Figures 13 to 15 The optical element 160 may be any one of liquid crystal polarization gratings, electro-optical devices, acousto-optic devices, optical phased array devices, and the like. For detailed introductions to liquid crystal polarization gratings, electro-optical devices, acousto-optic devices, optical phased array devices, and the like, please refer to the relevant descriptions in the first to fourth cases below.

[0331] The TOF depth sensing module of the embodiment of the present application is introduced in detail above in combination with the accompanying drawings. The image generation method of the embodiment of the present application is introduced below in combination with the accompanying drawings.

[0332] Fig.18 It is a schematic flowchart of the image generation method of an embodiment of the present application. Fig.18 The method shown can be executed by a terminal device including the TOF depth sensing module of the embodiment of the present application. Specifically, Fig.18 The method shown can be implemented by including Figure 3 The TOF depth sensing module is used to execute the terminal device shown. Fig.18 The method shown includes steps 2001 to 2006, each of which is described in detail below.

[0333] 2001. Use a control unit to control M light-emitting areas among N light-emitting areas of the array light source to emit light at M different times.

[0334] Wherein, M is less than or equal to N, M is a positive integer, and N is a positive integer greater than 1.

[0335] In the above step 2001, the light emission of the array light source may be controlled by a control unit.

[0336] Specifically, the control unit may send control signals to the M light-emitting areas of the array light source at M times respectively, so as to control the M light-emitting areas to emit light separately at M different times.

[0337] For example, Figure 6 As shown, the array light source 110 includes four independent light-emitting areas A, B, C and D. Then, the control unit can send control signals to the four independent light-emitting areas A, B, C and D at t0, t1, t2 and t3 respectively, so that the four independent light-emitting areas A, B, C and D emit light at t0, t1, t2 and t3 respectively.

[0338] 2002. A collimating lens is used to collimate the light beams generated by M light-emitting areas at M different moments to obtain collimated light beams.

[0339] Still Figure 6 Taking an example to illustrate, when the four independent light-emitting areas A, B, C and D of the array light source emit light beams at t0, t1, t2 and t3 respectively, the collimating lens can collimate the light beams emitted by the light-emitting areas A, B, C and D at t0, t1, t2 and t3 respectively to obtain collimated light beams.

[0340] 2003. A beam splitter is used to split the collimated light beam.

[0341] The beam splitter can specifically split each received beam of light into multiple beams of light. The number of beams obtained after the beam splitter splits the beam from a light-emitting area of ​​the array light source may be P×Q.

[0342] like Figure 6 As shown, the light emitting areas A, B, C and D of the array light source emit light beams at t0, t1, t2 and t3 respectively. Then, the light beams emitted by the light emitting areas A, B, C and D at t0, t1, t2 and t3 respectively are processed by the collimating lens and then incident on the beam splitter for processing. The result of the beam splitting processing of the light emitting areas A, B, C and D by the beam splitter can be shown as follows: Figure 6 Shown on the right.

[0343] Optionally, the beam splitting process in the above step 2003 specifically includes: using a beam splitter to perform one-dimensional or two-dimensional beam splitting process on the light beam generated after the collimation process.

[0344] 2004. Using a receiving unit to receive the reflected light beam from the target object.

[0345] The reflected light beam of the target object is a light beam obtained by the target object reflecting the light beam from the beam splitter.

[0346] Optionally, the receiving unit in step 2004 includes a receiving lens and a sensor, and receiving the reflected light beam of the target object by using the receiving unit in step 2004 includes: converging the reflected light beam of the target object to the sensor by using the receiving lens. The sensor here can also be called a sensor array, and the sensor array can be a two-dimensional sensor array.

[0347] Optionally, the resolution of the sensor is greater than or equal to P×Q, and the number of light beams obtained after the beam splitter splits the light beam from a light-emitting area of ​​the array light source is P×Q.

[0348] Wherein, the above P and Q are both positive integers. The resolution of the upload sensor is greater than or equal to the number of light beams after the beam splitter splits the light beam from a light emitting area of ​​the array light source, so that the sensor can receive the reflected light beam obtained by the target object reflecting the light beam from the beam splitter, so that the TOF depth sensing module can achieve normal reception of the reflected light beam.

[0349] 2005. Generate M depth maps according to TOFs corresponding to light beams emitted by M light-emitting areas of the array light source at M different moments.

[0350] The TOF corresponding to the light beams emitted by the M light-emitting areas of the above-mentioned array light source at M different times may specifically refer to the time difference information between the emission time of the light beams emitted by the M light-emitting areas of the array light source at M different times and the reception time of the corresponding reflected light beams.

[0351] For example, the array light source includes three light-emitting areas A, B and C, wherein the light-emitting area A emits a light beam at time T0, the light-emitting area B emits a light beam at time T1, and the light-emitting area C emits a light beam at time T2. Then, the TOF corresponding to the light beam emitted by the light-emitting area A at time T0 can specifically refer to the time difference information between the moment when the light beam emitted by the light-emitting area A at time T0 is collimated by the collimating lens and split by the beam splitter, and reaches the target object, and after being reflected by the target object, finally reaches the receiving unit (or is received by the receiving unit) and the moment T0. The TOF corresponding to the light beam emitted by the light-emitting area B at time T1 and the TOF corresponding to the light beam emitted by the light-emitting area C at time T2 also have similar meanings. Optionally, the above-mentioned M depth maps are depth maps corresponding to M area sets of the target object, and there is a non-overlapping area between any two area sets in the M area sets.

[0352] Optionally, generating M depth maps of the target object in step 2005 specifically includes:

[0353] 2005a. Determine the distances between the M regions of the target object and the TOF depth sensing module according to the TOFs corresponding to the light beams emitted by the M light-emitting regions at M different moments;

[0354] 2005b. Generate depth maps of the M regions of the target object according to the distances between the M regions of the target object and the TOF depth sensing module.

[0355] 2006. Obtain a final depth map of the target object according to the M depth maps.

[0356] Specifically, in step 2006, the M depth maps may be row-joined to obtain a depth map of the target object.

[0357] For example, through the above steps 2001 to 2005, the depth map of the target object at time t0-t3 is obtained. The depth maps at these four times are as follows: Fig.19 As shown, through Fig.19 The depth maps at time t0-t3 shown in the figure are stitched together to obtain the final depth map of the target object. Fig.69 shown.

[0358] When the structure of the TOF depth sensing module is different, the corresponding image generation method process is also different. Fig. 20The image generation method of the embodiment of the present application is introduced.

[0359] Fig. 20 It is a schematic flowchart of the image generation method of an embodiment of the present application. Fig. 20 The method shown can be executed by a terminal device including the TOF depth sensing module of the embodiment of the present application. Specifically, Fig. 20 The method shown can be implemented by including Fig.11 The TOF depth sensing module is used to execute the terminal device shown. Fig. 20 The method shown includes steps 3001 to 3006, each of which is described in detail below.

[0360] 3001. Use a control unit to control M light-emitting areas among N light-emitting areas of an array light source to emit light at M different times.

[0361] Among them, the above-mentioned N light-emitting areas do not overlap with each other, M is less than or equal to N, M is a positive integer, and N is a positive integer greater than 1.

[0362] The above-mentioned using the control unit to control M light-emitting areas among the N light-emitting areas of the array light source to emit light at M different times respectively may specifically mean using the control unit to control the M light-emitting areas to emit light in sequence at M different times respectively.

[0363] For example, Fig.16 As shown, the array light source includes four light-emitting areas 111, 112, 113 and 114. Then, the control unit can control 111, 112 and 113 to emit light at time T0, time T1 and time T2, respectively. Alternatively, the control unit can also control 111, 112, 113 and 114 to emit light at time T0, time T1, time T2 and time T3, respectively.

[0364] 3002. Use a beam splitter to split the light beams generated by M light-emitting areas at M different moments.

[0365] The above-mentioned beam splitter is specifically used to split each received beam of light into multiple beams of light.

[0366] The above-mentioned use of a beam splitter to split the light beams generated by the M light-emitting areas at M different moments may specifically refer to use of a beam splitter to split the light beams generated by the M light-emitting areas at M different moments.

[0367] For example, Fig.16As shown, the array light source includes four light-emitting areas 111, 112, 113 and 114. The control unit can control 111, 112 and 113 to emit light at time T0, time T1 and time T2 respectively. Then, the beam splitter can perform beam splitting processing on the light beam emitted by 111 at time T0, perform beam splitting processing on the light beam emitted by 112 at time T1, and perform beam splitting processing on the light beam emitted by 112 at time T2 (it should be understood that the time required for the light beam to reach the beam splitter from the light-emitting area is ignored here).

[0368] Optionally, the beam splitting process in the above step 3002 specifically includes: using a beam splitter to perform one-dimensional or two-dimensional beam splitting processes on the light beams generated by the M light-emitting areas at M different moments.

[0369] 3003. Use a collimating lens to collimate the light beam from the beam splitter.

[0370] For example, still Fig.16 For example, the beam splitter performs beam splitting processing on the light beams emitted by 111, 112 and 113 at time T0, time T1 and time T2 respectively. Then, the collimating lens can collimate the light beam 111 after the beam splitting processing by the beam splitter at time T0, collimate the light beam 112 after the beam splitting processing by the beam splitter at time T1, and collimate the light beam 113 after the beam splitter processing by the beam splitter at time T2.

[0371] 3004. Receive the reflected light beam from the target object using a receiving unit.

[0372] The reflected light beam of the target object is a light beam obtained by the target object reflecting the light beam from the collimating lens.

[0373] Optionally, the receiving unit in step 3004 includes a receiving lens and a sensor, and receiving the reflected light beam of the target object by using the receiving unit in step 3004 includes: converging the reflected light beam of the target object to the sensor by using the receiving lens. The sensor here may also be referred to as a sensor array, and the sensor array may be a two-dimensional sensor array.

[0374] Optionally, the resolution of the sensor is greater than or equal to P×Q, and the number of light beams obtained after the beam splitter splits the light beam from a light-emitting area of ​​the array light source is P×Q.

[0375] Wherein, the above P and Q are both positive integers. The resolution of the upload sensor is greater than or equal to the number of light beams after the beam splitter splits the light beam from a light emitting area of ​​the array light source, so that the sensor can receive the reflected light beam obtained by the target object reflecting the light beam from the beam splitter, so that the TOF depth sensing module can achieve normal reception of the reflected light beam.

[0376] 3005. Generate M depth maps according to TOFs corresponding to light beams emitted by M light-emitting areas of the array light source at M different moments.

[0377] The TOF corresponding to the light beams emitted by the M light-emitting areas of the above-mentioned array light source at M different times may specifically refer to the time difference information between the emission time of the light beams emitted by the M light-emitting areas of the array light source at M different times and the reception time of the corresponding reflected light beams.

[0378] For example, the array light source includes three light-emitting areas A, B and C, wherein the light-emitting area A emits a light beam at time T0, the light-emitting area B emits a light beam at time T1, and the light-emitting area C emits a light beam at time T2. Then, the TOF corresponding to the light beam emitted by the light-emitting area A at time T0 can specifically refer to the time difference information between the time when the light beam emitted by the light-emitting area A at time T0 is collimated by the collimating lens and split by the beam splitter, and reaches the target object, and after being reflected by the target object, finally reaches the receiving unit (or is received by the receiving unit) and the time T0. The TOF corresponding to the light beam emitted by the light-emitting area B at time T1 and the TOF corresponding to the light beam emitted by the light-emitting area C at time T2 also have similar meanings.

[0379] The M depth maps are depth maps corresponding to M region sets of the target object, respectively. There are non-overlapping regions between any two region sets in the M region sets.

[0380] Optionally, the above step 3005 generates M depth maps, specifically including:

[0381] 3005a, determining the distances between the M regions of the target object and the TOF depth sensing module according to the TOFs corresponding to the light beams emitted by the M light-emitting regions at M different times;

[0382] 3005b. Generate depth maps of the M regions of the target object according to the distances between the M regions of the target object and the TOF depth sensing module.

[0383] 3006. Obtain a final depth map of the target object according to the M depth maps.

[0384] Optionally, obtaining the final depth map of the target object in the above step 3006 includes: performing row stitching on M depth maps to obtain the depth map of the target object.

[0385] For example, the depth map obtained through the process of steps 3001 to 3005 above can be as follows: Fig.68 As shown, Fig.68 The depth map corresponding to the time t0-t3 is shown. By splicing the depth map corresponding to the time t0-t3, the following can be obtained: Fig.69 The final depth map of the target object is shown.

[0386] In an embodiment of the present application, by controlling different light-emitting areas of the array light source to emit light in time and controlling the beam splitter to split the light beams, the number of light beams emitted by the TOF depth sensing module within a period of time can be increased, thereby obtaining multiple depth maps, so that the final depth map obtained by stitching together multiple depth maps has a higher spatial resolution and a higher frame rate.

[0387] Fig. 20 The method shown is similar to Fig.18 The main processing process of the method is similar, the main difference is that Fig. 20 In the method shown, a beam splitter is first used to split the light beam emitted by the array light source, and then a collimating lens is used to collimate the split light beam. Fig.18 In the method shown, a collimating lens is first used to collimate the light beam emitted by the array light source, and then a beam splitter is used to split the collimated light beam.

[0388] When the image generation method of the embodiment of the present application is executed by a terminal device, the terminal device can be in different working modes, and the light emission mode of the array light source and the subsequent method of generating the final depth map of the target object are different in different working modes. The following is a detailed description of how to obtain the final depth map of the target object in different working modes in conjunction with the accompanying drawings.

[0389] Fig.21 It is a schematic flowchart of the image generation method of an embodiment of the present application.

[0390] Fig.21 The method shown includes steps 4001 to 4003, which are described in detail below.

[0391] 4001. Determine the working mode of the terminal device.

[0392] The above-mentioned terminal device includes a first working mode and a second working mode, wherein, in the first working mode, the control unit can control L of the N light-emitting areas of the array light source to emit light simultaneously, and in the second working mode, the control unit can control M of the N light-emitting areas of the array light source to emit light at M different times.

[0393] It should be understood that when it is determined in the above step 4001 that the terminal device is working in the first working mode, step 4002 is executed, and when it is determined in the above step 4001 that the terminal device is working in the second working mode, step 4003 is executed.

[0394] The specific process of determining the working mode of the terminal device in step 4001 is described in detail below.

[0395] Optionally, determining the working mode of the terminal device in the above step 4001 includes: determining the working mode of the terminal device according to the working mode selection information of the user.

[0396] The user's working mode selection information is used to select one of the first working mode and the second working mode as the working mode of the terminal device.

[0397] Specifically, when the above-mentioned image generation method is executed by a terminal device, the terminal device may obtain the user's work mode selection information from the user. For example, the user may input the user's work mode selection information through the operation interface of the terminal device.

[0398] The above-mentioned determination of the working mode of the terminal device according to the working mode selection information of the user enables the user to flexibly select and determine the working mode of the terminal device.

[0399] Optionally, determining the working mode of the terminal device in the above step 4001 includes: determining the working mode of the terminal device according to the distance between the terminal device and the target object.

[0400] Specifically, when the distance between the terminal device and the target object is less than or equal to the preset distance, it can be determined that the terminal device is operating in the first operating mode; and when the distance between the terminal device and the target object is greater than the preset distance, it can be determined that the terminal device is operating in the second operating mode.

[0401] When the distance between the terminal device and the target object is small, the array light source has sufficient luminous power to emit multiple light beams that reach the target object at the same time. Therefore, when the distance between the terminal device and the target object is small, by adopting the first working mode, multiple luminous areas of the array light source can emit light at the same time, which is convenient for obtaining depth information of more areas of the target object later, and can improve the frame rate of the depth map of the target object when the resolution of the depth map of the target object is constant.

[0402] When the distance between the terminal device and the target object is large, the second working mode can be used to obtain the depth map of the target object due to the limited total power of the array light source. Specifically, by controlling the array light source to emit light beams in time division, the light beams emitted by the array light source in time division can also reach the target object. This allows the terminal device to obtain the depth information of different areas of the target object in time division when it is far away from the target object, thereby obtaining the depth map of the target object.

[0403] Optionally, determining the working mode of the terminal device in the above step 4001 includes: determining the working mode of the terminal device according to a scene in which the target object is located.

[0404] Specifically, when the terminal device is in an indoor scene, it can be determined that the terminal device is working in the first working mode; when the terminal device is in an outdoor scene, it can be determined that the terminal device is working in the second working mode.

[0405] When the terminal device is in an indoor scene, since the distance between the terminal device and the target object is relatively close and the external noise is relatively weak, the array light source has sufficient luminous power to simultaneously emit multiple light beams reaching the target object. Therefore, when the distance between the terminal device and the target object is small, by adopting the first working mode, multiple luminous areas of the array light source can be illuminated at the same time, which is convenient for obtaining the depth information of more areas of the target object later, and can improve the frame rate of the depth map of the target object when the resolution of the depth map of the target object is constant.

[0406] When the terminal device is in an outdoor scene, since the distance between the terminal device and the target object is relatively far, the external noise is relatively large, and the total power of the array light source is limited, the second working mode can be used to obtain the depth map of the target object. Specifically, by controlling the array light source to emit light beams in time-sharing, the light beams emitted by the array light source in time-sharing can also reach the target object. This allows the terminal device to obtain the depth information of different areas of the target object in time-sharing when it is far away from the target object, thereby obtaining a depth map of the target object.

[0407] The working mode of the terminal device can be flexibly determined according to the distance between the terminal device and the target object or the scene where the target object is located, so that the terminal device works in a suitable working mode.

[0408] 4002. Obtain a final depth map of the target object in the first working mode.

[0409] 4003. Obtain a final depth map of the target object in the second working mode.

[0410] In the embodiment of the present application, the image generation method has different working modes. Therefore, the first working mode or the second working mode can be selected according to different situations to generate a depth map of the target object, which can improve the flexibility of generating the depth map of the target object and obtain a high-resolution depth map of the target object in both working modes.

[0411] Combine the following Fig. 22 The process of obtaining the final depth map of the target object in the first working mode is introduced in detail.

[0412] Fig. 22 is a schematic flow chart of obtaining a final depth map of a target object in a first working mode. Fig. 22 The process shown includes steps 4002A to 4002E, each of which is described in detail below.

[0413] 4002A. Control L light-emitting areas among the N light-emitting areas of the array light source to emit light simultaneously.

[0414] Wherein, L is less than or equal to N, L is a positive integer, and N is a positive integer greater than 1.

[0415] In step 4002A, the control unit may be used to control L light-emitting areas of the N light-emitting areas of the array light source to emit light simultaneously. Specifically, the control unit may send a control signal to L light-emitting areas of the N light-emitting areas of the array light source at time T to control the L light-emitting areas to emit light simultaneously at time T.

[0416] For example, the array light source includes four independent light-emitting areas A, B, C and D. Then, the control unit can send a control signal to the four independent light-emitting areas A, B, C and D at time T, so that the four independent light-emitting areas A, B, C and D emit light simultaneously at time T.

[0417] 4002B. Use a collimating lens to collimate the light beams emitted by the L light-emitting areas.

[0418] Assuming that the array light source includes four independent light-emitting areas A, B, C and D, the collimating lens can collimate the light beams emitted by the light-emitting areas A, B, C and D of the array light source at time T to obtain collimated light beams.

[0419] In step 4002B, the light beam is collimated by a collimating lens to obtain a nearly parallel light beam, which can increase the power density of the light beam and further improve the effect of subsequent scanning using the light beam.

[0420] 4002C. Use a beam splitter to split the light beam generated after the collimation process by the collimating lens.

[0421] The above-mentioned beam splitter is specifically used to split each received beam of light into multiple beams of light.

[0422] 4002D. Receive the reflected light beam from the target object using a receiving unit.

[0423] The reflected light beam of the target object is a light beam obtained by the target object reflecting the light beam from the beam splitter.

[0424] 4002E. Obtain a final depth map of the target object according to the TOF corresponding to the light beams emitted by the L light-emitting areas.

[0425] The TOF corresponding to the light beams emitted by the L light-emitting areas may specifically refer to the time difference information between the reception time of the reflected light beams corresponding to the light beams emitted by the L light-emitting areas of the array light source at time T and time T.

[0426] Optionally, the receiving unit includes a receiving lens and a sensor, and the step 4002D of using the receiving unit to receive the reflected light beam of the target object includes: using the receiving lens to converge the reflected light beam of the target object to the sensor.

[0427] The above-mentioned sensor may also be referred to as a sensor array, and the sensor array may be a two-dimensional sensor array.

[0428] Optionally, the resolution of the sensor is greater than P×Q, and the number of light beams obtained after the beam splitter splits the light beam from a light-emitting area of ​​the array light source is P×Q.

[0429] Wherein, the above P and Q are both positive integers. Since the resolution of the upload sensor is greater than the number of beams after the beam splitter splits the beam from a light emitting area of ​​the array light source, the sensor can receive the reflected beam obtained by the target object reflecting the beam from the beam splitter, so that the TOF depth sensing module can achieve normal reception of the reflected beam.

[0430] Optionally, generating a final depth map of the target object in step 4002E specifically includes:

[0431] (1) generating depth maps of L regions of the target object according to the TOFs corresponding to the light beams emitted by the L light-emitting regions;

[0432] (2) A depth map of the target object is synthesized based on the depth maps of L regions of the target object.

[0433] Above Fig. 22 The method shown can be Figure 3 The TOF depth sensor module shown may include Figure 3 The TOF depth sensing module is used to execute the terminal device shown.

[0434] When the relative position relationship between the collimating lens and the beam splitter in the TOF depth sensor module is different, the process of obtaining the final depth map of the target object in the first working mode is also different. Fig.23 The process of obtaining a final depth map of a target object in the first working mode is described.

[0435] Fig.23 is a schematic flow chart of obtaining a final depth map of a target object in a first working mode. Fig.23 The process shown includes steps 4002a to 4002e, each of which is described in detail below.

[0436] 4002a. Control L light-emitting areas among the N light-emitting areas of the array light source to emit light simultaneously.

[0437] Wherein, L is less than or equal to N, L is a positive integer, and N is a positive integer greater than 1.

[0438] In step 4002a, the control unit can be used to control L light-emitting areas of the N light-emitting areas of the array light source to emit light simultaneously. Specifically, the control unit can send a control signal to L light-emitting areas of the N light-emitting areas of the array light source at time T to control the L light-emitting areas to emit light simultaneously at time T.

[0439] For example, the array light source includes four independent light-emitting areas A, B, C and D. Then, the control unit can send a control signal to the four independent light-emitting areas A, B, C and D at time T, so that the four independent light-emitting areas A, B, C and D emit light simultaneously at time T.

[0440] 4002b. Use a beam splitter to split the light beams of the L light-emitting areas.

[0441] The above-mentioned beam splitter is specifically used to split each received beam of light into multiple beams of light.

[0442] 4002c. Use a collimating lens to collimate the light beam from the beam splitter to obtain a collimated light beam.

[0443] 4002d. Receive the reflected light beam from the target object using a receiving unit.

[0444] The reflected light beam of the target object is a light beam obtained by the target object reflecting the collimated light beam.

[0445] 4002e. Obtain a final depth map of the target object according to the TOF corresponding to the light beams emitted by the L light-emitting areas.

[0446] The TOF corresponding to the light beams emitted by the L light-emitting areas may specifically refer to the time difference information between the reception time of the reflected light beams corresponding to the light beams emitted by the L light-emitting areas of the array light source at time T and time T.

[0447] Optionally, the receiving unit includes a receiving lens and a sensor, and the step 4002d of using the receiving unit to receive the reflected light beam of the target object includes: using the receiving lens to converge the reflected light beam of the target object to the sensor.

[0448] The above-mentioned sensor may also be referred to as a sensor array, and the sensor array may be a two-dimensional sensor array.

[0449] Optionally, the resolution of the sensor is greater than P×Q, and the number of light beams obtained after the beam splitter splits the light beam from a light-emitting area of ​​the array light source is P×Q.

[0450] Wherein, the above P and Q are both positive integers. Since the resolution of the upload sensor is greater than the number of beams after the beam splitter splits the beam from a light emitting area of ​​the array light source, the sensor can receive the reflected beam obtained by the target object reflecting the beam from the beam splitter, so that the TOF depth sensing module can achieve normal reception of the reflected beam.

[0451] Optionally, generating a final depth map of the target object in step 4002e specifically includes:

[0452] (1) generating depth maps of L regions of the target object according to the TOFs corresponding to the light beams emitted by the L light-emitting regions;

[0453] (2) A depth map of the target object is synthesized based on the depth maps of L regions of the target object.

[0454] Above Fig.23 The process shown is similar to Fig. 22 The processes shown are all about how to obtain the final depth map of the target object in the first working mode. The main difference is that Fig.23 In the method, a beam splitter is first used to split the light beam emitted by the array light source, and then a collimating lens is used to collimate the split light beam. Fig. 22 In the method, a collimating lens is first used to collimate the light beam emitted by the array light source, and then a beam splitter can be used to split the collimated light beam.

[0455] Combine the following Fig.24 The process of obtaining the final depth map of the target object in the second working mode is introduced in detail.

[0456] Fig.24is a schematic flow chart of obtaining a final depth map of a target object in the second working mode. Fig.24 The process shown includes steps 4003A to 4003E, each of which is described in detail below.

[0457] 4003A. Control M light-emitting areas among the N light-emitting areas of the array light source to emit light at M different times.

[0458] Wherein, the above M is less than or equal to N, and both M and N are positive integers.

[0459] In step 4003A, the light emission of the array light source may be controlled by a control unit. Specifically, the control unit may send control signals to the M light emission areas of the array light source at M times to control the M light emission areas to emit light separately at M different times.

[0460] For example, the array light source includes four independent light-emitting areas A, B, C and D. Then, the control unit can send control signals to the four independent light-emitting areas A, B and C at t0, t1 and t2 respectively, so that the three independent light-emitting areas A, B and C emit light at t0, t1 and t2 respectively.

[0461] 4003B. Use a collimating lens to collimate the light beams generated by the M light-emitting areas at M different moments to obtain collimated light beams.

[0462] In the above step 4003B, the light beams generated by the M light-emitting areas at M different moments are collimated by using a collimating lens. Specifically, it may refer to the light beams generated by the M light-emitting areas at M different moments are collimated by using a collimating lens.

[0463] Assume that the array light source includes four independent light-emitting areas A, B, C and D, and the three independent light-emitting areas A, B and C in the array light source emit light at t0, t1 and t2 respectively under the control of the control unit. Then, the collimating lens can collimate the light beams emitted by the light-emitting areas A, B and C at t0, t1 and t2 respectively.

[0464] By collimating the light beam through a collimating lens, a nearly parallel light beam can be obtained, which can increase the power density of the light beam and further improve the effect of subsequent scanning using the light beam.

[0465] 4003C. Use a beam splitter to split the collimated light beam.

[0466] 4003D. Receive the reflected light beam from the target object using the receiving unit.

[0467] The above-mentioned beam splitter is specifically used to split each received beam of light into multiple beams of light. The above-mentioned reflected beam of the target object is a beam obtained by the target object reflecting the beam from the beam splitter.

[0468] 4003E. Generate M depth maps according to the TOFs corresponding to the light beams emitted by the M light-emitting areas at M different moments.

[0469] The TOF corresponding to the light beams emitted by the M light-emitting areas of the above-mentioned array light source at M different times may specifically refer to the time difference information between the emission time of the light beams emitted by the M light-emitting areas of the array light source at M different times and the reception time of the corresponding reflected light beams.

[0470] 4003F. Obtain a final depth map of the target object according to the M depth maps.

[0471] Optionally, the M depth maps are depth maps corresponding to M region sets of the target object, and there are non-overlapping regions between any two region sets in the M region sets.

[0472] Optionally, the receiving unit includes a receiving lens and a sensor, and the step 4003D of using the receiving unit to receive the reflected light beam of the target object includes: using the receiving lens to converge the reflected light beam of the target object to the sensor.

[0473] The above-mentioned sensor may also be referred to as a sensor array, and the sensor array may be a two-dimensional sensor array.

[0474] Optionally, the resolution of the sensor is greater than or equal to P×Q, and the number of light beams obtained after the beam splitter splits the light beam from a light-emitting area of ​​the array light source is P×Q.

[0475] Wherein, the above P and Q are both positive integers. The resolution of the upload sensor is greater than or equal to the number of light beams after the beam splitter splits the light beam from a light emitting area of ​​the array light source, so that the sensor can receive the reflected light beam obtained by the target object reflecting the light beam from the beam splitter, so that the TOF depth sensing module can achieve normal reception of the reflected light beam.

[0476] Optionally, generating M depth maps in the above step 4003E specifically includes:

[0477] (1) determining the distances between the M regions of the target object and the TOF depth sensing module according to the TOFs corresponding to the light beams emitted by the M light-emitting regions at M different times;

[0478] (2) generating depth maps of the M regions of the target object according to the distances between the M regions of the target object and the TOF depth sensing module;

[0479] (3) A depth map of the target object is synthesized based on the depth maps of the M regions of the target object.

[0480] Above Fig.24 The method shown can be Figure 3 The TOF depth sensor module shown may include Figure 3 The TOF depth sensing module is used to execute the terminal device shown.

[0481] When the relative position relationship between the collimating lens and the beam splitter in the TOF depth sensor module is different, the process of obtaining the final depth map of the target object in the second working mode is also different. Fig.25 The process of obtaining a final depth map of the target object in the second working mode is described.

[0482] Fig.25 is a schematic flow chart of obtaining a final depth map of a target object in the second working mode. Fig.25 The process shown includes steps 4003a to 4003f, each of which is described in detail below.

[0483] 4003a. Control M light-emitting areas among the N light-emitting areas of the array light source to emit light at M different times.

[0484] Wherein, M is less than or equal to N, and both M and N are positive integers.

[0485] In step 4003a, the light emission of the array light source can be controlled by the control unit. Specifically, the control unit can send control signals to the M light-emitting areas of the array light source at M times to control the M light-emitting areas to emit light separately at M different times.

[0486] For example, the array light source includes four independent light-emitting areas A, B, C and D. Then, the control unit can send control signals to the four independent light-emitting areas A, B and C at t0, t1 and t2 respectively, so that the three independent light-emitting areas A, B and C emit light at t0, t1 and t2 respectively.

[0487] 4003b. Use a beam splitter to split the light beams generated by M light-emitting areas at M different moments.

[0488] The above-mentioned beam splitter is specifically used to split each received beam of light into multiple beams of light.

[0489] The above-mentioned use of a beam splitter to split the light beams generated by the M light-emitting areas at M different moments may specifically refer to use of a beam splitter to split the light beams generated by the M light-emitting areas at M different moments.

[0490] For example, the array light source includes four independent light-emitting areas A, B, C and D. Under the control of the control unit, the light-emitting area A emits light at time T0, the light-emitting area B emits light at time T1, and the light-emitting area C emits light at time T2. Then, the beam splitter can perform beam splitting processing on the light beam emitted by the light-emitting area A at time T0, perform beam splitting processing on the light beam emitted by the light-emitting area B at time T1, and perform beam splitting processing on the light beam emitted by the light-emitting area C at time T2.

[0491] 4003c. A collimating lens is used to collimate the light beam from the beam splitter.

[0492] By collimating the light beam through a collimating lens, a nearly parallel light beam can be obtained, which can increase the power density of the light beam and further improve the effect of subsequent scanning using the light beam.

[0493] 4003d. Receive the reflected light beam from the target object using a receiving unit.

[0494] The reflected light beam of the target object is a light beam obtained by the target object reflecting the light beam from the collimating lens.

[0495] 4003e. Generate M depth maps according to the TOFs corresponding to the light beams emitted by the M light-emitting areas at M different moments.

[0496] The TOF corresponding to the light beams emitted by the M light-emitting areas of the above-mentioned array light source at M different times may specifically refer to the time difference information between the emission time of the light beams emitted by the M light-emitting areas of the array light source at M different times and the reception time of the corresponding reflected light beams.

[0497] 4003f. Obtain a final depth map of the target object based on the M depth maps.

[0498] Optionally, the M depth maps are depth maps corresponding to M region sets of the target object, and there are non-overlapping regions between any two region sets in the M region sets.

[0499] Optionally, the receiving unit includes a receiving lens and a sensor, and the step 4003d of using the receiving unit to receive the reflected light beam of the target object includes: using the receiving lens to converge the reflected light beam of the target object to the sensor.

[0500] The above-mentioned sensor may also be referred to as a sensor array, and the sensor array may be a two-dimensional sensor array.

[0501] Optionally, the resolution of the sensor is greater than or equal to P×Q, and the number of light beams obtained after the beam splitter splits the light beam from a light-emitting area of ​​the array light source is P×Q.

[0502] Wherein, the above P and Q are both positive integers. The resolution of the upload sensor is greater than or equal to the number of light beams after the beam splitter splits the light beam from a light emitting area of ​​the array light source, so that the sensor can receive the reflected light beam obtained by the target object reflecting the light beam from the beam splitter, so that the TOF depth sensing module can achieve normal reception of the reflected light beam.

[0503] Optionally, generating M depth maps in the above step 4003e specifically includes:

[0504] (1) determining the distances between the M regions of the target object and the TOF depth sensing module according to the TOFs corresponding to the light beams emitted by the M light-emitting regions at M different times;

[0505] (2) generating depth maps of the M regions of the target object according to the distances between the M regions of the target object and the TOF depth sensing module;

[0506] (3) A depth map of the target object is synthesized based on the depth maps of the M regions of the target object.

[0507] Above Fig.25 The process shown is similar to Fig.24 The processes shown are all about how to obtain the final depth map of the target object in the second working mode. The main difference is that Fig.25 In the method, a beam splitter is first used to split the light beam emitted by the array light source, and then a collimating lens is used to collimate the split light beam. Fig.24 In the method, a collimating lens is first used to collimate the light beam emitted by the array light source, and then a beam splitter can be used to split the collimated light beam.

[0508] Combination of the above Figures 1 to 25 A TOF depth sensing module and an image generation method in an embodiment of the present application are described in detail. Figure 26 to Figure 52 Another TOF depth sensing module and image generation method of an embodiment of the present application are introduced in detail.

[0509] Traditional TOF depth sensing modules generally use mechanical rotation or vibration components to drive optical structures (for example, reflectors, lenses, and prisms, etc.) or the emitting light source itself to rotate or vibrate to change the propagation direction of the laser beam to scan different areas of the target object. However, the size of this TOF depth sensing module is large and is not suitable for installation in some devices with some space (for example, mobile terminals). In addition, this type of TOF depth sensing module generally uses a continuous scanning method for scanning, and the resulting scanning trajectory is generally continuous. The flexibility is poor when scanning the target object, and it is impossible to quickly locate the region of interest (ROI). To this end, the embodiment of the present application provides a method that can irradiate different light beams in different directions without mechanical rotation and vibration, and can quickly locate the scanning area of ​​interest. The following is a detailed description with reference to the accompanying drawings.

[0510] Let's combine Fig.26 A brief introduction is given to the TOF depth sensing module of the embodiment of the present application.

[0511] Fig.26 It is a schematic diagram of using the TOF depth sensing module of an embodiment of the present application to perform distance measurement.

[0512] like Fig.26 As shown, the TOF depth sensing module may include a transmitting end (also known as a projecting end), a receiving end and a control unit, wherein the transmitting end is used to emit an outgoing light beam, and the receiving end is used to receive a reflected light beam of a target object (the reflected light beam is a light beam obtained by the target object reflecting the outgoing light beam), and the control unit may control the transmitting end and the receiving end to respectively emit and receive light beams.

[0513] exist Fig.26 In the invention, the transmitting end may generally include a laser light source, a collimating lens (optional), a polarization filter device, an optical element and a projection lens (optional), and the receiving end may generally include a receiving lens and a sensor, which may be collectively referred to as a receiving unit.

[0514] exist Fig.26 In the process, the TOF corresponding to the outgoing light beam can be recorded by a timing device to calculate the distance from the TOF depth sensing module to the target area, thereby obtaining the final depth map of the target object. The TOF corresponding to the outgoing light beam can refer to the time difference information between the moment when the reflected light beam is received by the receiving unit and the moment when the outgoing light beam is emitted.

[0515] The TOF depth sensing module of the embodiment of the present application can be used for 3D image acquisition. The TOF depth sensing module of the embodiment of the present application can be set in a smart terminal (for example, a mobile phone, a tablet, a wearable device, etc.) for acquiring depth images or 3D images, and can also provide gesture and limb recognition for 3D games or somatosensory games.

[0516] Combine the following Fig. 27 The TOF depth sensing module of the embodiment of the present application is introduced in detail.

[0517] Fig. 27 It is a schematic block diagram of the TOF depth sensing module of an embodiment of the present application.

[0518] Fig. 27 The TOF depth sensing module 200 shown includes a laser light source 210, a polarization filter device 220, an optical element 230, a receiving unit 240, and a control unit 250. The polarization filter device 220 is located between the laser light source 210 and the optical element 230. The modules or units in the TOF depth sensing module 200 are described in detail below.

[0519] Laser source 210:

[0520] The laser light source 210 is used to generate a laser beam. Specifically, the laser light source 210 can generate light of multiple polarization states.

[0521] Optionally, the laser beam emitted by the laser light source 210 is a single beam of quasi-parallel light, and the divergence angle of the laser beam emitted by the laser light source 210 is less than 1°.

[0522] Optionally, the above-mentioned laser light source may be a semiconductor laser light source.

[0523] The laser light source may be a vertical cavity surface emitting laser (VCSEL).

[0524] Optionally, the laser light source may be a Fabry-Perot laser (abbreviated as FP laser).

[0525] Compared with a single VCSEL, a single FP laser can achieve greater power, and its electro-optical conversion efficiency is also higher than that of VCSEL, which can improve the scanning effect.

[0526] Optionally, the wavelength of the laser beam emitted by the laser light source 210 is greater than 900 nm.

[0527] Since the intensity of light greater than 900nm in sunlight is relatively weak, when the wavelength of the laser beam is greater than 900nm, it helps to reduce the interference caused by sunlight and improve the scanning effect of the TOF depth sensing module.

[0528] Optionally, the wavelength of the laser beam emitted by the laser light source 210 is 940 nm or 1550 nm.

[0529] Since the intensity of light near 940nm or 1550nm in sunlight is relatively weak, when the wavelength of the laser beam is 940nm or 1550nm, the interference caused by sunlight can be greatly reduced, and the scanning effect of the TOF depth sensing module can be improved.

[0530] Polarization filter device 220:

[0531] The polarization filter device 220 is used to filter the laser beam to obtain a beam with a single polarization state.

[0532] The light beam of a single polarization state filtered by the polarization filter device 220 is one of the multiple polarization states of the light beam generated by the laser light source 210 .

[0533] For example, the laser light source 210 generates a laser beam including linear polarized light, left-handed circularly polarized light and right-handed circularly polarized light in different directions. Then, the polarization filter device 220 can filter out left-handed circularly polarized light and right-handed circularly polarized light in the laser beam, thereby obtaining a beam of linear polarized light with a polarization state in a specific direction.

[0534] Optical Components 230:

[0535] The optical element 230 is used to adjust the direction of the light beam in a single polarization state.

[0536] The refractive index parameter of the optical element 230 is controllable. When the refractive index of the optical element 230 is different, the optical element 230 can adjust the light beam in a single polarization state to different directions.

[0537] The propagation direction of the laser beam is described below in conjunction with the accompanying drawings. The propagation direction of the laser beam can be defined by a spatial angle. Fig.28 As shown, the spatial angle of the laser beam includes the angle θ between the laser beam and the z-axis direction of the rectangular coordinate system of the exit surface and the angle between its projection on the XY plane and the X-axis direction When scanning with a laser beam, the spatial angle θ or Changes will happen.

[0538] Control unit 250:

[0539] The control unit 250 is used to control the refractive index parameter of the optical element 230 to change the propagation direction of the light beam in a single polarization state.

[0540] The control unit 250 can generate a control signal, which can be a voltage signal or a radio frequency drive signal. The control signal can change the refractive index parameter of the optical element 230, thereby changing the emission direction of the light beam in a single polarization state received by the optical element 20.

[0541] Receiving unit 240:

[0542] The receiving unit 240 is used to receive the reflected light beam from the target object.

[0543] The reflected light beam of the target object is a light beam obtained by the target object reflecting a light beam in a single polarization state.

[0544] Specifically, the light beam in a single polarization state will irradiate the surface of the target object after passing through the optical element 230 . A reflected light beam will be generated due to reflection from the surface of the target object. The reflected light beam can be received by the receiving unit 240 .

[0545] The receiving unit 240 may specifically include a receiving lens 241 and a sensor 242 . The receiving lens 241 is used to receive the reflected light beam and converge the reflected light beam to the sensor 242 .

[0546] In the embodiment of the present application, since the birefringence of the optical element is different and the light beam can be adjusted to different directions, the propagation direction of the light beam can be adjusted by controlling the birefringence parameters of the optical element, thereby realizing the adjustment of the propagation direction of the light beam in a non-mechanical rotation manner, and realizing discrete scanning of the light beam, which can more flexibly measure the depth or distance of the surrounding environment and the target object.

[0547] That is to say, in the embodiment of the present application, by controlling the refractive index parameters of the optical element 230, the spatial angle of the light beam in a single polarization state can be changed, so that the optical element 230 can deflect the propagation direction of the light beam in a single polarization state, and then output an outgoing light beam with a scanning direction and a scanning angle that meet the requirements, thereby realizing discrete scanning, having high flexibility during scanning, and being able to quickly locate the ROI.

[0548] Optionally, the control unit 250 is further configured to generate a depth map of the target object according to the TOF corresponding to the laser beam.

[0549] The TOF corresponding to the above laser beam may specifically refer to the time difference information between the moment when the reflected beam corresponding to the laser beam is received by the receiving unit and the moment when the laser light source emits the laser beam. The reflected beam corresponding to the laser beam may specifically refer to the beam generated after the laser beam reaches the target object after being processed by the polarization filter device and the optical element and reflected after passing through the target object.

[0550] Fig.29 It is a schematic block diagram of the TOF depth sensing module of an embodiment of the present application.

[0551] like Fig.29 As shown, the TOF depth sensing module 200 further includes: a collimating lens 260, which is located between the laser light source 210 and the polarization filter device 220, and is used to collimate the laser beam; the polarization filter device 220 is used to filter the beam after the collimation by the collimating lens to obtain a beam with a single polarization state.

[0552] Optionally, the light emitting area of ​​the laser light source 210 is less than or equal to 5×5 mm 2 .

[0553] Optionally, the clear aperture of the collimating lens is less than or equal to 5 mm.

[0554] Since the above-mentioned laser light source and collimating lens are relatively small in size, the TOF depth sensing module including the above-mentioned devices (laser light source, collimating lens) can be easily integrated into the terminal device, which can reduce the space occupied in the terminal device to a certain extent.

[0555] Optionally, the average output optical power of the TOF depth sensing module 200 is less than 800mw.

[0556] When the average output optical power of the TOF depth sensing module is less than or equal to 800mw, the power consumption of the TOF depth sensing module is relatively small, and it is convenient to be installed in devices that are sensitive to power consumption, such as terminal devices.

[0557] Fig.30 It is a schematic diagram of the TOF depth sensing module of an embodiment of the present application scanning a target object.

[0558] like Fig.30As shown, the optical element 230 can emit an outgoing light beam 1 at time T0. At a time T1, if the scanning direction and the scanning angle need to be changed, the optical element can be directly controlled to emit an outgoing light beam 2 at time T1. At the next time T2, if the scanning direction and the scanning angle need to be changed, the optical element can be controlled to emit an outgoing light beam 3 at time T2 by sending a control signal. The TOF depth sensing module 200 can directly output outgoing light beams in different directions at different times, thereby realizing the scanning of the target object.

[0559] Combine the following Fig.31 The effect of the TOF depth sensing module 200 in achieving discrete scanning is described in detail.

[0560] Fig.31 It is a schematic diagram of the scanning trajectory of the TOF depth sensing module of an embodiment of the present application.

[0561] like Fig.31 As shown, the TOF depth sensing module can start scanning from scanning point A. When it is necessary to switch from scanning point A to scanning point B for scanning, the optical element 230 can be directly controlled by the control unit 250 so that the outgoing light beam directly irradiates scanning point B without gradually moving from scanning point A to scanning point B (no need to move from A to B along the dotted line between AB in the figure). Similarly, when it is necessary to switch from scanning point B to scanning point C for scanning, the optical element 230 can also be controlled by the control unit 250 so that the outgoing light beam directly irradiates scanning point C without gradually moving from scanning point B to scanning point C (no need to move from B to C along the dotted line between BC in the figure).

[0562] Therefore, the TOF depth sensing module 200 can realize discrete scanning, has high scanning flexibility, and can quickly locate the area that needs to be scanned.

[0563] Since the TOF depth sensing module 200 can realize discrete scanning, the TOF depth sensing module 200 can use multiple scanning trajectories to scan a certain area during scanning. The selection of scanning methods is more flexible and also facilitates the timing control design of the TOF depth sensing module 200.

[0564] The following is combined Fig.32 , taking a 3×3 two-dimensional dot matrix as an example to illustrate the scanning method of the TOF depth sensing module 200.

[0565] Fig.32 It is a schematic diagram of the scanning method of the TOF depth sensing module of an embodiment of the present application.

[0566] like Fig.32As shown, the TOF depth sensing module 200 can start scanning from the point in the upper left corner of the two-dimensional dot matrix and scan to the point in the lower right corner of the two-dimensional dot matrix. Such scanning methods include scanning methods A to scanning methods F. In addition to scanning from the point in the upper left corner of the two-dimensional dot matrix, scanning can also be started from the center point of the two-dimensional dot matrix until all the points of the two-dimensional dot matrix are scanned, thereby completing the entire scanning of the two-dimensional dot matrix. Such scanning methods include scanning methods G to scanning methods J.

[0567] In addition, it is also possible to start scanning from any point in the two-dimensional array until all points in the two-dimensional array are scanned. Fig.32 As shown in the scanning method K in , the scanning can start from the point in the first row and second column of the two-dimensional array until the center point in the two-dimensional array is scanned, thereby completing the entire scanning of the two-dimensional array dot matrix.

[0568] Optionally, the optical element 230 is any one of a liquid crystal polarization grating, an optical phased array, an electro-optical device, and an acousto-optical device.

[0569] The specific structure of the optical element 230 is described in detail below in conjunction with the accompanying drawings.

[0570] The first case: the optical element 230 is a liquid crystal polarization grating (LCPG). In the first case, the birefringence of the optical element 230 is controllable. When the birefringence of the optical element is different, the optical element can adjust a light beam in a single polarization state to different directions.

[0571] Liquid crystal polarization grating is a new type of grating device based on the principle of geometric phase. It acts on circularly polarized light and has electro-optical tunability and polarization tunability.

[0572] Liquid crystal polarization grating is a grating formed by the periodic arrangement of liquid crystal molecules. Its general production method is to control the liquid crystal molecule director (the long axis direction of the liquid crystal molecule) to change linearly and periodically in one direction through the photo-controlled orientation technology. By controlling the polarization state of the incident light, the circular deflected light can be diffracted to the +1st or -1st order, and the beam deflection can be achieved by switching between the additive diffraction order and the zeroth order.

[0573] Fig.33 It is a schematic block diagram of the TOF depth sensing module of an embodiment of the present application.

[0574] like Fig.33 As shown, the optical element 230 is a liquid crystal polarization grating, and the control unit 250 can control the laser light source to emit a laser beam to the liquid crystal polarization grating, and control the liquid crystal polarization grating to deflect the direction of the laser beam through a control signal to obtain an outgoing beam.

[0575] Optionally, the liquid crystal polarization grating includes a horizontal LCPG component and a vertical LCPG component.

[0576] Fig.34 It is a schematic block diagram of the TOF depth sensing module of an embodiment of the present application.

[0577] like Fig.34 As shown, the liquid crystal polarization grating is composed of a horizontal LCPG component and a vertical LCPG component. The horizontal LCPG component can realize discrete random scanning in the horizontal direction, and the vertical LCPG component can realize discrete random scanning in the vertical direction. When the horizontal LCPG component and the vertical LCPG component are combined together, two-dimensional discrete random scanning in the horizontal and vertical directions can be realized.

[0578] It should be understood that Fig.34 Only the case where the LCPG in the horizontal direction is in front and the LCPG in the vertical direction is in the back is shown (the distance between the LCPG in the horizontal direction and the laser light source is smaller than the distance between the LCPG in the vertical direction and the laser light source). In fact, in the present application, in the liquid crystal polarization grating, the LCPG in the vertical direction may be in front and the LCPG in the horizontal direction may be in the back (the distance between the LCPG in the vertical direction and the laser light source is smaller than the distance between the LCPG in the horizontal direction and the laser light source).

[0579] In the present application, when the liquid crystal polarization grating includes a LCPG component in the horizontal direction and a LCPG component in the vertical direction, two-dimensional discrete random scanning in the horizontal direction and the vertical direction can be achieved.

[0580] Optionally, in the first case, the liquid crystal polarization grating may further include a transverse polarization control plate and a longitudinal polarization control plate.

[0581] When the liquid crystal polarization grating includes a polarization control plate, the polarization state of the light beam can be controlled.

[0582] Fig.35 It is a schematic diagram of the structure of the liquid crystal polarization grating according to an embodiment of the present application.

[0583] like Fig.35 As shown, the liquid crystal polarization grating includes not only a transverse LCPG and a longitudinal LCPG, but also a transverse polarization control plate and a longitudinal polarization control plate. Fig.35 In the embodiment, the transverse LCPG is located between the transverse polarization control plate and the longitudinal polarization control plate, and the longitudinal polarization control plate is located between the transverse LCPG and the longitudinal LCPG.

[0584] Fig.36 It is a schematic diagram of the structure of the TOF depth sensing module of an embodiment of the present application.

[0585] like Fig.36 As shown, the structure of the liquid crystal polarization grating in the TOF depth sensing module is as follows Fig.35 As shown, the distances between the transverse polarization control plate, the transverse LCPG, the longitudinal polarization control plate and the longitudinal LCPG and the laser light source increase in sequence.

[0586] Optionally, the above Fig.35 The various components in the liquid crystal polarization grating shown in the figure can be combined in the following ways.

[0587] Combination 1: 124;

[0588] Combination 2: 342;

[0589] Combination 3:3412.

[0590] Among them, in the above combination mode 1, 1 can represent the closely attached transverse polarization control sheet and the longitudinal polarization control sheet, in which case the two closely attached polarization control sheets are equivalent to one polarization control sheet, therefore, 1 is used in the combination mode 1 to represent the closely attached transverse polarization control sheet and the longitudinal polarization control sheet. Similarly, in the above combination mode 2, 3 can represent the closely attached transverse polarization control sheet and the longitudinal polarization control sheet, in which case the two closely attached polarization control sheets are equivalent to one polarization control sheet, therefore, 3 is used in the combination mode 2 to represent the closely attached transverse polarization control sheet and the longitudinal polarization control sheet.

[0591] When the optical element 230 of combination 1 or combination 2 is placed in the TOF depth sensing module, the transverse polarization control plate or the longitudinal polarization control plate is located on the side close to the laser light source, and the transverse LCPG and the longitudinal LCPG are located on the side away from the laser light source.

[0592] When the optical element 230 of combination mode 3 is placed in the TOF depth sensing module, the distances between the longitudinal polarization control plate, the longitudinal LCPG, the transverse polarization control plate, the transverse LCPG and the laser light source become larger in sequence.

[0593] It should be understood that the above three combinations of liquid crystal polarization gratings and Fig.35 The combination is only an example. In fact, the components in the optical element of the present application can also be combined in different ways. As long as the distance between the transverse polarization control plate and the laser light source is smaller than the distance between the transverse LCPG and the laser light source, and the distance between the transverse polarization control plate and the laser light source is smaller than the distance between the transverse LCPG and the laser light source.

[0594] like Fig.37 As shown, by inputting a periodic control signal ( Fig.37In the embodiment, the period of the control signal is Λ), which can periodically change the physical properties of the liquid crystal polarization grating. Specifically, it can change the arrangement of the liquid crystal molecules inside the liquid crystal polarization grating (the liquid crystal molecules are generally rod-shaped, and the direction of the liquid crystal molecules will change due to the influence of the control signal), thereby achieving the deflection of the direction of the laser beam.

[0595] When the liquid crystal polarization grating and the polarizer are combined at the same time, the light beam can be controlled in different directions.

[0596] like Fig.38 As shown, the incident light can be controlled in three different directions by the left-handed and right-handed circular polarizers and the voltage control of the LCPG. The deflection angle of the outgoing light can be determined according to the following diffraction grating equation.

[0597]

[0598] In the above diffraction grating equation, θ m is the direction angle of the m-order output light, λ is the wavelength of the laser, Λ is the period of LCPG, and θ is the incident angle of the incident light. From the above diffraction grating equation, we can know that the deflection angle θ m The size depends on the size of the LCPG grating period, the wavelength and the size of the incident angle. Here, m can only be 0 or ±1. When m is 0, it means that the direction is not deflected and the direction remains unchanged. When m is 1, it means that it deflects to the left or counterclockwise relative to the incident direction. When m is -1, it means that it deflects to the right or clockwise relative to the incident direction (the meanings of m +1 and m -1 can also be opposite).

[0599] A single LCPG can achieve three angles of deflection, and thus obtain three angles of outgoing light beams. Therefore, by cascading multiple layers of LCPG, more angles of outgoing light beams can be obtained. Therefore, a combination of N layers of polarization control plates (polarization control plates are used to control the polarization of incident light and achieve the conversion of left-handed light and right-handed light) and N layers of LCPG can theoretically achieve three angles of deflection. N Deflection angle.

[0600] For example, Fig.35 As shown, the optical elements of the TOF depth sensing module are composed of devices 1, 2, 3 and 4, wherein devices 1, 2, 3 and 4 represent transverse polarization control plates, transverse LCPG, longitudinal polarization control plates and longitudinal LCPG, respectively. The deflection direction and angle of the light beam can be controlled by controlling the voltage of each group of polarization control plates and LCPG.

[0601] Taking 3x3 point scanning as an example, Fig.36 The devices 1, 2, 3 and 4 shown are applied as follows Fig.39 The voltage signal shown ( Fig.391, 2, 3, and 4 in the figure represent the Fig.36 The voltage signals on the devices 1, 2, 3 and 4 shown in the figure can control the laser beam emitted by the laser light source to achieve Fig.40 The scanning trajectory is shown.

[0602] Specifically, assuming that the incident light is left-handed circularly polarized light, the transverse LCPG deflects leftward under the left-handed incident light, and the longitudinal LCPG deflects downward under the left-handed incident light. The deflection direction of the light beam at each moment is described in detail below.

[0603] When the two ends of the transverse polarization control plate are high voltage signals, the polarization state of the light beam passing through the transverse polarization control plate remains unchanged, and when the two ends of the transverse polarization control plate are low voltage signals, the polarization state of the light beam passing through the transverse polarization control plate will change. Similarly, when the two ends of the longitudinal polarization control plate are high voltage signals, the polarization state of the light beam passing through the longitudinal polarization control plate remains unchanged, and when the two ends of the longitudinal polarization control plate are low voltage signals, the polarization state of the light beam passing through the longitudinal polarization control plate will change.

[0604] At time 0, the incident light of device 1 is left-handed circularly polarized light. Since device 1 is applied with a low voltage, the light emitted after passing through device 1 is right-handed circularly polarized light; the incident light of device 2 is right-handed circularly polarized light. Since device 2 is applied with a high voltage, the light emitted after passing through device 2 is still right-handed circularly polarized light; the incident light of device 3 is right-handed circularly polarized light. Since device 3 is applied with a low voltage, the light emitted after passing through device 3 is left-handed circularly polarized light; the incident light of device 4 is left-handed circularly polarized light. Since device 4 is applied with a high voltage, the light emitted after passing through device 4 is still left-handed circularly polarized light; therefore, at time 0, after passing through device 1 to device 4, the direction of the incident light does not change, and the polarization state does not change. Fig.40 As shown, the scanning point corresponding to time 0 is Fig.40 The position shown by the center is t0.

[0605] At time t0, the incident light of device 1 is left-handed circularly polarized light. Since device 1 is applied with a high voltage, the light emitted after passing through device 1 is still left-handed circularly polarized light; the incident light of device 2 is left-handed circularly polarized light. Since device 2 is applied with a low voltage, the light emitted after passing through device 2 is right-handed circularly polarized light deflected to the left; the incident light of device 3 is right-handed circularly polarized light deflected to the left. Since device 3 is applied with a low voltage, the light emitted after passing through device 3 is left-handed circularly polarized light deflected to the left; the incident light of device 4 is left-handed circularly polarized light deflected to the left. Since device 4 is applied with a high voltage, the light emitted after passing through device 4 is still left-handed circularly polarized light deflected to the left; that is, relative to time 0, the light beam emitted after passing through device 4 at time t0 is deflected to the left. Fig.40 The corresponding scanning point in is the position shown as t0.

[0606] At time t1, the incident light of device 1 is left-handed circularly polarized light. Since device 1 is applied with a high voltage, the light emitted after passing through device 1 is still left-handed circularly polarized light; the incident light of device 2 is left-handed circularly polarized light. Since device 2 is applied with a low voltage, the light emitted after passing through device 2 is right-handed circularly polarized light deflected to the left; the incident light of device 3 is right-handed circularly polarized light deflected to the left. Since device 3 is applied with a high voltage, the light emitted after passing through device 3 is right-handed circularly polarized light deflected to the left; the incident light of device 4 is right-handed circularly polarized light deflected to the left. Since device 4 is applied with a low voltage, the light emitted after passing through device 4 is left-handed circularly polarized light deflected to the left and upward; that is, relative to time 0, the light beam emitted after passing through device 4 at time t1 is deflected to the left and upward. Fig.40 The corresponding scanning point in is the position shown as t1.

[0607] At time t2, the incident light of device 1 is left-handed circularly polarized light. Since device 1 is applied with a low voltage, the light emitted after passing through device 1 is right-handed circularly polarized light; the incident light of device 2 is right-handed circularly polarized light. Since device 2 is applied with a high voltage, the light emitted after passing through device 2 is still right-handed circularly polarized light; the incident light of device 3 is right-handed circularly polarized light. Since device 3 is applied with a high voltage, the light emitted after passing through device 3 is still right-handed circularly polarized light; the incident light of device 4 is right-handed circularly polarized light. Since device 4 is applied with a low voltage, the light emitted after passing through device 4 is left-handed circularly polarized light deflected upward; that is, relative to time 0, the light beam emitted after passing through device 4 at time t2 is deflected upward. Fig.40 The corresponding scanning point in is the position shown by t2.

[0608] At time t3, the incident light of device 1 is left-handed circularly polarized light, and since a low voltage is applied to device 1, the light emitted after passing through device 1 is right-handed circularly polarized light; the incident light of device 2 is right-handed circularly polarized light, and since a low voltage is applied to device 2, the light emitted after passing through device 2 is right-handed circularly polarized light deflected to the right; the incident light of device 3 is right-handed circularly polarized light deflected to the right, and since a low voltage is applied to device 3, the light emitted after passing through device 3 is left-handed circularly polarized light deflected to the right; the incident light of device 4 is left-handed circularly polarized light deflected to the right, and since a low voltage is applied to device 4, the light emitted after passing through device 4 is left-handed circularly polarized light deflected to the right and upward; that is, relative to time 0, the light beam emitted after passing through device 4 at time t3 is deflected to the right and upward, Fig.40 The corresponding scanning point in is the position shown as t3.

[0609] At time t4, the incident light of device 1 is left-handed circularly polarized light. Since device 1 is applied with a low voltage, the light emitted after passing through device 1 is right-handed circularly polarized light; the incident light of device 2 is right-handed circularly polarized light. Since device 2 is applied with a low voltage, the light emitted after passing through device 2 is left-handed circularly polarized light deflected to the right; the incident light of device 3 is left-handed circularly polarized light deflected to the right. Since device 3 is applied with a low voltage, the light emitted after passing through device 3 is right-handed circularly polarized light deflected to the right; the incident light of device 4 is right-handed circularly polarized light deflected to the right. Since device 4 is applied with a high voltage, the light emitted after passing through device 4 is still right-handed circularly polarized light deflected to the right; that is, relative to time 0, the light beam emitted after passing through device 4 at time t0 is deflected to the right. Fig.40 The corresponding scanning point in is the position shown as t4.

[0610] At time t5, the incident light of device 1 is left-handed circularly polarized light, and since device 1 is applied with a low voltage, the light emitted after passing through device 1 is right-handed circularly polarized light; the incident light of device 2 is right-handed circularly polarized light, and since device 2 is applied with a low voltage, the light emitted after passing through device 2 is right-handed circularly polarized light deflected to the right; the incident light of device 3 is right-handed circularly polarized light deflected to the right, and since device 3 is applied with a high voltage, the light emitted after passing through device 3 is still right-handed circularly polarized light deflected to the right; the incident light of device 4 is right-handed circularly polarized light deflected to the right, and since device 4 is applied with a low voltage, the light emitted after passing through device 4 is left-handed circularly polarized light deflected to the right and downward; that is, relative to time 0, the light beam emitted after passing through device 4 at time t5 is deflected to the right and downward. Fig.40 The corresponding scanning point in is the position shown as t5.

[0611] At time t6, the incident light of device 1 is left-handed circularly polarized light. Since device 1 is applied with a low voltage, the light emitted after passing through device 1 is right-handed circularly polarized light; the incident light of device 2 is right-handed circularly polarized light. Since device 2 is applied with a high voltage, the light emitted after passing through device 2 is still right-handed circularly polarized light; the incident light of device 3 is right-handed circularly polarized light. Since device 3 is applied with a low voltage, the light emitted after passing through device 3 is left-handed circularly polarized light; the incident light of device 4 is left-handed circularly polarized light. Since device 4 is applied with a low voltage, the light emitted after passing through device 4 is right-handed circularly polarized light deflected downward; that is, relative to time 0, the light beam emitted after passing through device 4 at time t6 is deflected downward. Fig.40 The corresponding scanning point in is the position shown as t6.

[0612] At time t7, the incident light of device 1 is left-handed circularly polarized light, and since device 1 is applied with a high voltage, the light emitted after passing through device 1 is still left-handed circularly polarized light; the incident light of device 2 is left-handed circularly polarized light, and since device 2 is applied with a low voltage, the light emitted after passing through device 2 is right-handed circularly polarized light deflected to the left; the incident light of device 3 is right-handed circularly polarized light deflected to the left, and since device 3 is applied with a low voltage, the light emitted after passing through device 3 is left-handed circularly polarized light deflected to the left; the incident light of device 4 is left-handed circularly polarized light deflected to the left, and since device 4 is applied with a low voltage, the light emitted after passing through device 4 is right-handed circularly polarized light deflected to the left and downward; that is, relative to time 0, the light beam emitted after passing through device 4 at time t7 is deflected to the left and upward, Fig.40 The corresponding scanning point in is the position shown as t7.

[0613] It should be understood that this is just a combination of Fig.39 and Fig.40 The possible scanning trajectories of the TOF depth sensing module are explained. Arbitrary discrete random scanning can be achieved by changing the voltage of each group of polarization control plates and LCPG.

[0614] For example, by changing the voltage of each set of polarization control plates and LCPG, Fig.32 Various scanning trajectories are shown.

[0615] When using traditional laser radar to scan a target object, it is often necessary to first perform a coarse scan of the target area, and then perform a fine scan with higher resolution after the region of interest (ROI) is found. However, the TOF depth sensing module of the embodiment of the present application can realize discrete scanning, so it can directly locate the region of interest for fine scanning, which can greatly save the time required for fine scanning.

[0616] For example, Fig.41 As shown, the total number of points in the area to be scanned (the entire rectangular area including the human body contour) is M, and the ROI ( Fig.41 The image area located within the human body contour image occupies 1 / N of the total area of ​​the area to be scanned.

[0617] In the right Fig.41When scanning the area to be scanned as shown, it is assumed that the point scanning rate of the traditional laser radar and the laser scanning radar of the embodiment of the present application is K points / second, and a fine scan is required when scanning the ROI area, and the resolution during the fine scan needs to be increased to four times the original (that is, 4K points / second). Then, the time required to complete the fine scan of the ROI using the TOF depth sensing module of the embodiment of the present application is t1, and the time required to complete the fine scan of the ROI using the traditional laser radar is t2. Since the TOF depth sensing module of the embodiment of the present application can realize discrete scanning, it can directly locate the ROI and perform a fine scan of the ROI, and the required scanning time is shorter. However, the traditional laser radar performs linear scanning, and it is difficult to accurately locate the ROI. Therefore, the traditional laser radar needs to perform a fine scan of the entire area to be scanned, which greatly increases the scanning time. Fig.42 As shown, the TOF depth sensing module of the embodiment of the present application can directly locate the ROI and perform a fine scan of the ROI (by Fig.42 It can be seen that the density of scanning points in the ROI is significantly greater than the density of scanning points outside the ROI).

[0618] In addition, the above t1 and t2 can be calculated using the following two formulas (2) and (3) respectively.

[0619]

[0620] It can be seen from the above formulas (2) and (3) that the time required for fine scanning of ROI using the TOF depth sensing module of the embodiment of the present application is only 1 / N of the time required for fine scanning of ROI by a traditional lidar, which greatly shortens the time required for fine scanning of ROI.

[0621] Since the TOF depth sensing module of the embodiment of the present application can realize discrete scanning, the TOF depth sensing module of the embodiment of the present application can realize fine scanning of ROI areas of any shape (cars, people, buildings and random patches), especially some asymmetric areas and discrete ROI blocks. In addition, the TOF depth sensing module of the embodiment of the present application can also realize uniform or non-uniform distribution of point density in the scanning area.

[0622] The second situation: the optical element 230 is an electro-optical device.

[0623] In the second case, when the optical element 230 is an electro-optical device, the control signal can be a voltage signal, which can be used to change the refractive index of the electro-optical device so that the electro-optical device deflects the laser light beam in different directions while keeping its position relative to the laser light source unchanged, thereby obtaining an output light beam whose scanning direction matches the control signal.

[0624] Alternatively, if Fig.43 As shown, the electro-optic device may include a transverse electro-optic crystal (horizontally deflected electro-optic crystal) and a longitudinal electro-optic crystal (vertically deflected electro-optic crystal). The transverse electro-optic crystal can deflect the laser beam in the horizontal direction, and the longitudinal electro-optic crystal can deflect the laser beam in the vertical direction.

[0625] Optionally, the electro-optical crystal may be any one of a potassium tantalate niobate (KTN) crystal, a deuterated potassium dihydrogen phosphate (DKDP) crystal, and a lithium niobate (LN) crystal.

[0626] The working principle of the electro-optical crystal is briefly introduced below with reference to the accompanying drawings.

[0627] like Fig.44 As shown, when a voltage signal is applied to the electro-optic crystal, due to the second-order photoelectric effect of the electro-optic crystal, a refractive index difference will be generated in the electro-optic crystal (that is, the refractive index of different regions in the electro-optic crystal will be different), thereby deflecting the incident light beam, as shown in FIG. Fig.44 As shown, the outgoing light beam is deflected to a certain extent relative to the direction of the incident light beam.

[0628] The deflection angle of the outgoing light beam relative to the incident light beam can be calculated according to the following formula (4).

[0629]

[0630] In the above formula (4), θ max represents the maximum deflection angle of the outgoing beam relative to the incident beam, n is the refractive index of the electro-optical crystal, g 11y is the second-order electro-optic coefficient, E max represents the maximum electric field strength that can be applied to the electro-optical crystal, is the second-order electro-optic coefficient gradient in the y direction.

[0631] From the above formula (4), it can be seen that by adjusting the intensity of the external electric field (that is, adjusting the voltage applied to the electro-optical crystal), the deflection angle of the light beam can be controlled, thereby scanning the target area. In addition, in order to achieve a larger deflection angle, multiple electro-optical crystals can be cascaded.

[0632] like Fig.43 As shown, the optical element includes a horizontal deflection electro-optic crystal and a vertical deflection electro-optic crystal. The two electro-optic crystals are responsible for the horizontal and vertical beam deflection respectively. Fig.45After the control voltage signal is shown, the following can be achieved: Fig.46 Specifically, in Fig.45 In the figure, 1 and 2 represent the control voltage signals applied to the horizontal deflection electro-optical crystal and the vertical deflection electro-optical crystal, respectively.

[0633] The third situation: the optical element 230 is an acousto-optic device.

[0634] like Fig.47 As shown, the optical element 230 is an acousto-optic device. The acousto-optic device may include a transducer. When the optical element 230 is an acousto-optic device, the control signal may be a radio frequency control signal. The radio frequency control signal may be used to control the transducer to generate sound waves of different frequencies to change the refractive index of the acousto-optic device, thereby causing the acousto-optic device to deflect the laser beam in different directions while keeping its position relative to the laser light source unchanged, thereby obtaining an outgoing beam whose scanning direction matches the control signal.

[0635] like Fig.48 As shown, the acousto-optic device includes an absorber, quartz and a piezoelectric transducer. After the acousto-optic device receives an electrical signal, the piezoelectric transducer can generate an acoustic wave signal under the action of the electrical signal. When the acoustic wave signal is transmitted in the acousto-optic device, it will change the refractive index distribution of the quartz, thereby forming a grating, so that the quartz can deflect the incident light beam at a certain angle. When the control signals input at different times are different, the acousto-optic device can generate outgoing light beams in different directions at different times. Fig.48 As shown, the deflection direction of the outgoing light beam of the quartz at different times (T0, T1, T2, T3 and T4) can be different.

[0636] When the electrical signal incident on the acousto-optic device is a periodic signal, the refractive index distribution of the quartz in the acousto-optic device changes periodically, thereby forming a periodic grating. The periodic grating can be used to achieve periodic deflection of the incident light beam.

[0637] In addition, the intensity of the outgoing light of the acousto-optic device is directly related to the power of the RF control signal input to the acousto-optic device, and the diffraction angle of the incident light beam is also directly related to the frequency of the RF control signal. By changing the frequency of the RF control signal, the angle of the outgoing light beam can also be adjusted accordingly. Specifically, the deflection angle of the outgoing light beam relative to the incident light beam can be determined according to the following formula (5).

[0638]

[0639] In the above formula (5), θ is the angle of deflection of the outgoing beam relative to the incident beam, λ is the wavelength of the incident beam, and f s is the frequency of the RF control signal, v sis the speed of the sound wave. Therefore, the optical deflector can scan the laser beam in a larger angle range and accurately control the exit angle of the laser beam.

[0640] The fourth situation: the optical element 230 is an optical phase array (OPA) device.

[0641] The following is a detailed description of the case where the optical element 230 is an OPA device in pairs 49 and 50 in conjunction with the accompanying drawings.

[0642] like Fig.49 As shown, the optical element 230 is an OPA device, through which the incident light beam can be deflected, thereby obtaining an outgoing light beam whose scanning direction matches the control signal.

[0643] OPA devices are generally composed of a one-dimensional or two-dimensional phase shifter array. When there is no phase difference between the phase shifters, the time for light to arrive at the equal phase plane is the same, the light propagates forward, no interference occurs, and thus no beam deflection occurs.

[0644] After each phase shifter adds a phase difference (taking the example of giving each optical signal a uniform phase difference, the phase difference between the second waveguide and the first waveguide is Δ, the phase difference between the third waveguide and the first waveguide is 2Δ, and so on), the equal phase plane is no longer perpendicular to the waveguide direction, but has a certain deflection. The beams that meet the equal phase relationship will coherently constructive, and the beams that do not meet the equal phase condition will cancel each other out. Therefore, the direction of the light beam is always perpendicular to the equal phase plane.

[0645] like Fig.50 As shown in the figure, assuming that the spacing between adjacent waveguides is d, the optical path difference of the light beams output by adjacent waveguides to reach the equal phase plane is ΔR = d·sinθ. Among them, θ represents the beam deflection angle. Since this optical path difference is caused by the phase difference of the array element, ΔR = Δ·λ / 2π, so the beam can be deflected by introducing the phase difference in the array element. This is the principle of OPA to deflect the beam.

[0646] Therefore, the deflection angle θ = arcsin (Δ·λ / (2π*d)). By controlling the phase difference of adjacent phase shifters, such as π / 12 and π / 6, the beam deflection angle is arcsin (λ / (24d)) and arcsin (λ / (12d)). In this way, by controlling the phase of the phase shifter array, deflection in any two-dimensional direction can be achieved. The phase shifter can be made of liquid crystal material, and different phase differences can be generated by applying different voltages.

[0647] Alternatively, if Fig.51 As shown, the TOF depth sensing module 200 further includes:

[0648] The collimating lens 260 is located between the laser light source 210 and the polarization filter device 220. The collimating lens 260 is used to collimate the laser beam; the polarization filter device 220 is used to filter the beam processed by the collimating lens 260 to obtain a beam with a single polarization state.

[0649] In addition, the collimating lens 260 can also be located between the polarization filter device 220 and the optical element 230. In this case, the polarization filter device 220 first performs polarization filtering on the light beam generated by the laser light source to obtain a light beam with a single polarization state. Next, the collimating lens 260 collimates the light beam with a single polarization state.

[0650] Optionally, the collimating lens 260 can also be located on the right side of the optical element 230 (the distance between the collimating lens 260 and the laser light source 210 is greater than the distance between the optical element 230 and the laser light source 210). In this case, after the optical element 230 adjusts the direction of the light beam with a single polarization state, the collimating lens 260 collimates the light beam with a single polarization state after the direction adjustment.

[0651] Combination of the above Figure 26 to Figure 51 The TOF depth sensing module 200 of the embodiment of the present application is introduced in detail. Fig.52 The image generation method of the embodiment of the present application is described.

[0652] Fig.52 It is a schematic flowchart of the image generation method of an embodiment of the present application.

[0653] Fig.52 The method shown can be executed by the TOF depth sensing module of the embodiment of the present application or a terminal device including the TOF depth sensing module of the embodiment of the present application. Specifically, Fig.52 The method shown can be Fig. 27 The TOF depth sensing module 200 shown may include Fig. 27 The TOF depth sensing module 200 is shown to be executed by the terminal device. Fig.52 The method shown includes steps 4001 to 4005, each of which is described in detail below.

[0654] 5001. Control the laser light source to generate a laser beam.

[0655] Among them, the above-mentioned laser light source can generate light with multiple polarization states.

[0656] For example, the above laser light source can generate light in various polarization states, such as linear polarization, left-handed circular polarization, and right-handed circular polarization.

[0657] 5002. Filter the laser beam using a polarization filter device to obtain a beam with a single polarization state.

[0658] The single polarization state may be any one of linear polarization, left-handed circular polarization and right-handed circular polarization.

[0659] For example, in step 5001, the laser light beam generated by the laser light source includes linearly polarized light, left-handed circularly polarized light and right-handed circularly polarized light. Then, in step 5002, the polarization states of left-handed circularly polarized light, right-handed circularly polarized light and right-handed circularly polarized light in the laser light beam can be filtered out, and only linear polarized light in a specific direction is retained. Optionally, the polarization filter device can also include a 1 / 4 wave plate to convert the filtered linear polarized light into left-handed circularly polarized light (or right-handed circularly polarized light).

[0660] 5003. Control the optical element to have different birefringence parameters at M different moments to obtain outgoing light beams in M ​​different directions.

[0661] The birefringence parameters of the optical element are controllable. When the birefringence of the optical element is different, the optical element can adjust the light beam in a single polarization state to different directions. The M is a positive integer greater than 1. The M reflected light beams are light beams obtained by the target object reflecting the outgoing light beams in M ​​different directions.

[0662] At this time, the optical element may be a liquid crystal polarization grating. For details about the liquid crystal polarization grating, please refer to the description of the first case above.

[0663] Optionally, the optical element having different birefringence parameters at M moments may specifically include the following two situations:

[0664] Case 1: The birefringence parameters of the optical element at any two of the M moments are different;

[0665] Case 2: There are at least two moments in the M moments for the optical element, and the birefringence parameters of the optical element at the at least two moments are different.

[0666] In case 1, assuming that M=5, the optical element corresponds to 5 different birefringence parameters at 5 moments respectively.

[0667] In case 2, assuming that M=5, then the optical element only needs to have two moments in the five moments to correspond to different birefringence parameters.

[0668] 5004. Receive M reflected light beams using a receiving unit.

[0669] 5005. Generate a depth map of the target object according to the TOFs corresponding to the outgoing light beams in M ​​different directions.

[0670] The TOF corresponding to the above-mentioned M outgoing light beams in different directions may specifically refer to the time difference information between the moment when the reflected light beam corresponding to the above-mentioned M outgoing light beams in different directions is received by the receiving unit and the moment when the above-mentioned M outgoing light beams in different directions are emitted.

[0671] Assuming that the M outgoing light beams in different directions include the outgoing light beam 1, the reflected light beam corresponding to the outgoing light beam 1 may refer to a light beam generated after the outgoing light beam 1 reaches the target object and is reflected after passing through the target object.

[0672] In the embodiment of the present application, since the birefringence of the optical element is different and the light beam can be adjusted to different directions, the propagation direction of the light beam can be adjusted by controlling the birefringence parameters of the optical element, thereby realizing the adjustment of the propagation direction of the light beam in a non-mechanical rotation manner, and realizing discrete scanning of the light beam, which can more flexibly measure the depth or distance of the surrounding environment and the target object.

[0673] Optionally, generating a depth map of the target object in step 5005 specifically includes:

[0674] 5005a. Determine the distances between M regions of the target object and the TOF depth sensing module according to the TOFs corresponding to the outgoing light beams in M ​​different directions.

[0675] 5005b. Generate depth maps of the M regions of the target object according to the distances between the M regions of the target object and the TOF depth sensing module; and synthesize the depth map of the target object according to the depth maps of the M regions of the target object.

[0676] exist Fig.52 In the method shown, the light beam can also be collimated.

[0677] Optionally, before the above step 5002, Fig.52 The method shown also includes:

[0678] 5006. Perform collimation processing on the laser beam to obtain a collimated beam.

[0679] After the laser beam is collimated, a beam of light in a single polarization state is obtained in the above step 5002, which specifically includes: using a polarization filter device to filter the collimated beam to obtain light in a single polarization state.

[0680] Before using a polarization filter device to filter the laser beam to obtain a beam of a single polarization state, by collimating the laser beam, a nearly parallel beam can be obtained, which can increase the power density of the beam and further improve the effect of subsequent scanning using the beam.

[0681] The light beam after the collimation process may be a quasi-parallel light with a divergence angle less than 1 degree.

[0682] It should be understood that Fig.52 In the method shown, collimation processing can also be performed on a light beam in a single polarization state. Specifically, Fig.52 The method shown also includes:

[0683] 5007. Perform collimation processing on the light beam in a single polarization state to obtain a collimated light beam.

[0684] The above step 5007 may be located between step 5002 and step 5003 , or the above step 5007 may be located between step 5003 and step 5004 .

[0685] When the above-mentioned step 5007 is located between step 5002 and step 5003, after the polarization filter device filters the laser light beam generated by the laser light source, a light beam in a single polarization state is obtained. Next, the light beam in the single polarization state is collimated by a collimating lens to obtain a collimated light beam. Next, the propagation direction of the light beam in the single polarization state is controlled by an optical element.

[0686] When the above step 5007 is located between step 5003 and step 5004, after the optical element changes the propagation direction of the light beam in a single polarization state, the collimating lens collimates the light beam in a single polarization state to obtain a collimated light beam.

[0687] It should be understood that Fig.52 In the method shown, step 5006 and step 5007 are optional steps, and either step 5006 or 5007 can be selected for execution.

[0688] Combination of the above Figure 26 to Figure 52 A TOF depth sensing module and an image generation method in an embodiment of the present application are described in detail. Figures 53 to 69 Another TOF depth sensing module and image generation method of an embodiment of the present application are introduced in detail.

[0689] Pulse-type TOF technology is often used for scanning in traditional TOF depth sensing modules, but pulse-type TOF technology requires that the sensitivity of the photodetector be high enough to achieve the ability of single-photon detection. Commonly used photodetectors often use single-photon avalanche diodes (SPADs). Due to the complex interface and processing circuit of SPAD, the resolution of commonly used SPAD sensors is low, which is insufficient to meet the requirements of high spatial resolution of depth sensing. To this end, an embodiment of the present application provides a TOF depth sensing module and an image generation method, which improves the spatial resolution of depth sensing by block lighting and time-division multiplexing. This type of TOF depth sensing module and image generation method are described in detail below in conjunction with the accompanying drawings.

[0690] Let's combine Fig.53 A brief introduction is given to the TOF depth sensing module of the embodiment of the present application.

[0691] Fig.53 It is a schematic diagram of using the TOF depth sensing module of an embodiment of the present application to perform distance measurement.

[0692] like Fig.53 As shown, the TOF depth sensing module may include a transmitting end (also known as a projecting end), a receiving end and a control unit, wherein the transmitting end is used to emit an outgoing light beam, and the receiving end is used to receive a reflected light beam of a target object (the reflected light beam is a light beam obtained by the target object reflecting the outgoing light beam), and the control unit may control the transmitting end and the receiving end to respectively emit and receive light beams.

[0693] exist Fig.53 In the embodiment, the transmitting end may generally include a laser light source, a polarization filter device, a collimating lens (optional), a first optical element and a projection lens (optional), and the receiving end may generally include a receiving lens, a second optical element and a sensor. Fig.53 In the process, the TOF corresponding to the outgoing light beam can be recorded by a timing device to calculate the distance from the TOF depth sensing module to the target area, thereby obtaining the final depth map of the target object. The TOF corresponding to the outgoing light beam can refer to the time difference information between the moment when the reflected light beam is received by the receiving unit and the moment when the outgoing light beam is emitted.

[0694] like Fig.53 As shown, the FOV of the laser beam can be adjusted by the beam shaping device and the first optical element, and different scanning beams can be emitted at the time t0-t17. The target FOV can be achieved by splicing the FOV of the beam emitted at the time t0-t17, and the resolution of the TOF depth sensing module can be improved.

[0695] The TOF depth sensing module of the embodiment of the present application can be used for 3D image acquisition. The TOF depth sensing module of the embodiment of the present application can be set in a smart terminal (for example, a mobile phone, a tablet, a wearable device, etc.) for acquiring depth images or 3D images, and can also provide gesture and limb recognition for 3D games or somatosensory games.

[0696] Combine the following Fig.54 The TOF depth sensing module of the embodiment of the present application is introduced in detail.

[0697] Fig.54 It is a schematic block diagram of the TOF depth sensing module of an embodiment of the present application.

[0698] Fig.54 The TOF depth sensing module 300 shown includes: a laser light source 310, a polarization filter device 320, a beam shaping device 330, a first optical element 340, a second optical element 350, a receiving unit 360 and a control unit 370. Fig.54 As shown, the transmitting end of the TOF depth sensing module 300 includes a laser light source 310, a polarization filter device 320, a beam shaping device 330 and a first optical element 340, and the receiving end of the TOF depth sensing module 300 includes a second optical element 350 and a receiving unit 360. The first optical element 340 and the second optical element 350 are respectively located at the transmitting end and the receiving end of the TOF depth sensing module 300, wherein the first optical element mainly controls the direction of the light beam at the transmitting end to obtain an outgoing light beam, and the second optical element mainly controls the direction of the reflected light beam to deflect the reflected light beam to the receiving unit.

[0699] These modules or units in the TOF depth sensing module 300 are introduced in detail below.

[0700] Laser source 310:

[0701] The laser light source 310 is used to generate a laser beam. Specifically, the laser light source 310 can generate light of multiple polarization states.

[0702] Optionally, the laser beam emitted by the laser light source 310 is a single beam of quasi-parallel light, and the divergence angle of the laser beam emitted by the laser light source 310 is less than 1°.

[0703] Optionally, the laser light source 310 is a semiconductor laser light source.

[0704] The laser light source may be a vertical cavity surface emitting laser (VCSEL).

[0705] Optionally, the laser light source 310 is a Fabry-Perot laser (abbreviated as FP laser).

[0706] Compared with a single VCSEL, a single FP laser can achieve greater power, and its electro-optical conversion efficiency is also higher than that of VCSEL, which can improve the scanning effect.

[0707] Optionally, the wavelength of the laser beam emitted by the laser light source 310 is greater than 900 nm.

[0708] Since the intensity of light greater than 900nm in sunlight is relatively weak, when the wavelength of the laser beam is greater than 900nm, it helps to reduce the interference caused by sunlight and improve the scanning effect of the TOF depth sensing module.

[0709] Optionally, the wavelength of the laser beam emitted by the laser light source 310 is 940 nm or 1550 nm.

[0710] Since the intensity of light near 940nm or 1550nm in sunlight is relatively weak, when the wavelength of the laser beam is 940nm or 1550nm, the interference caused by sunlight can be greatly reduced, and the scanning effect of the TOF depth sensing module can be improved.

[0711] The light emitting area of ​​the laser light source 310 is less than or equal to 5×5 mm 2 .

[0712] Since the size of the above-mentioned laser light source is relatively small, the TOF depth sensing module 300 including the laser light source can be easily integrated into the terminal device, which can reduce the space occupied in the terminal device to a certain extent.

[0713] Optionally, the average output optical power of the TOF depth sensing module is less than 800mw.

[0714] When the average output optical power of the TOF depth sensing module is less than or equal to 800mw, the power consumption of the TOF depth sensing module is relatively small, and it is convenient to be installed in devices that are sensitive to power consumption, such as terminal devices.

[0715] Polarization filter device 320:

[0716] The polarization filter device 320 is used to filter the laser beam to obtain a beam with a single polarization state.

[0717] The light beam of a single polarization state filtered by the polarization filter device 320 is one of the multiple polarization states of the light beam generated by the laser light source 310 .

[0718] For example, the laser light source 310 generates a laser beam including linearly polarized light, left-handed circularly polarized light and right-handed circularly polarized light. Then, the polarization filter device 320 can filter out the left-handed circularly polarized light, right-handed circularly polarized light and right-handed circularly polarized light in the laser beam, and only retain the linear polarized light in a specific direction. Optionally, the polarization filter device can also include a quarter wave plate to convert the filtered linear polarized light into left-handed circularly polarized light (or right-handed circularly polarized light).

[0719] Beam shaping device 330:

[0720] The beam shaping device 330 is used to adjust the laser beam to obtain a first beam.

[0721] The FOV of the first light beam ranges from [5°×5°, 20°×20°].

[0722] It should be understood that the horizontal FOV of the FOV of the first light beam may be between 5° and 20° (including 5° and 20°), and the vertical FOV of the FOV of the first light beam may be between 5° and 20° (including 5° and 20°).

[0723] Control unit 370:

[0724] The control unit 370 is used to control the first optical element to control the direction of the first light beam at M different moments respectively, so as to obtain outgoing light beams in M ​​different directions.

[0725] The total FOV covered by the M outgoing light beams in different directions includes [50°×50°, 80°×80°].

[0726] The control unit 370 is further configured to control the second optical element to deflect M reflected light beams obtained by the target object reflecting outgoing light beams in M ​​different directions to the receiving unit.

[0727] In an embodiment of the present application, the FOV of the light beam is adjusted by a beam shaping device so that the first light beam has a larger FOV, and scanning is performed in a time-division multiplexing manner (the first optical element emits light beams in different directions at different times), which can improve the spatial resolution of the depth map of the target object finally obtained.

[0728] Fig.55 It is a schematic block diagram of the TOF depth sensing module of an embodiment of the present application.

[0729] like Fig.55As shown, the TOF depth sensing module further includes: a collimating lens 380, which is located between the laser light source 310 and the polarization filter device 320, and is used to collimate the laser beam; the polarization filter device 320 is used to filter the beam collimated by the collimating lens 380 to obtain a beam with a single polarization state.

[0730] Fig.56 is a schematic block diagram of a TOF depth sensing module according to an embodiment of the present application. Fig.56 In the embodiment, the collimating lens 380 can also be located between the polarization filter device 320 and the beam shaping device 330. The collimating lens 380 is used to collimate the light beam in a single polarization state; the beam shaping device 330 is used to adjust the FOV after the collimation process of the collimating lens 380 to obtain the first light beam.

[0731] The above-mentioned collimation process of the light beam by the collimating lens can obtain a nearly parallel light beam, which can increase the power density of the light beam and further improve the effect of subsequent scanning using the light beam.

[0732] Optionally, the clear aperture of the collimating lens is less than or equal to 5 mm.

[0733] Since the size of the collimating lens is relatively small, the TOF depth sensing module including the collimating lens can be easily integrated into the terminal device, which can reduce the space occupied in the terminal device to a certain extent.

[0734] It should be understood that the collimating lens may also be located between the beam shaping device 330 and the first optical element 340. In this case, the collimating lens collimates the light beam shaped by the beam shaping device 330, and the collimated light beam is then processed by the first optical element.

[0735] In addition, the collimating lens 380 can be located at any possible position in the TOF depth sensing module 300 and collimate the light beam in any possible process.

[0736] Optionally, a horizontal distance between the first optical element and the second optical element is less than or equal to 1 cm.

[0737] Optionally, the first optical element and / or the second optical element is a rotating mirror device.

[0738] The rotating mirror device controls the emission direction of the emitted light beam by rotating.

[0739] The rotating mirror device may specifically be a micro-electromechanical system galvanometer or a multi-faceted rotating mirror.

[0740] The first optical element may be any one of a liquid crystal polarization grating, an electro-optical device, an acousto-optic device, an optical phased array device, etc., and the second optical element may also be any one of a liquid crystal polarization grating, an electro-optical device, an acousto-optic device, an optical phased array device, etc. For details of the liquid crystal polarization grating, the electro-optical device, the acousto-optic device, the optical phased array device, etc., please refer to the descriptions of the first to fourth cases above.

[0741] like Fig.35 As shown, the liquid crystal polarization grating includes not only a transverse LCPG and a longitudinal LCPG, but also a transverse polarization control plate and a longitudinal polarization control plate. Fig.35 In the embodiment, the transverse LCPG is located between the transverse polarization control plate and the longitudinal polarization control plate, and the longitudinal polarization control plate is located between the transverse LCPG and the longitudinal LCPG.

[0742] Optionally, the above Fig.35 The various components in the liquid crystal polarization grating shown in the figure can be combined in the following ways.

[0743] Combination 1: 124;

[0744] Combination 2: 342;

[0745] Combination 3:3412.

[0746] In the above combination mode 1, 1 may represent a transverse polarization control plate and a longitudinal polarization control plate in close contact, and in the above combination mode 2, 3 may represent a transverse polarization control plate and a longitudinal polarization control plate in close contact.

[0747] Among them, when the first optical element 340 or the second optical element 350 of combination method 1 or combination method 2 is placed in the TOF depth sensing module, the transverse polarization control plate or the longitudinal polarization control plate is located on the side close to the laser light source, and the transverse LCPG and the longitudinal LCPG are both located on the side away from the laser light source.

[0748] When the first optical element 340 or the second optical element 350 of combination mode 3 is placed in the TOF depth sensing module, the distances between the longitudinal polarization control plate, the longitudinal LCPG, the transverse polarization control plate, the transverse LCPG and the laser light source increase in sequence.

[0749] It should be understood that the above three combinations of liquid crystal polarization gratings and Fig.35The combination is only an example. In fact, the components in the optical element of the present application can also be combined in different ways. As long as the distance between the transverse polarization control plate and the laser light source is smaller than the distance between the transverse LCPG and the laser light source, and the distance between the transverse polarization control plate and the laser light source is smaller than the distance between the transverse LCPG and the laser light source.

[0750] Optionally, the second optical element comprises: a transverse polarization control plate, a transverse liquid crystal polarization grating, a longitudinal polarization control plate and a longitudinal liquid crystal polarization grating, and the distances between the two elements and the sensor are successively larger.

[0751] Optionally, the beam shaping device is composed of a diffusion lens and a rectangular aperture.

[0752] Combination of the above Figure 53 to Figure 56 The TOF depth sensing module of the embodiment of the present application is introduced. Fig.57 The image generation method of the embodiment of the present application is introduced in detail.

[0753] Fig.57 It is a schematic flowchart of the image generation method of an embodiment of the present application.

[0754] Fig.57 The method shown can be executed by a TOF depth sensing module or a terminal device including the TOF depth sensing module of the embodiment of the present application. Specifically, Fig.57 The method shown can be Fig.54 The TOF depth sensor module shown may include Fig.54 The TOF depth sensing module is used to execute the terminal device shown. Fig.57 The method shown includes steps 5001 to 5006, each of which is described in detail below.

[0755] 5001. Control the laser light source to generate a laser beam;

[0756] 5002. Filter the laser beam using a polarization filter device to obtain a beam with a single polarization state.

[0757] The single polarization state mentioned above is one of the multiple polarization states.

[0758] For example, the multiple polarization states may include linear polarization, left-hand circular polarization and right-hand circular polarization, and the single polarization state may be any one of linear polarization, left-hand circular polarization and right-hand circular polarization.

[0759] 5003. Use a beam shaping device to adjust the laser beam to obtain a first beam.

[0760] Optionally, the above step 5003 specifically includes: adjusting the angular spatial intensity distribution of the light beam in a single polarization state by using a beam shaping device to obtain the first light beam.

[0761] Wherein, the FOV range of the first light beam includes [5°×5°, 20°×20°];

[0762] 5004. Control the first optical element to control the direction of the first light beam from the beam shaping device at M different times respectively, to obtain outgoing light beams in M ​​different directions.

[0763] The total FOV covered by the M outgoing light beams in different directions includes [50°×50°, 80°×80°].

[0764] 5005. Control the second optical element to deflect M reflected light beams obtained by the target object reflecting outgoing light beams in M ​​different directions to a receiving unit.

[0765] 5006. Generate a depth map of the target object according to the TOFs corresponding to the outgoing light beams in M ​​different directions.

[0766] In an embodiment of the present application, the FOV of the light beam is adjusted by a beam shaping device so that the first light beam has a larger FOV, and scanning is performed in a time-division multiplexing manner (the first optical element emits light beams in different directions at different times), which can improve the spatial resolution of the depth map of the target object finally obtained.

[0767] Optionally, the above step 5006 specifically includes: generating depth maps of the M regions of the target object according to the distances between the M regions of the target object and the TOF depth sensing module; and synthesizing the depth map of the target object according to the depth maps of the M regions of the target object.

[0768] Optionally, the above step 5004 specifically includes: the control unit generates a first voltage signal, the first voltage signal is used to control the first optical element to control the direction of the first light beam at M different times respectively, so as to obtain M outgoing light beams in different directions; the above step 5005 includes: the control unit generates a second voltage signal, the second voltage signal is used to control the second optical element to deflect M reflected light beams obtained by the target object reflecting the outgoing light beams in M ​​different directions to the receiving unit.

[0769] The first voltage signal and the second voltage signal have the same voltage value at the same time.

[0770] In the above Fig.54In the TOF depth sensing module 300 shown, the transmitting end and the receiving end respectively use different optical elements to control the emission and reception of the light beam. Optionally, in the TOF depth sensing module of an embodiment of the present application, the transmitting end and the receiving end may also use the same optical elements to control the emission and reception of the light beam.

[0771] Combine the following Fig.58 The situation in which the transmitting end and the receiving end use the same optical element to realize the reflection and reception of the light beam is described in detail.

[0772] Fig.58 It is a schematic block diagram of the TOF depth sensing module of an embodiment of the present application.

[0773] Fig.58 The TOF depth sensing module 400 shown includes: a laser light source 410, a polarization filter device 420, a beam shaping device 430, an optical element 440, a receiving unit 450 and a control unit 460. Fig.58 As shown, the transmitting end of the TOF depth sensing module 400 includes a laser light source 410, a polarization filter device 420, a beam shaping device 430, and an optical element 440. The receiving end of the TOF depth sensing module 400 includes an optical element 440 and a receiving unit 450. The transmitting end and the receiving end of the TOF depth sensing module 400 share the optical element 440. The optical element 440 can control the light beam of the transmitting end to obtain an outgoing light beam, and can also control the reflected light beam so that the reflected light beam is deflected to the receiving unit 450.

[0774] These modules or units in the TOF depth sensing module 400 are introduced in detail below.

[0775] Laser source 410:

[0776] The laser light source 410 is used to generate a laser beam;

[0777] Optionally, the laser beam emitted by the laser light source 410 is a single beam of quasi-parallel light, and the divergence angle of the laser beam emitted by the laser light source 410 is less than 1°.

[0778] Optionally, the laser light source 410 is a semiconductor laser light source.

[0779] The laser light source 410 may be a vertical cavity surface emitting laser (VCSEL).

[0780] Optionally, the laser light source 410 may also be a Fabry-Perot laser (abbreviated as FP laser).

[0781] Compared with a single VCSEL, a single FP laser can achieve greater power, and its electro-optical conversion efficiency is also higher than that of VCSEL, which can improve the scanning effect.

[0782] Optionally, the wavelength of the laser beam emitted by the laser light source 410 is greater than 900 nm.

[0783] Since the intensity of light greater than 900nm in sunlight is relatively weak, when the wavelength of the laser beam is greater than 900nm, it helps to reduce the interference caused by sunlight and improve the scanning effect of the TOF depth sensing module.

[0784] Optionally, the wavelength of the laser beam emitted by the laser light source 410 is 940 nm or 1550 nm.

[0785] Since the intensity of light near 940nm or 1550nm in sunlight is relatively weak, when the wavelength of the laser beam is 940nm or 1550nm, the interference caused by sunlight can be greatly reduced, and the scanning effect of the TOF depth sensing module can be improved.

[0786] The light emitting area of ​​the laser light source 410 is less than or equal to 5×5 mm 2 .

[0787] Since the size of the above-mentioned laser light source is relatively small, the TOF depth sensing module 400 including the laser light source can be easily integrated into the terminal device, which can reduce the space occupied in the terminal device to a certain extent.

[0788] Optionally, the average output optical power of the TOF depth sensing module 400 is less than 800mw.

[0789] When the average output optical power of the TOF depth sensing module is less than or equal to 800mw, the power consumption of the TOF depth sensing module is relatively small, and it is convenient to be installed in devices that are sensitive to power consumption, such as terminal devices.

[0790] The polarization filter device 420 is used to filter the laser beam to obtain a beam with a single polarization state;

[0791] The beam shaping device 430 is used to adjust the FOV of the light beam in a single polarization state to obtain a first light beam;

[0792] The control unit 460 is used to control the optical element 440 to control the direction of the first light beam at M different moments respectively, so as to obtain outgoing light beams in M ​​different directions;

[0793] The control unit 460 is further used to control the optical element 440 to deflect M reflected light beams obtained by the target object reflecting the outgoing light beams in M ​​different directions to the receiving unit 450 .

[0794] Wherein, the single polarization state is one of multiple polarization states;

[0795] For example, the multiple polarization states may include linear polarization, left-hand circular polarization and right-hand circular polarization, and the single polarization state may be any one of linear polarization, left-hand circular polarization and right-hand circular polarization.

[0796] The FOV range of the above-mentioned first light beam includes [5°×5°, 20°×20°]; the total FOV range covered by the above-mentioned M outgoing light beams in different directions includes [50°×50°, 80°×80°].

[0797] In an embodiment of the present application, the FOV of the light beam is adjusted by a beam shaping device so that the first light beam has a larger FOV. At the same time, scanning is performed in a time-division multiplexing manner (the optical element emits light beams in different directions at different times), which can improve the spatial resolution of the depth map of the target object finally obtained.

[0798] Optionally, the control unit 460 is further configured to generate a depth map of the target object according to TOFs corresponding to the outgoing light beams in M ​​different directions.

[0799] The TOF corresponding to the above-mentioned M outgoing light beams in different directions may specifically refer to the time difference information between the moment when the reflected light beam corresponding to the above-mentioned M outgoing light beams in different directions is received by the receiving unit and the moment when the above-mentioned M outgoing light beams in different directions are emitted.

[0800] Assuming that the M outgoing light beams in different directions include the outgoing light beam 1, the reflected light beam corresponding to the outgoing light beam 1 may refer to a light beam generated after the outgoing light beam 1 reaches the target object and is reflected after passing through the target object.

[0801] Optionally, the above definitions of the laser light source 310 , polarization filter device 320 , and beam shaping device 330 in the TOF depth sensing module 300 are also applicable to the laser light source 410 , polarization filter device 420 , and beam shaping device 430 in the TOF depth sensing module 400 .

[0802] Optionally, the optical element is a rotating mirror device.

[0803] The rotating mirror device controls the emission direction of the emitted light beam by rotating.

[0804] Optionally, the rotating mirror device is a micro-electromechanical system galvanometer or a multi-faceted rotating mirror.

[0805] The case where the optical element is a rotating mirror device is described below with reference to the accompanying drawings.

[0806] Fig.59 It is a schematic block diagram of the TOF depth sensing module of an embodiment of the present application.

[0807] like Fig.59 As shown, the TOF depth sensing module further includes: a collimating lens 470, which is located between the laser light source 410 and the polarization filter device 420, and is used to collimate the laser beam; the polarization filter device 420 is used to filter the beam collimated by the collimating lens 470 to obtain a beam with a single polarization state.

[0808] Fig.60 is a schematic block diagram of a TOF depth sensing module according to an embodiment of the present application. Fig.60 In the embodiment, the collimating lens 470 can also be located between the polarization filter device 420 and the beam shaping device 430. The collimating lens 470 is used to collimate the light beam in a single polarization state; the beam shaping device 430 is used to adjust the FOV after the collimation process of the collimating lens 470 to obtain the first light beam.

[0809] The above-mentioned collimation process of the light beam by the collimating lens can obtain a nearly parallel light beam, which can increase the power density of the light beam and further improve the effect of subsequent scanning using the light beam.

[0810] Optionally, the clear aperture of the collimating lens is less than or equal to 5 mm.

[0811] Since the size of the collimating lens is relatively small, the TOF depth sensing module including the collimating lens can be easily integrated into the terminal device, which can reduce the space occupied in the terminal device to a certain extent.

[0812] It should be understood that the collimating lens may also be located between the beam shaping device 430 and the optical element 440 . In this case, the collimating lens collimates the light beam shaped by the beam shaping device 430 , and the collimated light beam is then processed by the optical element 440 .

[0813] In addition, the collimating lens 470 can be located at any possible position in the TOF depth sensing module 400 and can collimate the light beam in any possible process.

[0814] like Fig.61As shown, the TOF depth sensing module includes a laser light source, a light homogenizing device, a beam splitter, a microelectromechanical systems (MEMS) galvanometer, a receiving lens and a sensor. The MEMS in the figure include an electrostatic galvanometer, an electromagnetic galvanometer and a multi-faceted rotating mirror. Since the rotating mirror devices all work in a reflective manner, the optical path in the TOF depth sensing module is a reflective optical path, and the emission and reception are coaxial optical paths. The polarizing device and lens can be shared through the beam splitter. Fig.61 In the embodiment, the polarizing device is specifically a MEMS galvanometer.

[0815] Optionally, the optical element 440 is a liquid crystal polarization element.

[0816] Optionally, the optical element 440 includes: a transverse polarization control plate, a transverse liquid crystal polarization grating, a longitudinal polarization control plate and a longitudinal liquid crystal polarization grating.

[0817] Optionally, in the above-mentioned optical element 440, the distances between the transverse polarization control plate, the transverse liquid crystal polarization grating, the longitudinal polarization control plate and the longitudinal liquid crystal polarization grating and the laser light source increase successively, or the distances between the longitudinal polarization control plate, the longitudinal liquid crystal polarization grating, the transverse polarization control plate and the transverse liquid crystal polarization grating and the laser light source increase successively.

[0818] Optionally, the beam shaping device 430 is composed of a diffusion lens and a rectangular aperture.

[0819] The optical element may be any one of a liquid crystal polarization grating, an electro-optical device, an acousto-optical device, an optical phased array device, etc. For details of the liquid crystal polarization grating, the electro-optical device, the acousto-optical device, the optical phased array device, etc., please refer to the descriptions of the first to fourth cases above.

[0820] Fig.62 It is a schematic flowchart of the image generation method of an embodiment of the present application.

[0821] Fig.62 The method shown can be executed by a TOF depth sensing module or a terminal device including the TOF depth sensing module of the embodiment of the present application. Specifically, Fig.62 The method shown can be Fig.58 The TOF depth sensor module shown may include Fig.58 The TOF depth sensing module is used to execute the terminal device shown. Fig.62 The method shown includes steps 6001 to 6006, each of which is described in detail below.

[0822] 6001. Control the laser light source to generate a laser beam.

[0823] 6002. Use a polarization filter device to filter the laser beam to obtain a beam with a single polarization state.

[0824] Wherein, the single polarization state is one of multiple polarization states;

[0825] For example, the multiple polarization states may include linear polarization, left-hand circular polarization and right-hand circular polarization, and the single polarization state may be any one of linear polarization, left-hand circular polarization and right-hand circular polarization.

[0826] 6003. Use a beam shaping device to adjust a light beam in a single polarization state to obtain a first light beam.

[0827] The FOV range of the first light beam includes [5°×5°, 20°×20°].

[0828] 6004. Control the optical element to control the direction of the first light beam from the beam shaping device at M different times respectively, to obtain outgoing light beams in M ​​different directions.

[0829] The total FOV covered by the M outgoing light beams in different directions includes [50°×50°, 80°×80°].

[0830] 6005. Control the optical element to deflect M reflected light beams obtained by the target object reflecting the outgoing light beams in M ​​different directions to a receiving unit.

[0831] 6006. Generate a depth map of the target object according to the TOFs corresponding to the outgoing light beams in M ​​different directions.

[0832] In an embodiment of the present application, the FOV of the light beam is adjusted by a beam shaping device so that the first light beam has a larger FOV. At the same time, scanning is performed in a time-division multiplexing manner (the optical element emits light beams in different directions at different times), which can improve the spatial resolution of the depth map of the target object finally obtained.

[0833] Optionally, the above step 6006 specifically includes: determining the distances between M regions of the target object and the TOF depth sensing module according to the TOFs corresponding to the outgoing light beams in M ​​different directions; generating depth maps of the M regions of the target object according to the distances between the M regions of the target object and the TOF depth sensing module; and synthesizing the depth map of the target object according to the depth maps of the M regions of the target object.

[0834] Optionally, the above step 6003 specifically includes: adjusting the angular spatial intensity distribution of the light beam in a single polarization state by using a beam shaping device to obtain the first light beam.

[0835] Combine the following Fig.63The specific working process of the TOF depth sensing module 400 of the embodiment of the present application is introduced in detail.

[0836] Fig.63 It is a schematic diagram of the structure of the TOF depth sensing module of an embodiment of the present application.

[0837] Fig.63 The specific implementation and functions of each component of the TOF depth sensing module shown are as follows:

[0838] (1) The laser light source is a VCSEL array.

[0839] VCSEL light sources are arrays that can emit beams with good directionality.

[0840] (2) The polarizer is a polarization filter device, which can be located in front of (below) or behind (above) the light homogenizing device.

[0841] (3) The light homogenizing device may be a diffractive optical element (DOE) or an optical diffuser (also called a diffuser).

[0842] The light beam array is processed by the light homogenizing device and is arranged into a substantially uniform light beam block.

[0843] (3) The optical element is a multi-layer LCPG (liquid crystal polarization grating).

[0844] It should be understood that in the above Fig.63 In the figure, only the situation that the polarizer is located below the light homogenizing device is shown. In fact, the polarizer can also be located above the light homogenizing device.

[0845] For the specific principle of how the liquid crystal polarization grating controls the direction of the light beam, please refer to Fig.37 and Fig.38 Description of relevant content.

[0846] exist Fig.63 In the process, the emitted light is reflected by the target through the cooperation of the multi-layer liquid crystal polarization grating and the 1 / 4 wave plate. When it returns to the polarizer, it has just passed through an extra 1 / 2 optical path. This design makes the polarizer deflect the return light in the opposite direction to the emitted light. Under the quasi-coaxial approximation, the obliquely emitted light returns along the original path after reflection, deflected back in the direction parallel to the emitted light, and reaches the receiving lens. The receiving end uses the beam deflection device to divide the target blocks that are selectively illuminated by the emitted light and image them onto the entire receiver (SPAD array). When the target is illuminated by blocks, each block is received by the entire receiver, and the images at each moment are spliced ​​to obtain a complete image. In this way, the time division multiplexing of the receiver is realized, and the purpose of doubling the resolution is achieved.

[0847] (4) The receiving lens is completed by an ordinary lens, which images the received light on the receiver.

[0848] (5) The receiver is a SPAD array.

[0849] SPAD can detect single photons, and the time of the detected single photon pulse can be accurately recorded. Every time the VCSEL emits light, the SPAD starts. The VCSEL periodically emits light beams, and the SPAD array can count the time when each pixel receives the reflected light in each cycle. By counting the time distribution of the reflected signal, the reflected signal pulse can be fitted, thereby calculating the delay time.

[0850] The key component of this embodiment is the beam deflection device shared by the projection end and the receiving end, that is, the liquid crystal polarization device. In this embodiment, the beam deflection device includes a multilayer LCPG, also known as an electrically controlled liquid crystal polarization device.

[0851] Fig.64 It is a schematic diagram of the structure of a liquid crystal polarizing device according to an embodiment of the present application.

[0852] An optional specific structure of the liquid crystal polarizing device is as follows Fig.64 As shown, in Fig.64 In the figure, 1 represents the horizontal single-angle LCPG, 2 represents the horizontal double-angle LCPG, 3 represents the vertical single-angle LCPG, 4 represents the vertical double-angle LCPG, and 5 represents the polarization control plate. There are 5 polarization control plates, located at Fig.64 The left sides of the four LCPGs shown are numbered 5.1, 5.2, 5.3 and 5.4.

[0853] The control unit can be used to Fig.64 The liquid crystal polarization device shown in the figure is controlled, and the control timing can be as follows Fig.65 As shown in (scanning starts from time t0 and continues until time t15). The timing diagram of the drive signal generated by the control unit is as follows Fig.66 shown.

[0854] Fig.66 The voltage driving signals of the polarization control plates 5.1, 5.2, 5.3 and 5.4 of the device from time t0 to time t15 are shown, and the voltage driving signals include two types of signals, low level and high level, wherein the low level is represented by 0 and the high level is represented by 1. Then, the voltage driving signals of the polarization control plates 5.1, 5.2, 5.3 and 5.4 from time t0 to time t15 are specifically shown in Table 1.

[0855] Table 1

[0856]

[0857]

[0858] For example, in Table 1, in the time interval t0, the voltage driving signal of the polarization control plate 5.1 is a low level signal, and the voltage driving signals of the polarization control plates 5.2 to 5.4 are high level signals. Therefore, the voltage signal corresponding to time t0 is 0111.

[0859] like Fig.64 As shown in FIG. 1 , the electrically controlled liquid crystal polarizer is composed of LCPG and a polarization control plate. The voltage driving signal for realizing 4*4 scanning is as follows: Fig.66 As shown, 5.1, 5.2, 5.3, and 5.4 represent the voltage driving signals applied to the four polarization control plates, respectively. The entire FOV is divided into 4*4 blocks, and t0 to t15 are the time intervals for illuminating each block. Fig.66 When the voltage driving signal is shown, the state of the light beam passing through each device when passing through the liquid crystal deflection device is as shown in Table 2.

[0860] Table 2

[0861]

[0862] The following is an explanation of the meanings represented in Table 2. In each item in Table 2, the value in the bracket is a voltage signal, L represents left-hand rotation, R represents right-hand rotation, and values ​​such as 1 and 3 represent the angle of light beam deflection, where the deflection angle represented by 3 is greater than the deflection angle represented by 1.

[0863] For example, for R1-1, R means right-handed, the first 1 means the left side (if the first is -1, it means the right side), and the second -1 means the top side (if the second is 1, it means the bottom side).

[0864] For another example, for L 3-3, L means left rotation, the first 3 means the rightmost side (if the first digit is -3, it means the leftmost side), and the second -3 means the top side (if the second digit is 3, it means the bottom side).

[0865] When the liquid crystal polarizer is applied Fig.66 When the voltage driving signal is shown, the scanning area of ​​the TOF depth sensor module at different times is as follows Fig.67 shown.

[0866] The depth map obtained in the embodiment of the present application is described below in conjunction with the accompanying drawings. Fig.68 As shown in FIG. 1 , assuming that the depth map corresponding to the target object from time t0 to time t3 can be obtained by time-sharing scanning, wherein the resolution of the depth map corresponding to time t0 to time t3 is 160×120, by splicing the depth maps corresponding to time t0 to time t3, the following can be obtained: Fig.69The final depth map of the target object shown in FIG. 1 has a resolution of 320×240. Fig.68 and Fig.69 It can be seen that by splicing the depth maps obtained at different times, the resolution of the final depth map can be improved.

[0867] Combination of the above Figures 53 to 69 A TOF depth sensing module and an image generation method in an embodiment of the present application are described in detail. Figures 70 to 78 Another TOF depth sensing module and image generation method of an embodiment of the present application are introduced in detail.

[0868] In the TOF depth sensing module, liquid crystal devices can be used to adjust the direction of the light beam, and a polarizer is generally added to the transmitting end of the TOF depth sensing module to achieve the emission of polarized light. However, in the process of polarized light emission, due to the polarization selection of the polarizer, half of the energy will be lost when the light beam is emitted. This lost energy will be absorbed or scattered by the polarizer and converted into heat, causing the temperature of the TOF depth sensing module to rise, affecting the stability of the TOF depth sensing module. Therefore, how to reduce the heat loss of the TOF depth sensing module is a problem that needs to be solved.

[0869] Specifically, in the TOF depth sensing module of the embodiment of the present application, the heat loss of the TOF depth sensing module can be reduced by transferring the polarizer from the transmitting end to the receiving end. The TOF depth sensing module of the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0870] Let's combine Fig.70 A brief introduction is given to the TOF depth sensing module of the embodiment of the present application.

[0871] Fig.70 This is a schematic diagram of the TOF depth sensing module of the embodiment of the present application when it is working. Fig.70 As shown, the TOF depth sensing module may include a transmitting end (also known as a projecting end), a receiving end and a control unit, wherein the transmitting end is used to emit an outgoing light beam, and the receiving end is used to receive a reflected light beam of a target object (the reflected light beam is a light beam obtained by the target object reflecting the outgoing light beam), and the control unit may control the transmitting end and the receiving end to respectively emit and receive light beams.

[0872] exist Fig.70 In the invention, the transmitting end may generally include a laser light source, a collimating lens (optional), a light homogenizing device, an optical element, and a projection lens (optional); the receiving end generally includes: a beam selecting device and a receiving unit, wherein the receiving unit may specifically include a receiving lens and a sensor.

[0873] Fig.70 The TOF depth sensing module shown will project two or more projection lights in different states (state A, state B) at the same time. After the projection lights in two different states are reflected and reach the receiving end, the beam selection device selects the reflected light in a certain state to enter the sensor according to the instruction, and performs depth imaging on the light in a specific state. Then the beam deflection device can scan in different directions to achieve coverage of the target FOV.

[0874] Fig.70 The TOF depth sensing module shown can be used for 3D image acquisition. The TOF depth sensing module of the embodiment of the present application can be set in a smart terminal (for example, a mobile phone, a tablet, a wearable device, etc.) for acquiring depth images or 3D images, and can also provide gesture and limb recognition for 3D games or somatosensory games.

[0875] Combine the following Fig.71 The TOF depth sensing module of the embodiment of the present application is introduced in detail.

[0876] Fig.71 The TOF depth sensing module 500 shown includes: a laser light source 510 , an optical element 520 , a beam selection device 530 , a receiving unit 540 and a control unit 550 .

[0877] These modules or units in the TOF depth sensing module 500 are introduced in detail below.

[0878] Laser source 510:

[0879] The laser light source 510 is used to generate a laser beam.

[0880] Optionally, the above-mentioned laser light source may be a semiconductor laser light source.

[0881] The laser light source may be a vertical cavity surface emitting laser (VCSEL).

[0882] Optionally, the laser light source may be a Fabry-Perot laser (abbreviated as FP laser).

[0883] Compared with a single VCSEL, a single FP laser can achieve greater power, and its electro-optical conversion efficiency is also higher than that of VCSEL, which can improve the scanning effect.

[0884] Optionally, the wavelength of the laser beam emitted by the laser light source 510 is greater than 900 nm.

[0885] Since the intensity of light greater than 900nm in sunlight is relatively weak, when the wavelength of the laser beam is greater than 900nm, it helps to reduce the interference caused by sunlight and improve the scanning effect of the TOF depth sensing module.

[0886] Optionally, the wavelength of the laser beam emitted by the laser light source 510 is 940 nm or 1550 nm.

[0887] Since the intensity of light near 940nm or 1550nm in sunlight is relatively weak, when the wavelength of the laser beam is 940nm or 1550nm, the interference caused by sunlight can be greatly reduced, and the scanning effect of the TOF depth sensing module can be improved.

[0888] Optionally, the light emitting area of ​​the laser light source 510 is less than or equal to 5×5 mm 2 .

[0889] Since the size of the above-mentioned laser light source is relatively small, the TOF depth sensing module including the laser light source is relatively easy to be integrated into the terminal device, which can reduce the space occupied in the terminal device to a certain extent.

[0890] Optionally, the average output optical power of the TOF depth sensing module is less than 800mw.

[0891] When the average output optical power of the TOF depth sensing module is less than or equal to 800mw, the power consumption of the TOF depth sensing module is relatively small, and it is convenient to be installed in devices that are sensitive to power consumption, such as terminal devices.

[0892] Optics 520:

[0893] The optical element 520 is arranged in the exit direction of the laser beam, and the optical element 520 is used to control the direction of the laser beam to obtain a first exit beam and a second exit beam, wherein the exit direction of the first exit beam is different from the exit direction of the second exit beam, and the polarization direction of the first exit beam is orthogonal to the polarization direction of the second exit beam.

[0894] Alternatively, if Fig.35 The above-mentioned optical element 520 may include: a transverse polarization control plate, a transverse liquid crystal polarization grating, a longitudinal polarization control plate and a longitudinal liquid crystal polarization grating, wherein the distances between the transverse polarization control plate, the transverse liquid crystal polarization grating, the longitudinal polarization control plate and the longitudinal liquid crystal polarization grating and the laser light source increase in sequence.

[0895] Alternatively, in the above optical element 520, the distances between the longitudinal polarization control plate, the longitudinal liquid crystal polarization grating, the transverse polarization control plate, and the transverse liquid crystal polarization grating and the laser light source become larger in sequence.

[0896] Receiving unit 540:

[0897] The receiving unit 540 may include a receiving lens 541 and a sensor 542 .

[0898] Control unit 550 and beam selection device 530:

[0899] The control unit 550 is used to control the operation of the beam selection device 530 through a control signal. Specifically, the control unit 550 can generate a control signal, which is used to control the beam selection device 530 to transmit the third reflected light beam and the fourth reflected light beam to the sensor at different time intervals, respectively, wherein the third reflected light beam is a light beam obtained by the target object reflecting the first outgoing light beam, and the fourth reflected light beam is a light beam obtained by the target object reflecting the second outgoing light beam.

[0900] The above-mentioned beam selection device 530 can transmit light beams of different polarization states to the receiving unit at different times under the control of the control unit 550. The beam selection device 530 here adopts a time-sharing mode to transmit the received reflected light beam to the receiving unit 540. Compared with the beam splitter 630 in the TOF depth sensing module 600 described below, it can make more full use of the receiving resolution of the receiving unit 540, and the resolution of the depth map finally obtained is relatively high.

[0901] Optionally, the control signal generated by the control unit 550 is used to control the beam selection device 530 to respectively transmit the third reflected light beam and the fourth reflected light beam to the sensor at different time intervals.

[0902] That is, under the control of the control signal generated by the control unit 550, the above-mentioned beam selection device can respectively transmit the third reflected light beam and the fourth reflected light beam to the receiving unit at different times.

[0903] Optionally, as an embodiment, the above-mentioned beam selection device 530 is composed of a 1 / 4 wave plate + a half-wave plate + a polarizer.

[0904] like Fig.72 As shown, the TOF depth sensing module 500 may further include:

[0905] The collimating lens 560 is arranged in the emission direction of the laser beam, and the collimating lens 560 is arranged between the laser light source and the optical element. The collimating lens 560 is used to collimate the laser beam to obtain a collimated beam; the optical element 520 is used to control the direction of the collimated beam to obtain a first emission beam and a second emission beam.

[0906] The above-mentioned collimation process of the light beam through the collimating lens can obtain a nearly parallel light beam, which can increase the power density of the light beam and further improve the effect of subsequent scanning using the light beam.

[0907] Optionally, the clear aperture of the collimating lens is less than or equal to 5 mm.

[0908] Since the size of the collimating lens is relatively small, the TOF depth sensing module including the collimating lens can be easily integrated into the terminal device, which can reduce the space occupied in the terminal device to a certain extent.

[0909] like Fig.73 As shown, the TOF depth sensing module 500 may further include:

[0910] The light homogenizer 570 is arranged in the emission direction of the laser beam, and the collimating lens is arranged between the laser light source 510 and the optical element 520. The light homogenizer 570 is used to adjust the energy distribution of the laser beam to obtain a homogenized light beam; the optical element is used to control the direction of the homogenized light beam to obtain a first output light beam and a second output light beam.

[0911] Optionally, the light homogenizing device is a microlens diffuser or a diffraction optical diffuser (DOE Diffuser).

[0912] It should be understood that the above-mentioned TOF depth sensing module 500 can also include a collimating lens 560 and a light homogenizing device 570, and both the collimating lens 560 and the light homogenizing device 570 are located between the laser light source 510 and the optical element 520. For the collimating lens 560 and the light homogenizing device 570, either the distance between the collimating lens 560 and the laser light source is closer, or the distance between the light homogenizing device 570 and the laser light source is closer.

[0913] like Fig.74 As shown, the distance between the collimating lens 560 and the laser light source 510 is smaller than the distance between the light homogenizing device 570 and the laser light source 510 .

[0914] exist Fig.74 In the TOF depth sensing module 500 shown, the laser light beam emitted by the laser light source 510 is first collimated by the collimating lens 560 , and then evenly treated by the light evening device 570 before being transmitted to the optical element 520 for processing.

[0915] In the embodiment of the present application, the light power of the laser beam can be made more uniform in the angle space, or distributed according to a specific rule, through the uniform light processing, to prevent the local light power from being too small, thereby avoiding blind spots in the final depth map of the target object.

[0916] like Fig.75As shown, the distance between the collimating lens 560 and the laser light source 510 is greater than the distance between the light homogenizing device 570 and the laser light source 510 .

[0917] And in Fig.74 In the TOF depth sensing module 500 shown, the laser beam emitted by the laser light source 510 is first homogenized by the light homogenizer 570 , and then collimated by the collimating lens 560 before being transmitted to the optical element 520 for processing.

[0918] Combine the following Fig.76 The specific structure of the TOF depth sensing module 500 is described in detail.

[0919] Fig.76 It is a schematic diagram of the specific structure of the TOF depth sensing module 500 of an embodiment of the present application.

[0920] like Fig.76 As shown, the TOF depth sensing module 500 includes a projection end, a control unit and a receiving end. The projection end includes a laser light source, a light homogenizing device, and a beam deflection device; the receiving end includes a beam deflection device, a beam (dynamic) selection device, a receiving lens, and a two-dimensional sensor; the control unit is used to control the projection end and the receiving end to complete the scanning of the beam. In addition, Fig.76 The beam deflection device in Fig.71 The optical elements in Fig.76 The beam (dynamic) selection device in Fig.71 The beam selection device in.

[0921] The devices specifically used in each module or unit are described in detail below.

[0922] The laser light source may be a vertical cavity surface emitting laser (VCSEL) array light source;

[0923] The light homogenizing device may be a diffractive optical diffuser;

[0924] The beam deflection device may be a multilayer LCPG and a quarter wave plate;

[0925] The electrically controlled LCPG includes an electrically controlled horizontal LCPG component and an electrically controlled vertical LCPG component.

[0926] Among them, the use of multi-layer cascaded electrically controlled LCPG can realize two-dimensional block scanning in the horizontal and vertical directions. The 1 / 4 wave plate is used to convert the circularly polarized light from the LCPG into linearly polarized light, achieving the quasi-coaxial effect of the transmitting end and the receiving end.

[0927] The wavelength of the VCSEL array light source may be greater than 900 nm. Specifically, the wavelength of the VCSEL array light source may be 940 nm or 1550 nm.

[0928] Among them, the solar spectrum intensity in the 940nm band is relatively weak, which is conducive to reducing the noise caused by sunlight in outdoor scenes. In addition, the laser emitted by the above-mentioned VCSEL array light source can be continuous light or pulsed light. The VCSEL array light source can also be divided into several blocks to achieve time-sharing control, allowing different areas to light up at different times.

[0929] The function of the diffractive optical diffuser is to shape the light beam emitted by the VCSEL array light source into a uniform square or rectangular light source with a certain FOV (for example, a FOV of 5° x 5°).

[0930] The function of multilayer LCPG and 1 / 4 wave plate is to realize the scanning of light beam.

[0931] The receiving end and the transmitting end share multi-layer LCPG and 1 / 4 wave plate. The beam selection device at the receiving end is composed of 1 / 4 wave plate + electrically controlled half-wave plate + polarizer. The receiving lens at the receiving end can be a single lens or a combination of multiple lenses. The sensor at the receiving end is a single photon avalanche diode (SPAD) array. Because SPAD has the sensitivity of single photon detection, it can improve the detection distance of the Lidar system.

[0932] For the TOF depth sensing module 500, the polarization selection device at the transmitting end is moved to the receiving end. Fig.76 As shown, the laser emitted by the ordinary VCSEL array light source has no fixed polarization state and can be decomposed into a linear polarized laser parallel to the paper surface and a linear polarized laser perpendicular to the paper surface. After the linear polarized laser passes through the LCPG, it will be divided into two laser beams with different polarization states (left-handed circular polarization and right-handed circular polarization), which are distributed with different emission angles. After the two laser beams pass through the 1 / 4 wave plate, the corresponding polarization states are converted into linear polarization parallel to the paper surface and linear polarization perpendicular to the paper surface. The two laser beams with different polarization states are irradiated to the object in the target area. The reflected light beams generated by the two laser beams with different polarization states will be received by the 1 / 4 wave plate and LCPG shared with the transmitting end and become laser beams with the same divergence angle but different deflection states - left-handed circular polarization and right-handed circular polarization. The beam selection device at the receiving end consists of a 1 / 4 wave plate + an electrically controlled half-wave plate + a polarizer. After the received light passes through this 1 / 4 wave plate, the polarization state is converted into linear polarization parallel to the paper surface and linear polarization perpendicular to the paper surface. In this way, the electrically controlled half-wave plate is controlled in a time-sharing manner to rotate the polarization state of the linear polarized light by 90 degrees or not change the polarization state after passing through the half-wave plate, thereby realizing the time-sharing transmission of the linear polarization parallel to the paper surface and the linear polarization perpendicular to the paper surface. At the same time, the light in the other polarization state is absorbed or scattered by the polarizer.

[0933] Compared with the existing TOF depth sensing module in which the polarization selection device is located at the transmitting end, since the polarization selection device of the present application is located at the receiving end, the energy absorbed or scattered by the polarizer is significantly reduced. Assuming that the detection distance is R meters, the reflectivity of the target object is ρ, and the entrance pupil diameter of the receiving system is D, under the same receiving FOV, the incident energy P of the polarization selection device of the TOF depth sensing module 500 of the embodiment of the present application is t for:

[0934]

[0935] Among them, P is the energy emitted by the transmitter. At a distance of 1m, the energy can be reduced by about 10 4 times.

[0936] In addition, assuming that the TOF depth sensing module 500 of the embodiment of the present application and the traditional TOF depth sensing module patent use a non-polarized light source of the same power, since the outdoor light in the TOF depth sensing module 500 of the embodiment of the present application is non-polarized, half of the light entering the receiving detector will be absorbed or scattered, while the outdoor light in the TOF depth sensing module in the traditional solution will all enter the detector. Therefore, the signal-to-noise ratio of the embodiment of the present application will be increased by about twice under the same circumstances.

[0937] exist Fig.76 Based on the TOF depth sensing module 500 shown, the diffraction optical diffuser (DOE Diffuser) behind the VCSEL array light source can also be replaced with a microlens diffuser (Diffuser). Since the microlens diffuser is based on geometric optics to achieve uniform light, its transmission efficiency is relatively high and can reach more than 80%, while the transmission efficiency of the traditional diffraction optical diffuser (DOEDiffuser) is only about 70%. The morphology of the microlens diffuser is as follows Fig.77 As shown, it is composed of a series of randomly distributed micro-lenses. The position and morphology of each micro-lens are designed through simulation optimization, so that the shaped light beam is as uniform as possible and the transmission efficiency is high.

[0938] Fig.78 It is a schematic flowchart of the image generation method of an embodiment of the present application.

[0939] Fig.78 The method shown can be executed by a TOF depth sensing module or a terminal device including the TOF depth sensing module of the embodiment of the present application. Specifically, Fig.78 The method shown can be Fig.71 The TOF depth sensor module shown may include Fig.71 The TOF depth sensing module is used to execute the terminal device shown. Fig.78The method shown includes steps 7001 to 7006, each of which is described in detail below.

[0940] 7001. Control the laser light source to generate a laser beam;

[0941] 7002. Control the optical element to control the direction of the laser beam to obtain a first outgoing beam and a second outgoing beam.

[0942] 7003. Control the beam selection device to propagate the third reflected beam obtained by the target object reflecting the first outgoing beam and the fourth reflected beam obtained by the target object reflecting the second outgoing beam to different areas of the receiving unit.

[0943] 7004. Generate a first depth map of the target object according to the TOF corresponding to the first outgoing light beam;

[0944] 7005. Generate a second depth map of the target object according to the TOF corresponding to the second outgoing light beam.

[0945] The emitting direction of the first emergent light beam is different from the emitting direction of the second emergent light beam, and the polarization direction of the first emergent light beam is orthogonal to the polarization direction of the second emergent light beam.

[0946] In the embodiment of the present application, since there is no polarization filter device at the transmitting end, the light beam emitted by the laser light source can reach the optical element with almost no loss (polarization filter devices generally absorb more light energy and thus produce certain heat loss), which can reduce the heat loss of the terminal equipment.

[0947] Optionally, Fig.78 The method shown also includes: splicing the first depth map and the second depth map to obtain a depth map of the target object.

[0948] It should be understood that Fig.78 In the method shown, a third depth map, a fourth depth map, and so on may be generated in a similar manner. Next, all the depth maps may be spliced ​​or combined to obtain a final depth map of the target object.

[0949] Optionally, the terminal device further comprises a collimating lens, which is arranged between the laser light source and the optical element. Fig.78 The method shown also includes:

[0950] 7006. Using a collimating lens to collimate the laser beam to obtain a collimated beam;

[0951] The above step 7002 specifically includes: controlling the optical element to control the direction of the collimated light beam to obtain a first outgoing light beam and a second outgoing light beam.

[0952] In addition, the above-mentioned collimation process of the light beam through the collimating lens can obtain a nearly parallel light beam, which can increase the power density of the light beam and further improve the effect of subsequent scanning using the light beam.

[0953] Optionally, the terminal device further includes a light homogenizing device, which is arranged between the laser light source and the optical element. Fig.78 The method shown also includes:

[0954] 7007. Use a light homogenizing device to adjust the energy distribution of the laser beam to obtain a light beam after light homogenization.

[0955] The above step 7002 specifically includes: controlling the optical element to control the direction of the light beam after the light homogenization process to obtain a first outgoing light beam and a second outgoing light beam.

[0956] The light uniformity processing can make the optical power of the laser beam more uniform in the angle space, or distribute it according to a specific rule, so as to prevent the local optical power from being too small, thereby avoiding blind spots in the final depth map of the target object.

[0957] Based on the above steps 7001 to 7005, Fig.78 The method shown may further include step 7006 or step 7007.

[0958] Alternatively, based on the above steps 7001 to 7005, Fig.78 The method shown may further include step 7006 and step 7007. In this case, after executing step 7001, step 7006 may be executed first, then step 7007, and then step 7002, or step 7007 may be executed first, then step 7006, and then step 7002. That is, after the laser light source in step 7001 generates a laser beam, the laser beam may be collimated and homogenized (the energy distribution of the laser beam is adjusted by using a homogenizing device), and then the optical element is controlled to control the direction of the laser beam. Alternatively, after the laser light source in step 7001 generates a laser beam, the laser beam may be homogenized (the energy distribution of the laser beam is adjusted by using a homogenizing device) and collimated, and then the optical element is controlled to control the direction of the laser beam.

[0959] Combination of the above Figures 70 to 78 A TOF depth sensing module and an image generation method in an embodiment of the present application are described in detail. Figures 79 to 88 Another TOF depth sensing module and image generation method of an embodiment of the present application are introduced in detail.

[0960] Since liquid crystal devices have excellent polarization and phase adjustment capabilities, they are widely used in TOF depth sensing modules to achieve light beam deflection. However, due to the birefringence characteristics of liquid crystal materials, existing TOF depth sensing modules using liquid crystal devices generally add polarizers at the transmitting end to achieve the emission of polarized light. In the process of polarized light emission, due to the polarization selection effect of the polarizer, half of the energy will be lost in the outgoing light beam. This lost energy will be absorbed or scattered by the polarizer and converted into heat, causing the temperature of the TOF depth sensing module to rise, affecting the stability of the TOF depth sensing module. Therefore, how to reduce the heat loss of the TOF depth sensing module and improve the signal-to-noise ratio of the TOF depth sensing module is a problem that needs to be solved.

[0961] The present application provides a new TOF depth sensing module, which reduces the heat loss of the system by transferring the polarizer from the transmitting end to the receiving end, while improving the signal-to-noise ratio of the system relative to background stray light.

[0962] Let's combine Fig.79 A brief introduction is given to the TOF depth sensing module of the embodiment of the present application.

[0963] Fig.79 The TOF depth sensing module 600 shown includes: a laser light source 610 , an optical element 620 , a beam splitter 630 , a receiving unit 640 and a control unit 650 .

[0964] These modules or units in the TOF depth sensing module 600 are introduced in detail below.

[0965] Laser Source 610:

[0966] The laser light source 610 is used to generate a laser beam.

[0967] Optionally, the laser light source 610 is a vertical cavity surface emitting laser (VCSEL).

[0968] Optionally, the laser light source 610 is a Fabry-Perot laser (can be referred to as FP laser for short).

[0969] Compared with a single VCSEL, a single FP laser can achieve greater power, and its electro-optical conversion efficiency is also higher than that of VCSEL, which can improve the scanning effect of the TOF depth sensing module.

[0970] Optionally, the wavelength of the laser light beam emitted by the laser light source 610 is greater than 900 nm.

[0971] Since the intensity of light greater than 900nm in sunlight is relatively weak, when the wavelength of the laser beam is greater than 900nm, it helps to reduce the interference caused by sunlight, thereby improving the scanning effect of the TOF depth sensing module.

[0972] Optionally, the wavelength of the laser beam emitted by the laser light source 610 is 940 nm or 1550 nm.

[0973] Since the intensity of light near 940nm or 1550nm in sunlight is relatively weak, when the wavelength of the laser beam is 940nm or 1550nm, the interference caused by sunlight can be greatly reduced, thereby improving the scanning effect of the TOF depth sensing module.

[0974] Optionally, the light emitting area of ​​the laser light source 610 is less than or equal to 5×5 mm 2 .

[0975] Since the size of the above-mentioned laser light source is relatively small, the TOF depth sensing module including the laser light source is relatively easy to be integrated into the terminal device, which can reduce the space occupied in the terminal device to a certain extent.

[0976] Optics 620:

[0977] The optical element 620 is arranged in the exit direction of the laser beam, and the optical element 620 is used to control the direction of the laser beam to obtain a first exit beam and a second exit beam, wherein the exit direction of the first exit beam is different from the exit direction of the second exit beam, and the polarization direction of the first exit beam is orthogonal to the polarization direction of the second exit beam.

[0978] Alternatively, if Fig.35 As shown, the above-mentioned optical element 620 may include: a transverse polarization control plate, a transverse liquid crystal polarization grating, a longitudinal polarization control plate and a longitudinal liquid crystal polarization grating, wherein the distances between the transverse polarization control plate, the transverse liquid crystal polarization grating, the longitudinal polarization control plate and the longitudinal liquid crystal polarization grating and the laser light source increase successively.

[0979] Alternatively, in the optical element 620, the distances between the longitudinal polarization control plate, the longitudinal liquid crystal polarization grating, the transverse polarization control plate, the transverse liquid crystal polarization grating and the laser light source increase in sequence.

[0980] Receiving unit 640:

[0981] The receiving unit 640 may include a receiving lens 641 and a sensor 642 .

[0982] Beam Splitter 630:

[0983] The beam splitter 630 is used to transmit the third reflected beam obtained by the target object reflecting the first outgoing beam and the fourth reflected beam obtained by the target object reflecting the second outgoing beam to different areas of the sensor.

[0984] The above-mentioned beam splitter is a passive selection device, which is generally not controlled by the control unit and can propagate light beams of different polarization states in a mixed polarization state light beam to different areas of the receiving unit respectively.

[0985] Optionally, the above-mentioned beam splitter is implemented based on any one of a liquid crystal polarization grating LCPG, a polarization beam splitting prism PBS and a polarization filter.

[0986] In the present application, by transferring the polarizer from the transmitting end to the receiving end, the heat loss of the system can be reduced. In addition, by setting a beam splitter at the receiving end, the signal-to-noise ratio of the TOF depth sensing module can be improved.

[0987] like Fig.80 As shown, the TOF depth sensing module 600 may further include: a collimating lens 660, which is disposed in the emission direction of the laser light beam, and the collimating lens 660 is disposed between the laser light source 610 and the optical element 620, and the collimating lens 660 is used to collimate the laser light beam to obtain a collimated light beam; when the collimating lens 660 is disposed between the laser light source 610 and the optical element 620, the optical element 620 is used to control the direction of the collimated light beam to obtain a first emission light beam and a second emission light beam.

[0988] The above-mentioned collimation process of the light beam by the collimating lens can obtain a nearly parallel light beam, which can increase the power density of the light beam and further improve the effect of subsequent scanning using the light beam.

[0989] Optionally, the clear aperture of the collimating lens is less than or equal to 5 mm.

[0990] Since the size of the collimating lens is relatively small, the TOF depth sensing module including the collimating lens can be easily integrated into the terminal device, which can reduce the space occupied in the terminal device to a certain extent.

[0991] like Fig.81 As shown, the TOF depth sensing module 600 may further include:

[0992] The light homogenizer 670 is arranged in the emission direction of the laser beam, and the light homogenizer 670 is arranged between the laser light source and the optical element. The light homogenizer 670 is used to adjust the energy distribution of the laser beam to obtain a light beam after homogenization; when the light homogenizer 670 is arranged between the laser light source 610 and the optical element 620, the optical element 620 is used to control the direction of the light beam after homogenization to obtain a first output light beam and a second output light beam.

[0993] Optionally, the light homogenizing device may be a microlens diffuser or a diffractive optical diffuser.

[0994] It should be understood that the above-mentioned TOF depth sensing module 600 can include a collimating lens 660 and a light homogenizing device 670 at the same time. The collimating lens 660 and the light homogenizing device 670 can both be located between the laser light source 610 and the optical element 620. For the collimating lens 660 and the light homogenizing device 670, the distance between the collimating lens 660 and the laser light source can be closer, or the distance between the light homogenizing device 670 and the laser light source can be closer.

[0995] like Fig.82 As shown, the distance between the collimating lens 660 and the laser light source 610 is smaller than the distance between the light homogenizing device 670 and the laser light source 610 .

[0996] exist Fig.82 In the TOF depth sensing module 600 shown, the laser beam emitted by the laser light source 610 is first collimated by the collimating lens 660 , and then evenly treated by the light evening device 670 before being transmitted to the optical element 620 for processing.

[0997] like Fig.83 As shown, the distance between the collimating lens 660 and the laser light source 610 is greater than the distance between the light homogenizing device 670 and the laser light source 610 .

[0998] And in Fig.83 In the TOF depth sensing module 600 shown, the laser beam emitted by the laser light source 610 is first homogenized by the homogenizing device 670 , and then collimated by the collimating lens 660 before being transmitted to the optical element 620 for processing.

[0999] The specific structure of the TOF depth sensing module 600 is described in detail below in conjunction with the accompanying drawings.

[1000] Fig.84 Schematic diagram of the structure of the TOF depth sensing module 600 according to an embodiment of the present application.

[1001] like Fig.84As shown, the TOF depth sensing module 600 includes a projection end and a receiving end, wherein the laser light source of the projection end is a VCSEL light source, the light homogenizing device is a diffraction optical diffuser (DOE Diffuser), and the beam element is a multilayer LCPG and a 1 / 4 wave plate, wherein each layer of LCPG includes: an electrically controlled horizontal LCPG component and an electrically controlled vertical LCPG component. Using multilayer cascaded LCPGs, two-dimensional block scanning in the horizontal and vertical directions can be achieved.

[1002] The wavelength of the VCSEL array light source may be greater than 900 nm. Specifically, the wavelength of the VCSEL array light source may be 940 nm or 1650 nm.

[1003] When the wavelength of the VCSEL array light source is 940nm or 1650nm, the intensity of the solar spectrum is relatively weak, which is beneficial to reduce the noise caused by sunlight in outdoor scenes.

[1004] The laser emitted by the VCSEL array light source can be continuous light or pulsed light. The VCSEL array light source can also be divided into several blocks to achieve time-sharing control, so that different areas can be lit up in different time periods.

[1005] The function of the diffractive optical diffuser is to shape the light beam emitted by the VCSEL array light source into a uniform square or rectangular light source with a certain FOV (for example, a FOV of 5°×5°).

[1006] The function of multilayer LCPG and 1 / 4 wave plate is to realize the scanning of light beam.

[1007] The receiving end and the transmitting end share multiple layers of LCPG and a quarter wave plate. The receiving lens of the receiving end can be a single lens or a combination of multiple lenses. The sensor at the receiving end is a single photon avalanche diode (SPAD) array. Since SPAD has the sensitivity of single photon detection, it can improve the detection distance of the TOF depth sensing module 600. The receiving end includes a beam splitter, which is implemented by a single layer of LCPG. At the same time, the projection end will use two polarization states of light to project into different FOV ranges, and then pass through the multi-layer LCPG at the receiving end to merge into the same beam of light, and then pass through the beam splitter to split into two beams of light in different directions according to the different deflection states, and project to different positions of the SPAD array.

[1008] Fig.85 Schematic diagram of the structure of the TOF depth sensing module 600 according to an embodiment of the present application.

[1009] Fig.85 The TOF depth sensing module 600 shown in FIG. Fig.84 The difference between the TOF depth sensing module 600 shown in FIG. Fig.84In the example, the beam splitter is realized by a single layer of LCPG, while in Fig.85 In the present invention, the beam splitter is realized by a polarization beam splitter, which is usually made of coated edges and corners glued together. Since the polarization beam splitter is a ready-made product, the use of the polarization beam splitter as the beam splitter has certain cost advantages.

[1010] like Fig.85 As shown, the two orthogonal polarization states of the reflected light beam will be separated on the polarization beam splitter, one will directly enter the SPAD array sensor, and the other will be reflected and then reflected by another reflector into the SPAD array sensor.

[1011] Fig.86 It is a schematic diagram of the structure of the TOF depth sensing module of an embodiment of the present application.

[1012] and Fig.84 The difference between the TOF depth sensing module 600 shown in FIG. Fig.86 In the case of a beam splitter, a polarization filter is used to implement the beam splitter. Fig.86 In this case, a quarter wave plate can be used.

[1013] Polarization filters are processed similar to pixel art, and the polarization states that can be transmitted on adjacent pixels are different and correspond to each SPAD pixel. In this way, the SPAD sensor can receive two polarization state information at the same time.

[1014] Fig.87 is a schematic diagram of the polarization filter receiving a polarized light beam.

[1015] like Fig.87 As shown, different areas of the polarization filter can transmit H polarization or V polarization, where H polarization represents horizontal polarization and V polarization represents vertical polarization. Fig.87 In the example, different areas on the polarization filter only allow light beams of the corresponding polarization state to reach the corresponding position of the sensor. For example, H polarization only allows vertical and horizontal polarization light beams to reach the corresponding position of the sensor, and V polarization only allows vertical polarization light beams to reach the corresponding position of the sensor.

[1016] When the beam splitter adopts a polarization filter, since the polarization filter is relatively thin and small in size, it is easy to be integrated into a terminal device with a smaller size.

[1017] Fig.88 It is a schematic flowchart of the image generation method of an embodiment of the present application.

[1018] Fig.88 The method shown can be executed by a TOF depth sensing module or a terminal device including the TOF depth sensing module of the embodiment of the present application. Specifically, Fig.88 The method shown can be Fig.79 The TOF depth sensor module shown may include Fig.79 The TOF depth sensing module is used to execute the terminal device shown. Fig.88 The method shown includes steps 8001 to 8006, each of which is described in detail below.

[1019] 8001. Control the laser light source to generate a laser beam.

[1020] 8002. Control the optical element to control the direction of the laser beam to obtain a first outgoing beam and a second outgoing beam.

[1021] The emitting direction of the first emergent light beam is different from the emitting direction of the second emergent light beam, and the polarization direction of the first emergent light beam is orthogonal to the polarization direction of the second emergent light beam.

[1022] 8003. Control the beam splitter to propagate the third reflected beam obtained by the target object reflecting the first outgoing beam and the fourth reflected beam obtained by the target object reflecting the second outgoing beam to different areas of the receiving unit.

[1023] 8004. Generate a first depth map of the target object according to the TOF corresponding to the first outgoing light beam.

[1024] 8005. Generate a second depth map of the target object according to the TOF corresponding to the second outgoing light beam.

[1025] Above Fig.88 The process of the method shown is similar to Fig.78 The basic process of the method shown is the same, the main difference is that in Fig.78 In step 7003 of the method shown, the third reflected light beam and the fourth reflected light beam are propagated to different areas of the receiving unit through the beam selection device. Fig.88 In step 8003 of the method shown, the third reflected light beam and the fourth reflected light beam are propagated to different areas of the receiving unit through a beam splitter.

[1026] In the embodiment of the present application, since there is no polarization filter device at the transmitting end, the light beam emitted by the laser light source can reach the optical element with almost no loss (polarization filter devices generally absorb more light energy and thus produce certain heat loss), which can reduce the heat loss of the terminal equipment.

[1027] Optionally, Fig.88 The method shown also includes: splicing the first depth map and the second depth map to obtain a depth map of the target object.

[1028] It should be understood that Fig.88In the method shown, a third depth map, a fourth depth map, and so on may be generated in a similar manner. Next, all the depth maps may be spliced ​​or combined to obtain a final depth map of the target object.

[1029] Optionally, the terminal device further comprises a collimating lens, which is arranged between the laser light source and the optical element. Fig.88 The method shown also includes:

[1030] 8006. Using a collimating lens to collimate the laser beam to obtain a collimated beam;

[1031] The above step 8002 specifically includes: controlling the optical element to control the direction of the collimated light beam to obtain a first outgoing light beam and a second outgoing light beam.

[1032] In addition, the above-mentioned collimation process of the light beam through the collimating lens can obtain a nearly parallel light beam, which can increase the power density of the light beam and further improve the effect of subsequent scanning using the light beam.

[1033] Optionally, the terminal device further includes a light homogenizing device, which is arranged between the laser light source and the optical element. Fig.88 The method shown also includes:

[1034] 8007. Use a light homogenizing device to adjust the energy distribution of the laser beam to obtain a light beam after light homogenization.

[1035] The above step 8002 specifically includes: controlling the optical element to control the direction of the laser beam to obtain the first outgoing beam and the second outgoing beam, including: controlling the optical element to control the direction of the beam after the homogenization process to obtain the first outgoing beam and the second outgoing beam.

[1036] The light uniformity processing can make the optical power of the laser beam more uniform in the angle space, or distribute it according to a specific rule, so as to prevent the local optical power from being too small, thereby avoiding blind spots in the final depth map of the target object.

[1037] Based on the above steps 8001 to 8005, Fig.88 The method shown may further include step 8006 or step 8007.

[1038] Alternatively, based on the above steps 8001 to 8005, Fig.88The method shown may further include step 8006 and step 8007. In this case, after executing step 8001, step 8006 may be executed first, then step 8007, and then step 8002, or step 8007 may be executed first, then step 8006, and then step 8002. That is, after the laser light source in step 8001 generates a laser beam, the laser beam may be collimated and homogenized (the energy distribution of the laser beam is adjusted by a homogenizing device), and then the optical element is controlled to control the direction of the laser beam. Alternatively, after the laser light source in step 8001 generates a laser beam, the laser beam may be homogenized (the energy distribution of the laser beam is adjusted by a homogenizing device) and collimated, and then the optical element is controlled to control the direction of the laser beam.

[1039] Combination of the above Figures 79 to 88 A TOF depth sensing module and an image generation method in an embodiment of the present application are described in detail. Figure 89 to Figure 101 Another TOF depth sensing module and image generation method of an embodiment of the present application are introduced in detail.

[1040] Due to the excellent polarization and phase adjustment capabilities of liquid crystal devices, liquid crystal devices are often used to control the light beam in TOF depth sensing modules. However, due to the limitations of the liquid crystal material itself, its response time has a certain limit, usually in the millisecond range. As a result, the scanning frequency of TOF depth sensing modules using liquid crystal devices is relatively low (usually less than 1khz).

[1041] The present application provides a new TOF depth sensing module, which achieves an increase in the system scanning frequency by controlling the timing of the driving signals of the electrically controlled liquid crystal at the transmitting end and the receiving end to be staggered by a certain time (for example, half a cycle).

[1042] Let's combine Fig.89 A brief introduction is given to the TOF depth sensing module of the embodiment of the present application.

[1043] Fig.89 The TOF depth sensing module 700 shown includes: a laser light source 710 , an optical element 720 , a beam selection device 730 , a receiving unit 740 and a control unit 750 .

[1044] The functions of each module or unit in the above TOF depth sensing module are as follows:

[1045] Laser source 710:

[1046] The laser light source 710 is used to generate a laser beam.

[1047] Optionally, the laser light source 710 is a vertical cavity surface emitting laser (VCSEL).

[1048] Optionally, the laser light source 710 is a Fabry-Perot laser (can be referred to as FP laser for short).

[1049] Compared with a single VCSEL, a single FP laser can achieve greater power, and its electro-optical conversion efficiency is also higher than that of VCSEL, which can improve the scanning effect of the TOF depth sensing module.

[1050] Optionally, the wavelength of the laser light beam emitted by the laser light source 710 is greater than 900 nm.

[1051] Since the intensity of light greater than 900nm in sunlight is relatively weak, when the wavelength of the laser beam is greater than 900nm, it helps to reduce the interference caused by sunlight, thereby improving the scanning effect of the TOF depth sensing module.

[1052] Optionally, the wavelength of the laser beam emitted by the laser light source 710 is 940 nm or 1550 nm.

[1053] Since the intensity of light near 940nm or 1550nm in sunlight is relatively weak, when the wavelength of the laser beam is 940nm or 1550nm, the interference caused by sunlight can be greatly reduced, thereby improving the scanning effect of the TOF depth sensing module.

[1054] Optionally, the light emitting area of ​​the laser light source 710 is less than or equal to 5×5 mm 2 .

[1055] Since the size of the above-mentioned laser light source is relatively small, the TOF depth sensing module including the laser light source is relatively easy to be integrated into the terminal device, which can reduce the space occupied in the terminal device to a certain extent.

[1056] Optionally, the average output optical power of the TOF depth sensing module 700 is less than 800mw.

[1057] When the average output optical power of the TOF depth sensing module is less than or equal to 800mw, the power consumption of the TOF depth sensing module is relatively small, and it is convenient to be installed in devices that are sensitive to power consumption, such as terminal devices.

[1058] Optics 720:

[1059] The optical element 720 is disposed in the direction of the light beam emitted by the laser light source. The optical element 720 is used to deflect the laser light beam under the control of the control unit 750 to obtain an output light beam.

[1060] Beam selection device 730:

[1061] The beam selection device 730 is used to select a beam with at least two polarization states from the beams in each period of the reflected beam of the target object under the control of the control unit 750 to obtain a received beam, and transmit the received beam to the receiving unit 740.

[1062] Among them, the above-mentioned outgoing light beam is a periodically changing light beam, the size of the changing period of the outgoing light beam is the first time interval, in the outgoing light beam, the inclination angles of the light beams located in adjacent periods are different, the light beams located in the same period have at least two polarization states, and the inclination angles of the light beams located in the same period are the same and the azimuth angles are different.

[1063] In an embodiment of the present application, the direction and polarization state of the light beam emitted by the laser light source are adjusted through optical elements and beam selection devices, so that the inclination angles of the output light beams in adjacent periods are different, and the light beams in the same period have at least two polarization states, thereby improving the scanning frequency of the TOF depth sensing module.

[1064] In the present application, the control unit controls the control signal timing of the transmitting end and the receiving end to stagger by a certain time, thereby increasing the scanning frequency of the TOF depth sensing module.

[1065] Alternatively, if Fig.35 As shown, the optical element 720 comprises: a transverse polarization control plate, a transverse liquid crystal polarization grating, a longitudinal polarization control plate and a longitudinal liquid crystal polarization grating. The distances between the transverse polarization control plate, the transverse liquid crystal polarization grating, the longitudinal polarization control plate and the longitudinal liquid crystal polarization grating and the laser light source increase in sequence.

[1066] Alternatively, in the optical element 720, the distances between the longitudinal polarization control plate, the longitudinal liquid crystal polarization grating, the transverse polarization control plate, and the transverse liquid crystal polarization grating and the laser light source increase in sequence.

[1067] Optionally, the above-mentioned beam selection device is composed of a 1 / 4 wave plate + an electrically controlled half-wave plate + a polarizer.

[1068] like Fig.90As shown, the TOF depth sensing module may further include: a collimating lens 760, which is disposed between the laser light source 710 and the optical element 720, and is used to collimate the laser light beam; the optical element 720 is used to deflect the light beam collimated by the collimating lens under the control of the control unit 750 to obtain an outgoing light beam.

[1069] When the TOF depth sensing module includes a collimating lens, the collimating lens can be used to collimate the light beam emitted by the laser light source first, so that a nearly parallel light beam can be obtained, the power density of the light beam can be increased, and the effect of subsequent scanning using the light beam can be improved.

[1070] Optionally, the clear aperture of the collimating lens is less than or equal to 5 mm.

[1071] Since the size of the collimating lens is relatively small, the TOF depth sensing module including the collimating lens can be easily integrated into the terminal device, which can reduce the space occupied in the terminal device to a certain extent.

[1072] like Fig.91 As shown, the TOF depth sensing module 700 further includes: a light homogenizer 770, which is disposed between the laser light source 710 and the optical element 720, and is used to adjust the angular spatial intensity distribution of the laser beam; the optical element 720 is used to control the direction of the light beam homogenized by the light homogenizer 770 under the control of the control unit 750, to obtain an outgoing light beam.

[1073] Optionally, the light homogenizing device 770 is a microlens diffuser or a diffractive optical diffuser.

[1074] The light uniformity processing can make the optical power of the laser beam more uniform in the angle space, or distribute it according to a specific rule, so as to prevent the local optical power from being too small, thereby avoiding blind spots in the final depth map of the target object.

[1075] It should be understood that the above-mentioned TOF depth sensing module 700 can include a collimating lens 760 and a light homogenizing device 770 at the same time. The collimating lens 760 and the light homogenizing device 770 can both be located between the laser light source 710 and the optical element 720. For the collimating lens 760 and the light homogenizing device 770, the distance between the collimating lens 760 and the laser light source can be closer, or the distance between the light homogenizing device 770 and the laser light source can be closer.

[1076] Fig.92 It is a schematic diagram of the specific structure of the TOF depth sensing module of an embodiment of the present application.

[1077] like Fig.92As shown, the distance between the collimating lens 760 and the laser light source 710 is smaller than the distance between the light homogenizing device 770 and the laser light source 710 .

[1078] exist Fig.92 In the TOF depth sensing module 700 shown, the laser beam emitted by the laser light source 710 is first collimated by the collimating lens 760 , and then evenly treated by the light evening device 770 before being transmitted to the optical element 720 for processing.

[1079] Fig.93 It is a schematic diagram of the specific structure of the TOF depth sensing module of an embodiment of the present application.

[1080] like Fig.93 As shown, the distance between the collimating lens 760 and the laser light source 710 is greater than the distance between the light homogenizing device 770 and the laser light source 710 .

[1081] And in Fig.93 In the TOF depth sensing module 700 shown, the laser beam emitted by the laser light source 710 is first homogenized by the light homogenizer 770 , and then collimated by the collimating lens 760 before being transmitted to the optical element 720 for processing.

[1082] Combine the following Fig.94 and Fig.95 The working process of the TOF depth sensing module 700 is described.

[1083] Fig.94 As shown, assuming that the highest frequencies of the electronic control devices at the transmitting and receiving ends of the TOF depth sensing module 700 are both 1 / T, then the control timing of the transmitting and receiving ends is staggered by half a cycle (0.5T) through the control unit. Then, every 0.5T, the receiving sensor can receive light beams at different spatial positions.

[1084] like Fig.95 As shown, during the time from 0 to 0.5T, the receiving end sensor receives the light beam in angle 1 state A; during the time from 0.5T to T, the receiving end sensor receives the light beam in angle 1 state B; during the time from T to 1.5T, the receiving end sensor receives the light beam in angle 2 state A; during the time from 1.5T to 2T, the receiving end sensor receives the light beam in angle 2 state B. In this way, the scanning frequency of the system is increased from 1 / T to 2 / T, which is doubled.

[1085] The specific structure of the TOF depth sensing module 700 is described in detail below with reference to the accompanying drawings.

[1086] Fig.96 Schematic diagram of the structure of the TOF depth sensing module 700 according to an embodiment of the present application.

[1087] like Fig.96 As shown, the TOF depth sensing module 700 includes a projector, a receiver and a control unit. The projector includes: a light source, a light homogenizing device, and an optical element; the receiver includes: an optical element, a beam selection device, a receiving lens, and a two-dimensional sensor; the control unit is used to control the projector and the receiver to complete the scanning of the beam.

[1088] The laser light source at the projection end is a VCSEL light source, the light homogenizing device is a diffraction optical diffuser (DOEDiffuser), and the beam element is a multi-layer LCPG and a 1 / 4 wave plate, wherein each layer of LCPG includes: an electrically controlled horizontal LCPG component and an electrically controlled vertical LCPG component. Using multi-layer cascaded LCPGs, two-dimensional block scanning in the horizontal and vertical directions can be achieved.

[1089] The wavelength of the VCSEL array light source may be greater than 900 nm. Specifically, the wavelength of the VCSEL array light source may be 940 nm or 1650 nm.

[1090] When the wavelength of the VCSEL array light source is 940nm or 1650nm, the intensity of the solar spectrum is relatively weak, which is beneficial to reduce the noise caused by sunlight in outdoor scenes.

[1091] The laser emitted by the VCSEL array light source can be continuous light or pulsed light. The VCSEL array light source can also be divided into several blocks to achieve time-sharing control, so that different areas can be lit up in different time periods.

[1092] The function of the diffractive optical diffuser is to shape the light beam emitted by the VCSEL array light source into a uniform square or rectangular light source with a certain FOV (for example, a FOV of 5°×5°).

[1093] The function of multilayer LCPG and 1 / 4 wave plate is to realize the scanning of light beam.

[1094] This application can dynamically select light of different angles and different states to enter the sensor through time-sharing control of the transmitter and the receiver. Fig.96As shown, the laser emitted by the ordinary VCSEL array light source has no fixed polarization state and can be decomposed into a linear polarized laser parallel to the paper surface and a linear polarized laser perpendicular to the paper surface. After the linear polarized laser passes through the LCPG, it will be divided into two laser beams with different polarization states (left-handed circular polarization and right-handed circular polarization), which are distributed with different emission angles. After the two laser beams pass through the 1 / 4 wave plate, the corresponding polarization states are converted into linear polarization parallel to the paper surface and linear polarization perpendicular to the paper surface. The two laser beams with different polarization states are irradiated to the object in the target area. The reflected light beams generated by the two laser beams with different polarization states will be received by the 1 / 4 wave plate and LCPG shared with the transmitting end and become laser beams with the same divergence angle but different deflection states - left-handed circular polarization and right-handed circular polarization. The beam selection device at the receiving end consists of a 1 / 4 wave plate + an electrically controlled half-wave plate + a polarizer. After the received light passes through this 1 / 4 wave plate, the polarization state is converted into linear polarization parallel to the paper surface and linear polarization perpendicular to the paper surface. In this way, the electrically controlled half-wave plate is controlled in a time-sharing manner to rotate the polarization state of the linear polarized light by 90 degrees or not change the polarization state after passing through the half-wave plate, thereby realizing the time-sharing transmission of the linear polarization parallel to the paper surface and the linear polarization perpendicular to the paper surface. At the same time, the light in the other polarization state is absorbed or scattered by the polarizer.

[1095] exist Fig.96 In the example, the time-sharing control signals of the transmitter and receiver can be Fig.94 As shown, by staggering the control timing of the electrically controlled LCPG at the transmitting end and the electrically controlled half-wave plate at the receiving end by half a cycle (0.5T), the system scanning frequency can be increased by 1 times.

[1096] Fig.97 Schematic diagram of the structure of the TOF depth sensing module 700 according to an embodiment of the present application.

[1097] like Fig.97 As shown, in Fig.96 Based on the TOF depth sensing module shown in the figure, the diffraction optical diffuser (DOE Diffuser) behind the VCSEL array light source is replaced by a microlens diffuser (Diffuser). Since the microlens diffuser is based on geometric optics to achieve uniform light, its transmission efficiency is relatively high and can reach more than 80%, while the transmission efficiency of the traditional diffraction optical diffuser (DOEDiffuser) is only about 70%. The morphology of the microlens diffuser is as follows Fig.77 As shown, it is composed of a series of randomly distributed micro-lenses. The position and morphology of each micro-lens are designed through simulation optimization, so that the shaped light beam is as uniform as possible and the transmission efficiency is high.

[1098] Fig.97 The TOF depth sensor module shown is Fig.96 The driving principle of the TOF depth sensor module shown is the same as that of Fig.96The diffraction optical diffuser (DOE Diffuser) in the TOF depth sensing module shown is replaced with an optical diffuser to improve the transmission efficiency of the transmitting end, and the rest is not repeated here.

[1099] for Fig.97 For the TOF depth sensing module shown in the figure, the time-sharing control signals of the transmitter and the receiver can be as follows Fig.94 As shown, by staggering the control timing of the electrically controlled LCPG at the transmitting end and the electrically controlled half-wave plate at the receiving end by half a cycle (0.5T), the system scanning frequency can be increased by 1 times.

[1100] Fig.98 Schematic diagram of the structure of the TOF depth sensing module 700 according to an embodiment of the present application.

[1101] exist Fig.96 or Fig.97 Based on the TOF depth sensing module shown in FIG. 1 , the optical element can be changed from a multi-layer LCPG plus a 1 / 4 wave plate to a multi-layer flat liquid crystal cell, such as Fig.98 As shown. Multi-layer flat liquid crystal cells are used to achieve beam deflection at multiple angles and in horizontal and vertical directions. The beam selection device at the receiving end is composed of an electrically controlled half-wave plate and a polarizer.

[1102] The principle of beam deflection in a flat-panel liquid crystal cell is as follows Fig.99 and Fig.100 As shown in FIG. 1 , a wedge-shaped polymer interface is used to achieve beam deflection. The refractive index of the wedge-shaped polymer material must be equal to the ordinary light refractive index n0 of the liquid crystal. Fig.99 As shown in the figure, when the optical axis of the liquid crystal molecules is parallel to the x-direction, the incident light parallel to the paper surface will be deflected at a certain angle. The magnitude of the deflection angle can be controlled by controlling the voltage applied thereto, while the incident light perpendicular to the paper surface will propagate in a straight line. In this way, by stacking multiple layers of flat liquid crystal cells with different orientations (optical axes parallel to the x-direction or the y-direction), the deflected incident light can be projected to different angles at the same time.

[1103] By the same principle, by controlling the driving voltage of the flat-panel liquid crystal box at the transmitting end and the driving voltage of the electrically controlled half-wave plate at the receiving end, the control timing of the two can be staggered by half a cycle (0.5T), which can increase the liquid crystal scanning frequency.

[1104] Fig.101 It is a schematic flowchart of the image generation method of an embodiment of the present application.

[1105] Fig.101 The method shown can be executed by a TOF depth sensing module or a terminal device including the TOF depth sensing module of the embodiment of the present application. Specifically, Fig.101 The method shown can be Fig.89 The TOF depth sensor module shown may include Fig.89 The TOF depth sensing module is used to execute the terminal device shown. Fig.101 The method shown includes steps 9001 to 9004, each of which is described in detail below.

[1106] 9001. Control the laser light source to generate a laser beam.

[1107] 9002. Control the optical element to deflect the laser beam to obtain an output beam.

[1108] 9003. Control the beam selection device to select a beam with at least two polarization states from the beams in each period of the reflected beam of the target object to obtain a received beam, and transmit the received beam to a receiving unit.

[1109] 9004. Generate a depth map of the target object according to the TOF corresponding to the outgoing light beam.

[1110] Among them, the above-mentioned outgoing light beam is a periodically changing light beam, the size of the changing period of the outgoing light beam is the first time interval, in the outgoing light beam, the inclination angles of the light beams located in adjacent periods are different, the light beams located in the same period have at least two polarization states, and the inclination angles of the light beams located in the same period are the same and the azimuth angles are different.

[1111] The TOF corresponding to the above-mentioned outgoing light beam may specifically refer to the time difference information between the moment when the reflected light beam corresponding to the outgoing light beam is received by the receiving unit and the moment when the outgoing light source emits. The reflected light beam corresponding to the outgoing light beam may specifically refer to the light beam generated after the outgoing light beam reaches the target object after being processed by the optical element and the beam selection device and reflected after passing through the target object.

[1112] In an embodiment of the present application, the direction and polarization state of the light beam emitted by the laser light source are adjusted through optical elements and beam selection devices, so that the inclination angles of the output light beams in adjacent periods are different, and the light beams in the same period have at least two polarization states, thereby improving the scanning frequency of the TOF depth sensing module.

[1113] Optionally, the terminal device further comprises a collimating lens, which is arranged between the laser light source and the optical element. In this case, Fig.101 The method shown also includes:

[1114] 9005. Using a collimating lens to collimate the laser beam to obtain a collimated beam;

[1115] In the above step 9002, the laser beam is deflected to obtain an outgoing beam, which specifically includes: controlling the optical element to control the direction of the collimated beam to obtain the outgoing beam.

[1116] The above-mentioned collimation process of the light beam by the collimating lens can obtain a nearly parallel light beam, which can increase the power density of the light beam and further improve the effect of subsequent scanning using the light beam.

[1117] Optionally, the terminal device further includes a light homogenizing device, which is arranged between the laser light source and the optical element. In this case, Fig.101 The method shown also includes:

[1118] 9006. Using a light homogenizing device to adjust the energy distribution of the laser beam to obtain a light beam after light homogenization;

[1119] In the above step 9002, the laser beam is deflected to obtain an outgoing beam, which specifically includes: controlling the optical element to control the direction of the beam after the homogenization process to obtain the outgoing beam.

[1120] The light uniformity processing can make the optical power of the laser beam more uniform in the angle space, or distribute it according to a specific rule, so as to prevent the local optical power from being too small, thereby avoiding blind spots in the final depth map of the target object.

[1121] Combine the following Fig.102 and Fig.103 The FOV of the light beam processed by the light beam shaping device in the TOF depth sensing module 300 is introduced.

[1122] The beam shaping device 330 in the TOF depth sensing module 300 adjusts the laser beam to obtain a first beam, and the FOV of the first beam ranges from [5°×5°, 20°×20°].

[1123] Fig.102 is a schematic diagram of the FOV of the first beam.

[1124] like Fig.102 As shown, the first light beam is emitted from point O, the FOV of the first light beam in the vertical direction is angle A, the FOV in the horizontal direction is angle B, and rectangle E is the area projected by the first light beam on the target object (the first light beam projected on the target object can be a rectangular area, of course, it can also be other shapes). Among them, the value range of angle A is between 5° and 20° (including 5° and 20°), and similarly, the value range of angle B is also between 5° and 20° (including 5° and 20°).

[1125] In the above-mentioned TOF depth sensing module 300, the control unit 370 can be used to control the first optical element to control the direction of the first light beam at M different times respectively to obtain M outgoing light beams in different directions, and the total FOV covered by the M outgoing light beams in different directions ranges from [50°×50°, 80°×80°].

[1126] Fig.103 It is a schematic diagram of the FOV covered by M outgoing light beams in different directions.

[1127] Specifically, Fig.103 As shown, M outgoing light beams in different directions are emitted from point O, and the area covered on the target object is a rectangle F, wherein angle C is the superposition value of the FOV of the outgoing light beams in different directions in the vertical direction, and angle D is the superposition value of the FOV of the outgoing light beams in different directions in the horizontal direction. The value range of angle C is between 50° and 80° (including 50° and 80°), and similarly, the value range of angle D is also between 50° and 80° (including 50° and 80°).

[1128] The above combination Fig.102 and Fig.103 The explanation of the FOV of the first light beam generated by the TOF depth sensing module 300 and the M outgoing light beams in different directions is also applicable to the first light beam generated by the TOF depth sensing module 400 and the M outgoing light beams in different directions, and will not be repeated here.

[1129] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[1130] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[1131] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[1132] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[1133] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[1134] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magneti...

Claims

1. A TOF depth sensing module, characterized in that: The TOF depth sensing module includes a laser light source, a polarization filter device, a beam shaping device, an optical element, a receiving unit and a control unit, wherein the laser light source can generate light in a variety of polarization states, and the polarization filter device is located between the laser light source and the beam shaping device; The laser light source is used to generate a laser beam; The polarization filter device is used to filter the laser beam to obtain a beam in a single polarization state, where the single polarization state is one of the multiple polarization states; The beam shaping device is used to adjust the FOV of the light beam in a single polarization state to obtain a first light beam, wherein the FOV of the first light beam ranges from [5°×5°, 20°×20°]; The control unit is used to control the birefringence parameter of the optical element to control the direction of the first light beam to obtain an outgoing light beam; The control unit is further used to control the optical element to deflect the reflected light beam obtained by the target object reflecting the outgoing light beam to the receiving unit.

2. The TOF depth sensing module according to claim 1, characterized in that: The control unit is used for: Controlling the optical element to control the direction of the first light beam at M different moments respectively, to obtain M outgoing light beams in different directions, wherein the total FOV covered by the M outgoing light beams in different directions includes [50°×50°, 80°×80°]; The optical element is controlled to deflect M reflected light beams obtained by the target object reflecting the outgoing light beams in M ​​different directions to the receiving unit.

3. The TOF depth sensing module according to claim 2, characterized in that: The control unit is also used for: A depth map of the target object is generated according to the TOFs corresponding to the M outgoing light beams in different directions.

4. The TOF depth sensing module according to any one of claims 1 to 3, characterized in that: The optical element is a liquid crystal polarizing element.

5. The TOF depth sensing module according to any one of claims 1 to 3, characterized in that: The optical element is a rotating mirror device, wherein the rotating mirror device controls the emission direction of the emission light beam by rotating.

6. The TOF depth sensing module according to claim 4, characterized in that: The optical element comprises: a transverse polarization control plate, a transverse liquid crystal polarization grating, a longitudinal polarization control plate and a longitudinal liquid crystal polarization grating.

7. The TOF depth sensing module according to claim 6, characterized in that: The distances between the transverse polarization control plate, the transverse liquid crystal polarization grating, the longitudinal polarization control plate and the longitudinal liquid crystal polarization grating and the laser light source increase in sequence, or the distances between the longitudinal polarization control plate, the longitudinal liquid crystal polarization grating, the transverse polarization control plate and the transverse liquid crystal polarization grating and the laser light source increase in sequence.

8. The TOF depth sensing module according to claim 5, characterized in that: The rotating mirror device is a micro-electromechanical system galvanometer or a multi-faceted rotating mirror.

9. The TOF depth sensing module according to any one of claims 1 to 3, characterized in that: The beam shaping device is composed of a diffusion lens and a rectangular aperture.

10. The TOF depth sensing module according to any one of claims 1 to 3, characterized in that: The laser light source is a Fabry-Perot FP laser.

11. The TOF depth sensing module according to any one of claims 1 to 3, characterized in that: The laser light source is a vertical cavity surface emitting laser.

12. The TOF depth sensing module according to any one of claims 1 to 3, characterized in that: The light emitting area of ​​the laser light source is less than or equal to 5×5 mm 2 .

13. The TOF depth sensing module according to any one of claims 1 to 3, characterized in that: The average output optical power of the TOF depth sensing module is less than 800mw.

14. An image generation method, the image generation method is applied to a terminal device containing a TOF depth sensing module, the TOF depth sensing module includes a laser light source, a polarization filter device, a beam shaping device, an optical element, a receiving unit and a control unit, wherein: The laser light source is capable of generating light of multiple polarization states, and the polarization filter device is located between the laser light source and the beam shaping device. The image generation method includes: Controlling the laser light source to generate a laser beam; Filtering the laser beam using the polarization filter device to obtain a beam in a single polarization state, where the single polarization state is one of the multiple polarization states; Using the beam shaping device to adjust the light beam in a single polarization state to obtain a first light beam, wherein the FOV of the first light beam ranges from [5°×5°, 20°×20°]; Controlling the birefringence parameter of the optical element so that the optical element controls the direction of the first light beam from the beam shaping device at M different times, respectively, to obtain M outgoing light beams in different directions, wherein the total FOV covered by the M outgoing light beams in different directions includes [50°×50°, 80°×80°]; Controlling the optical element to deflect M reflected light beams obtained by the target object reflecting outgoing light beams in M ​​different directions to the receiving unit; Obtaining the TOFs corresponding to the M outgoing light beams in different directions respectively; A depth map of the target object is generated according to the TOFs corresponding to the M outgoing light beams in different directions.

15. The image generation method according to claim 14, characterized in that: The step of generating a depth map of the target object according to the TOFs respectively corresponding to the outgoing light beams in the M different directions includes: Determine the distances between the M regions of the target object and the TOF depth sensing module according to the TOFs corresponding to the M outgoing light beams in different directions respectively; Generate depth maps of the M regions of the target object according to the distances between the M regions of the target object and the TOF depth sensing module; A depth map of the target object is synthesized according to the depth maps of the M regions of the target object.

16. The image generation method according to claim 14 or 15, characterized in that: The step of adjusting the light beam in a single polarization state by using the light beam shaping device to obtain a first light beam comprises: The beam shaping device is used to adjust the angular spatial intensity distribution of the light beam in the single polarization state to obtain the first light beam.

Images