Wide-angle 3D sensing

Abstract

Aspects of the disclosure relate to depth sensing using a device. An example device includes a first light projector configured to project light toward a second light projector configured to project light toward the first light projector. The example device includes a reflection assembly located between the first light projector and the second light projector, the reflection assembly configured to: redirect the light projected by the first light projector onto a first portion of a scene, and redirect the light projected by the second light projector onto a second portion of the scene, and the first and second portions of the scene are adjacent to each other and non-overlapping with respect to each other. The example device includes a receiver configured to detect reflections of the redirected light projected by the first light projector and the second light projector.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. non-provisional application serial number 16 / 848,487, filed on April 14, 2020, entitled “WIDE-ANGLE 3D SENSING,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] In summary, this disclosure relates to depth sensing systems, and more specifically, to improving the speed and accuracy of active depth systems in generating depth information. Background Technology

[0004] Passive depth sensing systems measure ambient light reflected from objects or surfaces in a scene to determine the distance between the sensing system and the object or surface. Active depth sensing systems emit light pulses into the scene and measure the reflection of these pulses from objects or surfaces in the scene to determine the distance between the sensing system and the object or surface. Some active depth sensing systems may employ diffractive optics (DOEs) to diffract the emitted light pulses into additional emissions, which can increase the amount of light projected onto the scene. In some cases, the additional emissions can be used to create (and replicate) decoded light patterns on the scene.

[0005] The maximum diffraction angle of the decoded light pattern created by the DOE, and therefore the field of view (FOV) of the depth sensing system, may be limited by the feature size of the DOE. For depth sensing systems employing vertical-cavity surface-emitting lasers (VCSELs), distortion increases with the fan-out angle, which may further limit the FOV. Although the FOV of a depth sensing system can be effectively doubled by using two light projectors, image artifacts associated with stitching together depth information generated from the reflection of light emitted from different light projectors may hinder (or at least render impractical) the detection of objects and surfaces located along the “stitching area” of the scene. Summary of the Invention

[0006] The systems, methods, and apparatuses disclosed herein are innovative in several ways, but no single aspect is solely responsible for the intended properties of this disclosure.

[0007] Some aspects of the present disclosure relate to devices for depth sensing. An example device includes a first light projector, a second light projector, a reflection component, and a receiver. The example first light projector is configured to project light toward the example second light projector, which is configured to project light toward the example first light projector. The example reflection component is located between the first example light projector and the second example light projector and is configured to: redirect light projected by the first light projector onto a first portion of a scene and to redirect light projected by the second light projector onto a second portion of the scene. The first and second portions of the scene can be adjacent to each other and non-overlapping with respect to each other. The example receiver is configured to detect reflections of the redirected light projected by the first and second light projectors.

[0008] Some other aspects of the present disclosure relate to a method for depth sensing using a device. An example method includes projecting light from a first light projector of the device toward a second light projector of the device. The example method also includes projecting light from the second light projector toward the first light projector. The example method also includes redirecting, via a reflection component located between the first light projector and the second light projector, light projected by the first light projector onto a first portion of a scene and redirecting light projected by the second light projector onto a second portion of the scene. The first and second portions of the scene can be adjacent to each other and non-overlapping with respect to each other. The example method also includes detecting reflections of the redirected light projected by the first and second light projectors.

[0009] Some other aspects of the present disclosure relate to a non-transitory computer- readable medium storing instructions that, when executed by one or more processors of a device, cause the device to perform operations. Example operations include projecting light from a first light projector of the device toward a second light projector of the device. The example operations also include projecting light from the second light projector toward the first light projector. The example operations also include redirecting, via a reflection component located between the first light projector and the second light projector, light projected by the first light projector onto a first portion of a scene and redirecting light projected by the second light projector onto a second portion of the scene. The first and second portions of the scene can be adjacent to each other and non-overlapping with respect to each other. The example operations also include detecting reflections of the redirected light projected by the first and second light projectors.

[0010] The details of one or more implementations of the subject matter described in this disclosure are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, the drawings, and the claims. Note that the relative dimensions of the following drawings can not be drawn to scale. BRIEF DESCRIPTION OF DRAWINGS

[0011] ]Figure 1 An example of a time-of-flight (ToF) system is shown.

[0012] Figure 2A An example environment in which multipath interference (MPI) can impact ToF depth sensing is shown.

[0013] Figure 2B Another example environment in which MPI can impact ToF depth sensing is shown.

[0014] Figure 2C Another example environment in which MPI can impact ToF depth sensing is shown.

[0015] Figure 3 An example structured light (SL) system is shown.

[0016] Figure 4 A block diagram of an example device including a hybrid ToF and SL system is shown.

[0017] Figure 5 A timing diagram illustrating example operational timing for a number of components of ToF and SL systems is shown.

[0018] Figure 6A An example ToF and SL system is shown.

[0019] Figure 6B An example ToF and SL system operating in first and second modes is shown. Figure 6A

[0020] Figure 7 An example circuit diagram for demodulating a pixel cell is shown.

[0021] Figure 8A An example circuit diagram for a global shutter (GS) pixel array is shown.

[0022] Figure 8B Another example circuit diagram for a GS pixel array is shown.

[0023] Figure 9 An example circuit diagram for a hybrid ToF and SL pixel array operating in a rolling shutter (RS) implementation is shown.

[0024] Figure 10 A flow diagram illustrating an example process for depth sensing according to some implementations is shown.

[0025] Figure 11A A flow diagram illustrating an example process for depth sensing according to some implementations is shown.

[0026] Figure 11B ​A flow diagram illustrating an example process for depth sensing, in accordance with some implementations, is shown.

[0027] Figure 11C A flow diagram illustrating an example process for depth sensing, in accordance with some implementations, is shown.

[0028] Figure 12 A flow diagram illustrating an example process for depth sensing, in accordance with some implementations, is shown.

[0029] Figure 13A A flow diagram illustrating an example process for depth sensing, in accordance with some implementations, is shown.

[0030] Figure 13B A flow diagram illustrating an example process for depth sensing, in accordance with some implementations, is shown.

[0031] Figure 13C A flow diagram illustrating an example process for depth sensing, in accordance with some implementations, is shown.

[0032] Figure 14 A flow diagram illustrating an example process for depth sensing, in accordance with some implementations, is shown.

[0033] In the various figures, like reference numerals and names of elements indicate like elements. DETAILED DESCRIPTION

[0034] Implementations of the subject matter described in this disclosure can allow a field of view (FOV) of an active depth sensing system to be increased without distortion beyond the limits imposed by feature size of a diffractive optical element (DOE) and a maximum fan-out angle associated with a vertical cavity surface emitting laser (VCSEL), thereby allowing wide-angle 3D sensing by a relatively compact active depth sensing system. In some implementations, an active depth sensing system can include a first light projector configured to project light in a first direction, a second light projector configured to project light in a second direction opposite the first direction, and a reflective component positioned between the first light projector and the second light projector. The reflective component can be configured to redirect light projected by the first light projector onto a first portion of a scene and to redirect light projected by the second light projector onto a second portion of the scene adjacent to but not overlapping the first portion of the scene. The active depth sensing system can also include a controller to generate depth information based on reflections of the redirected light from the first portion and the second portion of the scene, while correcting for projection distortion based at least in part on one or more refraction angles associated with the reflective component.

[0035] In some implementations, the reflective assembly can include a first reflective element and a second reflective element. The first reflective element can be or include a prism configured to fold an optical path of light projected from the first light projector and to refract the folded light onto a first portion of the scene based at least in part on the folded optical path. The second reflective element can be or include a prism configured to fold an optical path of light projected from the second light projector and to refract the folded light onto a second portion of the scene based at least in part on the folded optical path. The prisms and folded optics employed by the reflective assembly can allow the active depth sensing system to seamlessly stitch together depth information generated from reflections of light emitted from the first light projector and the second light projector without stitching artifacts.

[0036] The active depth sensing system can also include a switchable diffuser configured to switch the light projector between a time-of-flight (ToF) sensing mode and a structured light (SL) sensing mode. In this way, the active depth sensing system can reap the benefits provided by both ToF and SL techniques while minimizing their respective shortcomings.

[0037] In the following description, numerous specific details are set forth (such as examples of specific components, circuits, and processes) to provide a thorough understanding of the present disclosure. The term“coupled” as used herein means connected, either directly or indirectly, through one or more intervening components or circuits. Additionally, in the following description and for purposes of explanation, specific nomenclature is set forth to provide a thorough understanding of the present disclosure. It will be apparent, however, to one skilled in the art that the present teachings can be practiced without these specific details. In other instances, well-known circuits and devices are shown in block diagram form to avoid obscuring the present teachings. Some portions of the detailed description that follow are presented in terms of processes, logic blocks, processing, and other symbolic representations of operations on data bits that can be stored within a computer memory. In the present disclosure, a process, logic block, process, etc., is conceived to be a self-consistent sequence of steps or instructions leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system.

[0038] It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussion, it is appreciated that throughout the present application, discussions utilizing terms such as "accessing," "receiving," "sending," "using," "selecting," "determining," "normalizing," "multiplying," "averaging," "monitoring," "comparing," "applying," "updating," "measuring," "deriving," "solving," or the like, refer to the actions and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system memories or registers into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.

[0039] In the drawings, a single block can be described as performing one or more functions; however, in actual practice, one or more of the functions performed by that block can be performed in one or more separate components, and / or using hardware, using software, or using a combination of hardware and software. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure. In addition, the example devices can include components other than those shown, including well-known components such as processors, memory, etc.

[0040] Aspects of the present disclosure are applicable to any appropriate electronic device (such as a security system, a smartphone, a tablet device, a laptop computer, a vehicle, a drone, or other device) that includes or is coupled to one or more active depth sensing systems. While described below with respect to a device having or coupled to two light projectors, aspects of the present disclosure are applicable to devices having any number of light projectors, and are therefore not limited to a specific device.

[0041] The term "device" is not limited to one physical object or a specific number of physical objects (such as one smart phone, one controller, one processing system, etc.). As used herein, a device can be any electronic device having one or more constituent parts that can implement at least some portions of the present disclosure. While the following specification and examples use the term "device" to describe various aspects of the present disclosure, the term "device" is not limited to a specific configuration, type, or number of objects. Additionally, the term "system" is not limited to a plurality of components or a specific embodiment. For example, a system can be implemented on one or more printed circuit boards or other substrates, and can have movable or stationary components. While the following specification and examples use the term "system" to describe various aspects of the present disclosure, the term "system" is not limited to a specific configuration, type, or number of objects.

[0042] Figure 1 An example of a ToF system 100 is shown. The ToF system 100 can be used to generate depth information of a scene including a surface 106, or can be used for other applications of ranging to other portions of a surface or scene. The ToF system 100 can include a transmitter 102 and a receiver 108. The transmitter 102 can be referred to as a "light projector," "transmitter," "projector," "emitter," etc., and should not be limited to a specific transmission component. Similarly, the receiver 108 can be referred to as a "light sensor," "detector," "sensor," "sensing element," "light detector," etc., and should not be limited to a specific receiving component.

[0043] The transmitter 102 can be configured to send, emit, or project a signal (e.g., a light field) onto a scene. While the ToF system is described in this example as emitting light (which can include near-infrared (NIR)), signals at other frequencies can be used, such as microwaves, radio frequency signals, sound, etc. The present disclosure should not be limited to a specific frequency range for the emitted signal.

[0044] The transmitter 102 transmits light 104 toward a scene that includes a surface 106. The transmitted light 104 includes light pulses 114 at known time intervals, such as periodically. The receiver 108 includes a sensor 110 to sense reflections 112 of the transmitted light 104. The reflections 112 include reflected light pulses 116, and the ToF system 100 determines a round-trip time 122 of the light by comparing a timing 118 of the transmitted light pulses and a timing 120 of the reflected light pulses 116. The light pulses can be amplitude modulated continuous wave (AMCW) light pulses. In some cases, the light reflected back to the receiver 108 can have a different phase than the light emitted from the transmitter 102. The phase difference can be used to determine the round-trip time of the emission. The distance of the surface 106 from the ToF system 100 can be calculated as half the round-trip time multiplied by the speed of emission (e.g., the speed of light for light emissions).

[0045] The sensor 110 can include an array of photodiodes to measure or sense the reflected light. Alternatively, the sensor 110 can include a CMOS sensor or other suitable light sensitive sensor that includes multiple pixels or regions to sense. The ToF system 100 identifies the reflected light pulses 116 sensed by the sensor 110 when the amplitude of the pulses is greater than a certain value. For example, the ToF system 100 measures the amplitude of ambient light and other interference in the absence of a signal and determines whether further measurements are greater than a previous measurement by a certain value.

[0046] In some implementations, the sensor 110 can include a sensor pixel that includes a photodiode (not shown for simplicity) to convert photons from the reflections 112 into an electrical current. The sensor pixel can include one or more capacitors to store energy from the electrical current. The ToF system 100 can calculate the distance between the ToF system 100 and the surface 106 in part by comparing the voltage to its corresponding phase. The ToF system 100 can open and close a shutter to expose the sensor 110 at multiple specific phase offsets relative to the pulse signal. During each exposure cycle, charge can be stored by one or more storage elements, such as by a capacitor.

[0047] As a non-limiting example, during a first exposure cycle, the first capacitor (Cl) can store a charge (Ql), and the second capacitor (C2) can store a charge (Q2), where Ql is the accumulated charge from the reflected signal when the shutter is opened with a 0° phase offset, and where Q2 is the accumulated charge from the reflected signal when the shutter is opened with a 180° phase offset. During a second exposure cycle, Cl can store a charge (Q3), and C2 can store a charge (Q4), where Q3 is the accumulated charge from the reflected signal when the shutter is opened with a 90° phase offset, and where Q4 is the accumulated charge from the reflected signal when the shutter is opened with a 270° phase offset. The ToF system 100 can calculate the phase offset between the pulsed signal and the reflected signal based on the charges stored on Cl and C2 for each of the exposure cycles

[0048]

[0049] The calculated phase offset between the pulsed signal and the reflected signal is proportional to the distance d between the respective sensor pixel and the surface 106:

[0050]

[0051] where c is the speed of light and f is the frequency of the modulated signal. Based on the determined distance from each pixel of the sensor 110 to the surface 106, the ToF system 100 can generate depth information for the surface 106.

[0052] Some environments (e.g., with corners, raised areas, and / or reflective surfaces) can cause different light pulses to reach the ToF system 100 along multiple reflection paths and recombine at the sensor 110, which is referred to as MPI. For the purposes of the discussion herein, MPI can also be referred to as “multipath effect” or “MPI effect.” MPI can cause the ToF system to overestimate the amount of charge accumulated for one or more phase offsets of the corresponding pulsed signal. The overestimation can cause the ToF system to inaccurately calculate the corresponding phase shift between the pulsed signal and the reflected signal Accordingly, the ToF system can inaccurately calculate the corresponding distance d from one or more of the sensor pixels to the object or scene, which can cause distortion (or “bumpiness”) of the corresponding depth information.

[0053] Figure 2AAn example environment 200 is shown in which MPI can affect ToF depth sensing. The ToF system includes emitter 215 and sensor 235. The scene includes object 225 and object 227. Object 227 can have a mirror-like surface. Emitter 215 sends pulse signal 220 and pulse signal 222 toward object 225. Pulse signal 220 and reflected signal 230 travel along direct path 240 to sensor 235. In contrast, pulse signal 222 and reflected signal 232 travel along indirect path 250 (e.g., reflect off of object 227) to sensor 235, such that reflected signal 232 can arrive at sensor 235 at a different time than reflected signal 230. In some aspects, sensor 235 can interpret reflected signals 230 and 232 as reflecting off of the same location on object 225. As such, when reflected signals 230 and 232 arrive at sensor 235 at two different times, sensor 235 can generate two different distances for that location on object 225, resulting in MPI.

[0054] Figure 2B Another example environment 260 is shown in which MPI can affect ToF depth sensing. The ToF system includes emitter 245 and sensor 265. The scene includes object 255 and object 257. Object 257 can have a semi-transparent surface. Emitter 245 sends pulse signal 221 and pulse signal 223. Pulse signal 221 and reflected signal 231 travel along path 241 (e.g., reflect off of object 255). Pulse signal 223 and reflected signal 233 travel along path 251 (e.g., reflect off of object 257). Reflected signal 233 can arrive at sensor 265 at a different time than reflected signal 231. In some aspects, sensor 265 can interpret reflected signals 231 and 233 as reflecting off of the same location on object 255. As such, when reflected signals 231 and 233 arrive at sensor 265 at two different times, sensor 265 can generate two different distances for that location on object 255, resulting in MPI.

[0055] Figure 2CAnother example environment 270 is shown in which MPI can affect ToF depth sensing. The ToF system includes emitter 275 and sensor 295. The scene includes object 285 and object 287, which can represent two walls that intersect at a corner point. Emitter 275 sends pulse signals 291, 292, and 293, and sensor 295 receives corresponding reflected signals 294. Two or more of these reflected signals 294 can arrive at sensor 295 at different times, possibly due to the reflective properties of object 285 or object 287. In some aspects, sensor 295 can interpret each reflected signal 294 as being reflected from the same location on object 285 or object 287. As such, if two or more reflected signals 294 arrive at sensor 295 at different times, sensor 295 can generate two or more different distances for that location on object 285 or 287, resulting in MPI. For example, a conventional ToF system can superimpose multiple reflected signals 294, resulting in an accurate distance calculation for the corresponding location on object 285 or object 287, and ultimately resulting in one or more regions in the corresponding depth map (not shown for simplicity) inaccurately appearing to have a uniform depth.

[0056] Figure 3 An example SL system 300 is shown. The SL system can send light in a distribution of points (or another suitable shape of focused light). A distribution of points can be referred to herein for discussion purposes as a “pattern,” “SL pattern,” “dot pattern,” and the like, and the pattern can be predefined. Light points can be projected onto a scene, and reflections of the light points can be received by the SL system. Depths of objects in the scene can be determined by comparing the received pattern of light to the sent pattern of light. In comparing the patterns, a portion of the predefined distribution for the sent light can be identified in the received light. The SL system can use an SL projector to project a distribution of light, such as a distribution of light points or other shapes.

[0057] SL system 300 (which can also be referred to herein as an SL system) can be used to generate depth information for a scene 306. For example, scene 306 can include a face, and SL system 300 can be used to identify or authenticate the face. SL system 300 can include a sender 302 and a receiver 308. Sender 302 can be referred to as a “sender,” “projector,” “emitter,” and the like, and should not be limited to a specific sending component. Throughout the following disclosure, the terms projector and sender can be used interchangeably. Receiver 308 can be referred to as a “detector,” “sensor,” “sensing element,” “photodetector,” and the like, and should not be limited to a specific receiving component.

[0058] Although the present disclosure refers to a distribution as a light distribution, any appropriate signal at other frequencies can be used (such as radio frequency waves, sound waves, etc.). Further, although the present disclosure refers to a distribution as including a plurality of light points, the light can be focused to any appropriate size and dimension. For example, the light can be projected in a line, a square, or any other appropriate dimension. Additionally, the present disclosure can refer to a distribution as a codeword distribution, where a defined portion of the distribution (such as a predefined block of light points) is referred to as a codeword. If the distribution of light points is known, then the codewords of the distribution can be known. However, the distribution can be organized in any manner, and the present disclosure should not be limited to a specific type of distribution or a specific type of signal or pulse.

[0059] The transmitter 302 can be configured to project or send a distribution 304 of light points onto a scene 306. White circles in the distribution 304 can indicate where no light is projected for a possible point location, and black circles in the distribution 304 can indicate where light is projected for a possible point location. In some example implementations, the transmitter 302 can include one or more light sources 324 (such as one or more lasers), a lens 326, and a light modulator 328. The transmitter 302 can also include an aperture 322 through which the transmitted light escapes from the transmitter 302. In some implementations, the transmitter 302 can further include a diffractive optical element (DOE) to diffract an emission from the one or more light sources 324 into additional emissions. In some aspects, the light modulator 328 can include, for example, a DOE to adjust the intensity of the emissions. In projecting the distribution 304 of light points onto the scene 306, the transmitter 302 can send one or more lasers from the light sources 324, through the lens 326 (and / or through the DOE or light modulator 328), and onto the scene 306. The transmitter 302 can be located on the same reference plane as the receiver 308, and the transmitter 302 and the receiver 308 can be separated by a distance, which is referred to as the baseline (312).

[0060] In some example implementations, the light projected by the transmitter 302 can be infrared (IR) light. The IR light can include a portion of the visible spectrum and / or a portion of the spectrum that is not visible to the human eye. In one example, the IR light can include NIR light (which can or can not include light within the visible spectrum) and / or IR light outside of the visible spectrum, such as far infrared (FIR) light. The term IR light should not be limited to light having a particular wavelength that is within or near the wavelength range of IR light. Further, IR light is provided as an example emission from a transmitter. In the following description, other appropriate wavelengths of light can be used, e.g., light outside of the IR light wavelength range in a portion of the visible spectrum or ultraviolet light. Alternatively, other signals having different wavelengths can be used, such as microwaves, radio frequency signals, and other appropriate signals.

[0061] The scene 306 can include objects at different depths from the SL system, such as from the transmitter 302 and the receiver 308. For example, objects 306A and 306B in the scene 306 can be at different depths. The receiver 308 can be configured to receive reflections 310 of the emitted distribution 304 of light points from the scene 306. To receive the reflections 310, the receiver 308 can capture an image. When capturing the image, the receiver 308 can receive the reflections 310, as well as (i) other reflections of the distribution 304 of light points from other portions of the scene 306 at different depths, and (ii) ambient light. Noise can also be present in the captured image.

[0062] In some example implementations, the receiver 308 can include a lens 330 to focus or direct the received light, including the reflections 310 from the objects 306A and 306B, onto a sensor 332 of the receiver 308. The receiver 308 can also include an aperture 320. Assuming only the reflections 310 are received for the example, the depths of the objects 306A and 306B can be determined based on the baseline 312, the displacement and distortion of the distribution 304 of light, such as a codeword, in the reflections 310, and the intensity of the reflections 310. For example, a distance 334 along the sensor 332 from a location 316 to a center 314 can be used to determine the depth of the object 306B in the scene 306. Similarly, a distance 336 along the sensor 332 from a location 318 to the center 314 can be used to determine the depth of the object 306A in the scene 306. The distances along the sensor 332 can be measured in terms of a number of pixels of the sensor 332 or a distance, such as millimeters.

[0063] In some example implementations, the sensor 332 can include an array of photodiodes (such as avalanche photodiodes) for capturing images. To capture an image, each photodiode in the array can capture light that hits the photodiode and can provide a value (a capture value) indicative of the intensity of the light. Thus, an image can be the capture values provided by the array of photodiodes.

[0064] In addition to or as an alternative to the sensor 330 including an array of photodiodes, the sensor 332 can include a complementary metal-oxide-semiconductor (CMOS) sensor. To capture an image by a photosensitive CMOS sensor, each pixel of the sensor can capture light that hits the pixel and can provide a value indicative of the intensity of the light. In some example implementations, the array of photodiodes can be coupled to the CMOS sensor. In this way, an electrical pulse generated by a photodiode of the array of photodiodes can trigger a corresponding pixel of the CMOS sensor to provide a capture value.

[0065] The sensor 332 can include at least a number of pixels that is equal to the number of possible light points in the distribution 304. For example, the array of photodiodes or the CMOS sensor can include a number of photodiodes or a number of pixels, respectively, that corresponds to the number of possible light points in the distribution 304. The sensor 332 can be logically divided into groups of pixels or groups of photodiodes (such as 4x4 groups) that correspond to the size of the bits of the codewords. A group of pixels or photodiodes can also be referred to as a bit, and a portion in an image captured from a bit of the sensor 332 can also be referred to as a bit. In some example implementations, the sensor 332 can include the same number of bits as the distribution 304.

[0066] If the light source 324 transmits IR light (such as NIR light at a wavelength of, for example, 940 nm), the sensor 332 can be an IR sensor to receive reflections of the NIR light. As shown, the distance 334 (corresponding to the reflection 310 from the object 306B) is less than the distance 336 (corresponding to the reflection 310 from the object 306A). Using triangulation based on the baseline 312 and the distances 334 and 336, the different depths of the objects 306A and 306B in the scene 306 can be determined in generating depth information for the scene 306. Determining the depths can further include determining a displacement or distortion of the distribution 304 in the reflection 310.

[0067] Although in the example implementation shown in FIG. 3, the sensor 332 is an IR sensor, in other example implementations, the sensor 332 can be a visible light sensor, a UV light sensor, or a combination thereof. Figure 3Several individual components are shown, but one or more of these components may be implemented together or include additional functionality. Not all described components may be required for the SL system 300, or the functionality of a component may be separated into individual components. Additional components, not shown, may also be present. For example, receiver 308 may include a bandpass filter to allow signals with a defined wavelength range to pass to sensor 332 (thus filtering out signals with wavelengths outside that range). In this way, some incidental signals (such as ambient light) can be prevented from interfering with the capture performed by sensor 332. The range of the bandpass filter may be centered on the transmission wavelength used for transmitter 302. For example, if transmitter 302 is configured to transmit NIR light with a wavelength of 940 nm, receiver 308 may include a bandpass filter configured to allow NIR light with wavelengths in, for example, the range of 920 nm to 960 nm. Therefore, regarding... Figure 3 The examples described are for illustrative purposes, and this disclosure should not be limited to the example SL system 300.

[0068] For the optical emitter (such as transmitter 302), the light source can be any suitable light source. In some example implementations, light source 324 may include one or more distributed feedback (DFB) lasers. In some other example implementations, light source 324 may include one or more vertical-cavity surface-emitting lasers (VCSELs).

[0069] A Distributed Object (DOE) is a material located in the projection path of light from a light source. A DOE can be configured to split a light spot into multiple light spots. For example, the material of the DOE can be a translucent or transparent polymer with a known refractive index. The surface of the DOE can include peaks and valleys (changing the depth of the DOE) such that as light passes through the DOE, the light spot is split into multiple light spots. For example, a DOE can be configured to receive one or more light spots from one or more lasers, and to project a desired distribution having a larger number of light spots than those emitted by one or more lasers. While the accompanying drawings may show that the depth of the DOE varies only along one axis of the DOE, the drawings are only for illustrating various aspects of this disclosure. Peaks and valleys on the surface of the DOE can be located at any part of the surface of the DOE, and any suitable variation in the depth of the various parts of the DOE can occur, and this disclosure should not be limited to a specific surface configuration for the DOE.

[0070] Figure 4A block diagram of an example device 400 configured for active depth sensing using ToF and SL techniques is shown. It will be understood that ToF and SL are example active depth techniques, and that device 400 can use other active depth techniques in some implementations. In some embodiments, device 400 can be configured to generate depth information using ToF techniques while using SL techniques to mitigate effects of MPI in the depth information. Device 400 can include or be coupled to a transmitter 401 (“first transmitter”), a sensor 402, a processor 404, a memory 406 storing instructions 408, and an active depth controller 410 (which can include one or more signal processors 412). Transmitter 401 can include or be coupled to a DOE 405. DOE 405 can optionally be included in or coupled to device 400. Transmitter 401 can include or be coupled to a diffuser 407. Diffuser 407 can optionally be included in or coupled to device 400.

[0071] Device 400 can also include or be coupled to a transmitter 421 (“second transmitter”). In some implementations, transmitter 421 can be the same as or similar to transmitter 401. Transmitter 421 can include or be coupled to a DOE 425, which can be the same as or similar to DOE 405. DOE 425 can optionally be included in or coupled to device 400. Transmitter 421 can include or be coupled to a diffuser 427, which can be the same as or similar to diffuser 407. Diffuser 427 can optionally be included in or coupled to device 400. Aspects described in this disclosure with respect to one or more of transmitter 401, DOE 405, or diffuser 407 can also apply to one or more of transmitter 421, DOE 425, or diffuser 427, respectively.

[0072] A reflective component 431 can be located between transmitter 401 and transmitter 421. Reflective component 431 can optionally be included in or coupled to one or more of device 400, transmitter 401, or transmitter 421. In some implementations, reflective component 431 can be configured to redirect light projected by transmitter 401 onto a first portion of a scene and to redirect light projected by transmitter 421 onto a second portion of the scene, as described in further detail with respect to FIG. 4B. Figure 6A and 6B are further described.

[0073] For the purposes of the discussion herein, the device 400 can be referred to as a “ToF and SL system.” Further, for the purposes of the discussion herein, the “ToF and SL system” can alternatively refer to one or more components of the device 400 (e.g., the active depth controller 410, the emitter 401, the sensor 402, the DOE 405, the diffuser 407, the emitter 421, the DOE 425, the diffuser 427, or one or more of the reflective components 431) or any other component that can be used for active depth sensing.

[0074] In some embodiments, one or more of the emitters 401 or 421 can be a single hybrid laser projector that is capable of switching between projecting a first light distribution (e.g., in the case that one or more of the diffusers 407 or 427 are used) during a first projection mode (e.g., a ToF projection mode) of one or more of the emitters 401 or 421 and projecting a second light distribution (e.g., in the case that one or more of the DOEs 405 or 425 are used) during a second projection mode (e.g., an SL projection mode) of one or more of the emitters 401 or 421. When operating in the SL projection mode, the DOEs 405 and 425 can enable the emitters 401 and 421, respectively, to send the second light distribution, which can be, for example, a known DOE dot pattern, a codeword DOE projection, etc. The diffusers 407 and 427 can be switchable such that the diffusers are “off’ (or “disabled” or “cut off”) when the device 400 is operating in the SL projection mode and the diffusers are “on” (or “enabled” or “turned on”) when the device 400 is operating in the ToF projection mode.

[0075] More specifically, when operating in the ToF projection mode, the respective one of the diffusers 407 and 427 is turned on, which enables the respective one of the emitters 401 and 421 to send the second light distribution (e.g., a flood distribution). Thus, the emitters 401 and 421 can be synchronized to project the second light distribution (e.g., a DOE distribution) during the SL projection mode and to project the second light distribution (e.g., a full flood frame) during the ToF projection mode. In some aspects, one or more of the light distributions can be time modulated, as described with respect to Figure 1 In some embodiments, one or more of the emitters 401 and 421 can include multiple projectors.

[0076] In some embodiments, the sensor 402 can be a single hybrid ToF and SL sensor that is used to receive reflected light according to ToF and SL sensing (or “readout”) modes. The sensor 402 can be configured to switch between operating in a first sensing mode (e.g., a ToF sensing mode) and a second sensing mode (e.g., an SL sensing mode). For example, the sensor 402 can be a composite CMOS image sensor that is configured to switch (or alternate) between operating in a ToF sensing mode and operating in an SL sensing mode. The sensing mode can depend on which distribution (e.g., a DOE or flood) the respective emitter of the emitters 401 and 421 is projecting. In some aspects, the sensor 402 can be based on a monolithic pixel array architecture, e.g., with time-division multiplexed readout (TDMR) capability. In other embodiments, the sensor 402 can include one or more general purpose ToF sensors that operate in conjunction with multiple projectors.

[0077] In some embodiments, the active depth controller 410 can be a computational element for computing depth information. The active depth controller 410 can be configured to alternate between computing depth information using ToF techniques and computing depth information using SL techniques. For purposes of discussion herein, depth information computed using SL techniques can also be referred to as “SL depth information,” “SL information,” and the like. Similarly, for purposes of discussion herein, depth information computed using ToF techniques can also be referred to as “ToF depth information,” “ToF information,” and the like. In some aspects, the active depth controller 410 can use SL depth information as a reference for computing or supplementing ToF depth information, which can help to compensate for MPI errors in the ToF depth information. That is, the active depth controller 410 can use sparse depth information from the SL mode as a baseline reference to compensate for multipath effects in the depth information from the ToF mode. In this way, the active depth controller 410 can generate high resolution and high accuracy depth information that does not have MPI artifacts. In some embodiments, the sensor 402 can be a reconfigurable instruction cell array (RICA), which is a proprietary, real-time, low-power, (re)programmable image signal processing (ISP), active sensing, processing engine. In some aspects, overlaying the hybrid NIR sensor described herein with a RICA programmable implementation can enable the active depth controller 410 to switch programming on-the-fly to switch computing SL depth information and ToF depth information while reducing the number of components for the sensor (e.g., the sensor 402). In other embodiments, the active depth controller 410 can be a general purpose sensor.

[0078] In some aspects, the active depth controller 410 can be configured to control (or otherwise operate) one or more of the emitter 401, the emitter 421, or the sensor 402 to synchronize their respective modes of operation such that the sensor 402 and the emitters 401 and 421 are simultaneously operating in their respective SL mode or ToF mode. In some aspects, the active depth controller 410 can be controlled by, work in conjunction with, or otherwise be operated by one or more other components of the device 400, such as at least one of the processor 404 or the memory 406.

[0079] The device 400 can optionally include or be coupled to a display 414 and a plurality of input / output (I / O) components 416. The sensor 402 can be or can otherwise be coupled to a camera, such as a single camera, a dual camera module, or a module with any number of other camera sensors (not shown). The signal processor 412 can be configured to process captures from the sensor 402. The device 400 can further include one or more optional sensors 420 coupled to the processor 404, such as a gyroscope, a magnetometer, an inertial sensor, a NIR sensor, etc. The device 400 can also include a power source 418, which can be coupled to or integrated into the device 400. The device 400 can include additional features or components not shown.

[0080] The memory 406 can be a non-transitory or non-transient computer-readable medium that stores computer-executable instructions 408 for performing all or part of one or more operations described in the present disclosure. The processor 404 can be one or more appropriate processors capable of executing scripts or instructions of one or more software programs stored within the memory 406, such as the instructions 408. In some aspects, the processor 404 can be one or more general purpose processors that execute the instructions 408 to cause the device 400 to perform any number of functions or operations. In additional or alternative aspects, the processor 404 can include integrated circuits or other hardware to perform functions or operations without the use of software. While the Figure 4 The processor 404, the memory 406, the active depth controller 410, the optional display 414, the optional I / O components 416, and the optional sensors 420 are shown in the example of FIG. 4 as being coupled to one another via the processor 404, but can be coupled to one another in various arrangements. For example, the processor 404, the memory 406, the active depth controller 410, the optional display 414, the optional I / O components 416, and / or the optional sensors 420 can be coupled to one another via one or more local buses (not shown for simplicity).

[0081] The display 414 can be any suitable display or screen that contemplates user interaction and / or presents items for viewing by a user, such as depth information or preview images of a scene. In some aspects, the display 414 can be a touch-sensitive display. The I / O components 416 can be or include any suitable mechanism, interface, or device for receiving input from a user, such as commands, and providing output to the user. For example, the I / O components 416 can include, but are not limited to, a graphical user interface, a keyboard, a mouse, a microphone and a speaker, a compressible bezel or border of the device 400, a physical button located on the device 400, and the like. The display 414 and / or the I / O components 416 can provide a user with preview images or depth information for a scene, and / or receive user input for adjusting one or more settings of the device 400, such as adjusting the intensity of the emission of one or more of the emitters 401 or 421, determining or switching one or more modes of operation of the device 400, adjusting the emission field of one or more of the emitters 401 or 421, and the like.

[0082] The active depth controller 410 can also include or can otherwise be coupled to a signal processor 412, which can be one or more processors to process captures from the sensors 402. The active depth controller 410 can be configured to switch at least one of the emitters 401, 421, or the sensors 402 between one or more modes of operation. The active depth controller 410 can alternatively or additionally include a combination of specific hardware and the ability to execute software instructions.

[0083] One or more of the emitters 401 or 421 can vary its emission field for different modes of operation. In some example implementations, one or more of the emitters 401 or 421 can include a focusing device to adjust the size of the emission / transmission field. In one example, a mirror attached to an actuator, such as a microelectromechanical system (MEMS) actuator, can adjust the focal point of the light emission from a respective one of the emitters 401 and 421. In another example, an adjustable holographic optical element (HOE) can adjust the focal point of the light emission from a respective one of the emitters 401 and 421. In a further example, a shapeable DOE, such as a piezoelectric material for adjusting the shape, can be adjusted to focus the diffracted emitted light points.

[0084] In some other example implementations, device 400 can use multiple light emitters (not shown) to emit light instead of or in combination with emitters 401 and 421. The emitters can include a first set of light emitters (e.g., of a first array of light emitters) to emit light having a first propagation field. The emitters can also include a second or different set of light emitters (e.g., of a second array of light emitters) to emit light having a second propagation field. At a common depth from one or more of emitters 401 or 421, the first field can be greater than the second field. In some example implementations, the first set of light emitters can be activated for a first mode of a respective one of emitters 401 and 421, and the second set of light emitters can be activated for a second mode of the respective one of emitters 401 and 421.

[0085] Figure 5 A timing diagram 500 is shown that illustrates example operations of a ToF and SL system including sensors 509, emitters 549, and controller 579. Sensors 509, emitters 549, and controller 579 can be example embodiments of sensors 402, emitters 401 and 421, and active depth controller 410, respectively, of Figure 4 It will be understood that ToF and SL are example active depth techniques, and that the system can use other active depth techniques in some implementations.

[0086] Example timing diagram 500 shows three projection cycles for emitters 549: a first projection cycle ending at time 501, a second projection cycle ending at time 502, and a third projection cycle ending at time 503. Each of emitters 549 can project a first light distribution during each of these projection cycles. The first light distribution can be a flood distribution for a first projection mode, such as a ToF projection mode. For example, each of emitters 549 can project a flood distribution 550, a flood distribution 552, and a flood distribution 554 during the ToF projection mode for each of the first, second, and third projection cycles, respectively. For purposes of discussion herein, a flood distribution can also be referred to as “flood illumination” or “diffuse light.” In some aspects, one or more of these flood distributions can be time modulated, as discussed with respect to Figure 1 and 4Each of the emitters 549 can also project a second light distribution during each of the projection cycles. The second light distribution can be a DOE distribution for a second projection mode, such as an SL projection mode. For example, each of the emitters 549 can project the DOE distribution 570, the DOE distribution 572, and the DOE distribution 574 during the SL projection mode for each of the first, second, and third projection cycles, respectively. For purposes of discussion herein, the DOE distributions can also be referred to as “DOE patterns,” “DOE projections,” “SL distributions,” “SL patterns,” and / or “SL projections.”

[0087] The example timing diagram 500 illustrates three sensing cycles for the sensor 509: a first sensing cycle ending at time 501, a second sensing cycle ending at time 502, and a third sensing cycle ending at time 503. The sensor 509 can read out two frames of ToF sensor data (during ToF sensing mode) and one frame of SL sensor data (during SL sensing mode) for each sensing cycle. The sensor 509 can be configured to operate in ToF sensing mode while the emitters 549 are configured to operate in ToF projection mode. The sensor 509 can be configured to operate in SL sensing mode while the emitters 549 are configured to operate in SL projection mode. During ToF sensing mode, the emitters 549 can emit laser pulses, and the sensor 509 can be exposed with a plurality of specific laser pulse phase offsets (e.g., phase 0°, phase 180°, phase 90°, and phase 270°) relative to respective pulse signals from each of the emitters 549. The sensor 509 can accumulate and store a charge quantity (Q) for each of the specific laser pulse phase offsets (or “phase offsets”).

[0088] For example, during a first exposure, the sensor 509 can read out ToF sensor data 510 for a first frame based on Ql and Q2, where Ql is charge accumulated at a 0° phase offset, and where Q2 is charge accumulated at a 180° phase offset. During a second exposure, the sensor 509 can read out ToF sensor data 512 for a second frame based on Q3 and Q4, where Q3 is charge accumulated at a 90° phase offset, and where Q4 is charge accumulated at a 270° phase offset. Similarly, the sensor 509 can read out ToF sensor data 514 for a first frame and ToF sensor data 516 for a second frame during a second sensing cycle, and the sensor 509 can read out ToF sensor data 518 for a first frame and ToF sensor data 520 for a second frame during a third sensing cycle. The sensor 509 can read out SL sensor data 530 for a frame during a first sensing cycle, SL sensor data 532 for a frame during a second sensing cycle, and SL sensor data 534 for a frame during a third sensing cycle.

[0089] After each sensing cycle, the controller 579 can use the SL sensor data to compute SL depth information (Z(SL)). For example, the controller 579 can compute SL depth information (Z(SL)) 580, Z(SL) 582, and Z(SL) 584 after each of the first, second, and third sensing cycles, respectively.

[0090] After computing Z(SL), the controller 579 can use Z(SL) to reduce, filter, and / or eliminate MPI associated with ToF sensor data for a corresponding sensing cycle. For purposes of discussion herein, the improved depth information can be referred to as Z(ToF+SL). For example, the controller 579 can compute Z(ToF+SL) 590, Z(ToF+SL) 592, and Z(ToF+SL) 594 after computing each of Z(SL) 580, Z(SL) 582, and Z(SL) 584, respectively. In some aspects, the controller 579 can compute Z(ToF+SL) using ToF sensor data for one frame of a corresponding sensing cycle, or, due to the susceptibility of ToF sensing technology to noise, the controller 579 can compute Z(ToF+SL) using ToF sensor data for more than one frame of a corresponding sensing cycle. In some implementations, the controller 579 can compute Z(SL) and Z(ToF+SL) using ToF sensor data of a first frame and ToF sensor data of a second frame at any time during a next sensing cycle. As one non-limiting example, the controller 579 can average ToF sensor data 510 of a first frame with ToF sensor data 512 of a second frame to compute Z(SL) 580 and Z(ToF+SL) 590 between time 501 and time 502. In some aspects, the controller can use ToF sensor data frames in different ways to compute Z(SL) and / or Z(ToF+SL).

[0091] In this way, the system can use sparse depth information from the SL mode as a reference to generate high-resolution and high-accuracy depth information without MPI artifacts to eliminate multipath effects from the ToF mode. According to embodiments described herein, the system can use a single sensor (e.g., sensor 509), a pair of emitters (e.g., emitters 549), and / or a single controller (e.g., controller 579) to generate depth information without MPI artifacts.

[0092] Figure 6A An example of a ToF and SL light projection system 600 is shown. The system 600 includes a first emitter 602 and a second emitter 608, which can be respectively Figure 4embodiments of the transmitter 401 and the transmitter 421. The system 600 also includes a reflective assembly 614 positioned between the first transmitter 602 and the second transmitter 608. The reflective assembly 614 redirects light projected by the first transmitter 602 and the second transmitter 608 onto a scene 632, which can include objects (not shown for simplicity). In some implementations, the system 600 can include one or more image capture devices, such as ToF cameras and SL cameras (not shown for simplicity) for detecting reflections of the redirected light. In some aspects, the ToF cameras and the SL cameras can include image sensors, such as Figure 4 The first DOE 604 is coupled to a front of the first transmitter 602, and the first diffuser 606 is coupled to a front of the first DOE 604. Similarly, the second DOE 610 is coupled to a front of the second transmitter 608, and the second diffuser 612 is coupled to a front of the second DOE 610. The first DOE 604, the first diffuser 606, the second DOE 610, and the second diffuser 612 can be example embodiments of the DOE 405, the diffuser 407, the DOE 425, and the diffuser 427, respectively, of the transmitter 401 and the transmitter 421. The first DOE 604, the first diffuser 606, the second DOE 610, and the second diffuser 612 can be example embodiments of the sensor 402 of the transmitter 401 and the transmitter 421. Figure 4 The first DOE 604 is coupled to a front of the first transmitter 602, and the first diffuser 606 is coupled to a front of the first DOE 604. Similarly, the second DOE 610 is coupled to a front of the second transmitter 608, and the second diffuser 612 is coupled to a front of the second DOE 610. The first DOE 604, the first diffuser 606, the second DOE 610, and the second diffuser 612 can be example embodiments of the DOE 405, the diffuser 407, the DOE 425, and the diffuser 427, respectively, of the transmitter 401 and the transmitter 421. The first DOE 604, the first diffuser 606, the second DOE 610, and the second diffuser 612 can be example embodiments of the sensor 402 of the transmitter 401 and the transmitter 421.

[0093] The first light projector 601 projects first light toward the second light projector 609, and the second light projector 609 projects second light toward the first light projector 601. In some implementations, the first projected light 620 is projected in a first direction along an axis (not shown for simplicity), and the second projected light 624 is projected in a second direction opposite the first direction along the axis. In some aspects, the first light projector and the second light projector can be physically separated by a distance, and the reflective assembly 614 can be positioned along an axis between the first light projector and the second light projector.

[0094] The reflection assembly 614 can include a first reflective element 616 that optically folds a first optical path of light projected from the first light projector 601 and a second reflective element 618 that optically folds a second optical path of light projected from the second light projector 609. In some implementations, the first reflective element 616 can be a first prism that includes a first reflective surface 617 to receive light projected from the first light projector 601, and the second reflective element 618 can be a second prism that includes a second reflective surface 619 to receive light projected from the second light projector 609. In some other implementations, one or more portions of the reflection assembly 614 can be glass. The reflection assembly 614 redirects the first projected light 620 and the second projected light 624 to a first portion 634 of the scene and a second portion 636 of the scene, respectively. Specifically, the first reflective element 616 refracts light folded by the first reflective element 616 onto the first portion 634 of the scene based at least in part on the first folded optical path, and the second reflective element 618 refracts light folded by the second reflective element 618 onto the second portion 636 of the scene based at least in part on the second folded optical path.

[0095] In some implementations, the first reflective element 616 and the second reflective element 618 can have the same (or nearly the same) refractive index. Thus, the reflection assembly 614 can symmetrically (or nearly symmetrically) refract light projected by the first light projector 601 and the second light projector 609 onto respective portions of the first portion 634 of the scene and the second portion 636 of the scene. In this way, the first portion 634 of the scene and the second portion 636 of the scene can be adjacent to each other. In some implementations, the first portion and the second portion do not overlap each other. That is, the first portion 634 of the scene and the second portion 636 of the scene can be aligned in such a way that the first portion 634 and the second portion 636 seamlessly cover the scene 632 with a negligible amount of gap, such as less than a first value, and a negligible amount of overlap, such as less than a second value.

[0096] Figure 6AThe example shows four example beams (A, B, C, and D) projected from the first light projector 601 and four example beams (E, F, G, and H) projected from the second light projector 609. For example, the first light projector 601 projects a beam A that enters the first reflective element 616 at point A. Based on the refractive properties of the first reflective element 616, the beam A is optically folded to point A’ on the first reflective surface 617 and exits the first reflective element 616 at a first angle (e.g., 0°) toward the scene 632. Similarly, the second light projector 609 projects a beam E that enters the second reflective element 618 at point E. Based on the refractive properties of the second reflective element 618, the beam E is optically folded to point E’ on the second reflective surface 619 and exits the second reflective element 618 at the first angle toward the scene 632. Since the first reflective element 616 and the second reflective element 618 have the same refractive index, points A and A’ are symmetric to points E and E’, respectively.

[0097] As another example, the first light projector 601 projects a beam B that enters the first reflective element 616 at point B. Based on the refractive properties of the first reflective element 616, the beam B is optically folded to point B’, then to point B”, and exits the first reflective element 616 at a second angle (e.g., greater than 70°) toward the scene 632. In a symmetric manner, the second light projector 609 projects a beam F that enters the second reflective element 618 at point F. Based on the refractive properties of the second reflective element 618, the beam F is optically folded to point F’, then to point F”, and exits the second reflective element 618 at the second angle toward the scene 632.

[0098] Similarly, the first light projector 601 projects a beam C that enters the first reflective element 616 at point C. Based on the refractive properties of the first reflective element 616, the beam C is optically folded to point C’, then to point C”, and exits the first reflective element 616 at a third angle (e.g., greater than 70°) toward the scene 632. In a symmetric manner, the second light projector 609 projects a beam G that enters the second reflective element 618 at point G. Based on the refractive properties of the second reflective element 618, the beam G is optically folded to point G’, then to point G”, and exits the second reflective element 618 at the third angle toward the scene 632.

[0099] Similarly, the first light projector 601 projects a light beam D, which enters the first reflective element 616 at point D. Based on the refractive properties of the first reflective element 616, the light beam D is optically folded to point D', then to point D''', and exits the first reflective element 616 at a third angle (e.g., 0°) toward the scene 632. In a symmetrical manner, the second light projector 609 projects a light beam H, which enters the second reflective element 618 at point H. Based on the refractive properties of the second reflective element 618, the light beam H is optically folded to point H', then to point H'''', and exits the second reflective element 618 at a third angle toward the scene 632.

[0100] Therefore, the reflector 614 symmetrically refracts the light beams AD and EH onto the scene 632, respectively. In this way, the light beams are projected onto the first part 634 and the second part 636 of the scene, and the system 600 has a wide (e.g., greater than 70 degrees) field of view (FOV) of the scene 632.

[0101] Figure 6B An example ToF and SL light projection system 650 is shown operating in a first mode (e.g., ToF mode, as shown on the left) and a second mode (e.g., SL mode, as shown on the right). The ToF and SL system 650 can be... Figure 6A A simplified example embodiment of the ToF and SL system 600. That is, the first light projector 651, the second light projector 659, and the reflector 664 can be respectively... Figure 6A An example embodiment of a first light projector 601, a second light projector 609, and a reflective component 614 is provided. The reflective component 614 can redirect light projected by the first light projector 601 and the second light projector 609 onto a scene 682, which may include objects (not shown for simplicity). In some implementations, system 650 may include one or more image capture devices, such as a ToF camera and an SL camera (not shown for simplicity), for detecting the reflection of the redirected light. In some aspects, the ToF camera and the SL camera may include image sensors, which may be... Figure 4 An example embodiment of sensor 402. System 650 may also include a controller (not shown for simplicity), such as Figure 4 Active depth controller 410.

[0102] In some implementations, each of the first and second diffusers may be an on / off diffuser configured to switch between a first light projector 651 and a second light projector 659, respectively, between a ToF sensing mode and an SL sensing mode. During ToF mode, the first and second emitters can operate in ToF projection mode, and the sensor 402 can operate in ToF sensing mode, as per [reference to...].Figure 5 The first transmitter and the second transmitter can operate in a SL projection mode and the sensor 402 can operate in a SL sensing mode during the SL mode, as also described with respect to Figure 5

[0103] During the ToF mode, the first diffuser and the second diffuser can be turned on, as indicated with the solid gray color. Thus, when the laser of the first light projector 651 emits light through the first DOE, the DOE pattern from the first DOE is diffused as it passes through the first diffuser, which provides flood illumination of the first portion 684 of the scene. Similarly, when the laser of the second light projector 659 emits light through the second DOE, the DOE pattern from the second DOE is diffused as it passes through the second diffuser, which provides flood illumination of the second portion 686 of the scene. Specifically, during the ToF mode, the first transmitter sends a first pulse signal and the first diffuser diffuses the first projected light to project a uniform flood pattern onto the first portion 684 of the scene.

[0104] Similarly, during the ToF mode, the second transmitter sends a second pulse signal and the second diffuser diffuses the second projected light to project a uniform flood pattern onto the second portion 686 of the scene. That is, the reflecting component 664 can refract light projected by each of the first light projector 651 and the second light projector 659 symmetrically (or nearly symmetrically) such that the portion of the scene onto which light is projected from the first light projector 651 (e.g., the first portion of the scene) can be adjacent to the portion of the scene onto which light is projected from the second light projector 659 (e.g., the second portion of the scene), yet not overlapping. In this way, the projection of light from the first light projector 651 and the second light projector 659 (refracted by the reflecting component 664) can be aligned in such a way that the projection of light from the first light projector 651 and the second light projector 659 seamlessly covers the scene 682 with a negligible amount of gap, such as less than a first value, and a negligible amount of overlap, such as less than a second value. Referring back to Figure 4 , the first reflected signal and the second reflected signal of the first pulse signal and the second pulse signal, respectively, can reach the sensor 402 and the sensor 402 captures a frame that includes the pulse signals. The active depth controller 410, such as the signal processor 412, can calculate ToF depth information based on the captured frame by determining the amount of time for light to be reflected back to the sensor 402.

[0105] ​During the SL mode, the first diffuser and the second diffuser can be turned off (as indicated with white). In some implementations, the first diffuser and the second diffuser act as transparent. Thus, when the lasers of the first light projector 651 emit light through the first DOE during the SL mode, the DOE distribution from the first DOE passes through the first diffuser unaffected, which projects the first projected light onto the first portion 684 of the scene (e.g., as a first distribution of light points, such as a dot pattern). Similarly, when the lasers of the second light projector 659 emit light through the second DOE during the SL mode, the DOE distribution from the second DOE passes through the second diffuser unaffected, which projects the second projected light onto the second portion 686 of the scene (e.g., a second distribution of light points, such as a dot pattern). In some implementations, the first distribution of light points and the second distribution of light points can be the same or similar.

[0106] Referring back to Figure 4 , the sensor 402 can capture a frame that includes reflections of the redirected light (where the light is reflected by objects in the scene 632). The active depth controller 410 (such as the signal processor 412) can detect the redirected light projected by the first light projector 651 and the second light projector 659 in the captured frame as first reflected light and second reflected light, respectively. The active depth controller 410 (such as the signal processor 412) can then compute SL depth information based on a manner in which the first projected light and the second projected light are distorted on the first portion 684 of the scene and the second portion 686 of the scene, respectively.

[0107] The active depth controller 410 can be configured to generate depth information based on the detected reflections of the redirected light and correct the generated depth information for projection distortion based at least in part on one or more angles of refraction associated with the reflection assembly 664. The controller 410 can use the generated SL depth information to reduce or eliminate multipath artifacts in ToF depth information during computation of the ToF depth information.

[0108] For example, projecting light through a prism at an angle instead of directly at the scene 632 can cause a certain level (or value) of distortion or skew in the resulting SL depth information. For the purposes of discussion herein, the distortion or skew can be referred to as “projection distortion.” Furthermore, the redirection of the first projected light 620 and the second projected light 624 by the reflection assembly 614 can take more time compared to if the first projected light 620 and the second projected light 624 were projected directly toward the scene 632. Thus, ToF depth information generated based on light redirected by the reflection assembly 614 can also include a certain level (or value) of projection distortion.

[0109] In some aspects of the disclosure, the system 600 can be configured to offset or compensate for projection distortion. In some implementations, the reflective component 614 can distort the projected light in a known manner. Accordingly, at least one of the first light projector 601 or the second light projector 609 can include a pre-distorted pattern to offset the projection distortion. That is, the pre-distorted pattern can be incorporated into at least one of the first DOE 604 or the second DOE 610 such that the respective DOE pattern offsets (or “cancels”) the known projection distortion. Additionally or alternatively, the system 600 can be configured to digitally compensate for the projection distortion after detecting the reflection of the redirected light (“post-capture”), such as by using Figure 4 the active depth controller 410.

[0110] In some implementations, the system 650 can be included in or coupled to a device, such as a camera or a cellular phone. With reference to Figure 4 , in the case of the device 400 as an example device, the active depth controller 410 can be configured to identify a face of a user of the device based at least in part on the generated depth information. In some aspects, the controller 410 can also be configured to resolve multipath interference (MPI) in a first portion of the generated depth information based at least in part on a second portion of the generated depth information.

[0111] Figure 7 An example circuit diagram for a demodulation pixel cell 700 is shown. In some implementations, the demodulation pixel cell 700 can be included in or coupled to a sensor, such as the sensor 402 of Figure 4 . The demodulation pixel cell 700 can be used to generate ToF depth information. The demodulation pixel cell 700 can include a photodiode 720 coupled to a ground potential 710. The photodiode 720 can convert light (e.g., photons) from a reflected signal into a current that flows to a transistor 730 and a transistor 760 coupled in parallel to the photodiode 720. The transistor 730 and the transistor 760 can block the current from flowing to a capacitor (Cl) and a capacitor (C2), respectively. Cl can be coupled to a ground potential 750, and C2 can be coupled to a ground potential 780. In some aspects, at least one of the transistor 730 or the transistor 760 can be a field effect transistor (FET). In some aspects, at least one of the transistor 730 and the transistor 760 can be a metal-oxide-semiconductor field effect transistor (MOSFET).

[0112] For example, during a first exposure cycle, when the shutter is opened at a first phase offset (e.g., Φ1=0°) relative to the transmitted signal, C1 can store a first charge (Q1) from the reflected signal, and when the shutter is opened at a second phase offset (e.g., Φ2=180°) relative to the transmitted signal, C2 can store a second charge (Q2) from the reflected signal. During a second exposure cycle, when the shutter is opened at a third phase offset (e.g., Φ1=90°) relative to the transmitted signal, C1 can store a third charge (Q3) from the reflected signal, and when the shutter is opened at a fourth phase offset (e.g., Φ2=270°) relative to the transmitted signal, C2 can store a fourth charge (Q4) from the reflected signal. The phase offsets between the transmitted signal and the reflected signal may be calculated based on the charges stored on C1 and C2 for each of the exposure cycles, which allows the corresponding ToF depth information to be calculated:

[0113]

[0114]

[0115]

[0116] where D represents the depth information, c represents the speed of light (i.e. ), f mod represents the modulation frequency of the transmitted signal, V0-V 180 represents the integrated electrical signal for Φ1 and Φ2 during the first exposure cycle, V 90 -V 270 represents the integrated electrical signal for Φ1 and Φ2 during the second exposure cycle, and σ depth represents the depth accuracy. Thus, the demodulation pixel cell 700 can capture ToF sensor data for generating ToF depth information.

[0117] Figure 8AAn example circuit diagram is shown for a global shutter (GS) pixel array 800. The GS pixel array 800 can also be referred to herein as a NIR-GS imager. The GS pixel array 800 includes two shared GS photodiodes PD1 and PD2. Each of PD1 and PD2 can absorb photons (e.g., from light reflected back from a scene and / or object) during an SL sensing mode. Each of PD1 and PD2 is coupled to a floating storage diode SD1 and SD2, respectively. SD1 and SD2 can operate as storage node elements for charge accumulation and readout from the photodiodes PD1 and PD2. Each of the storage diodes SD1 and SD2 is coupled to a transfer gate TG1 and TG2, respectively. TG1 and TG2 can be transistors with a relatively low voltage drop. Charge from PD1 and PD2 can flow to a transistor LOD1 and a transistor LOD2, respectively, which are each coupled to a power supply voltage Vddpix 865.

[0118] The GS pixel array 800 includes a capacitor FD for accumulating charge. The capacitor FD is coupled to a transistor TS1, which is coupled to, for example, the storage diode SD1. The capacitor FD is also coupled to a transistor TS2, which is coupled to, for example, the storage diode SD2. The capacitor FD is further coupled to a reset switch RST. When RST is closed, charge can flow to Vddpix 875. When RST is open, charge can flow to a source follower amplifier SF AMP. Because the source voltage of SF AMP remains proportional to the gate voltage, SF AMP can convert charge to voltage, and a select switch SEL is toggled. When SEL is open (e.g., during an SL mode, when TS1 and TS2 are open), Vddpix 875 can be isolated, and a relatively small amount of charge from each signal pulse in a signal pulse sequence (e.g., a pulse train) can accumulate on the capacitor FD. When SEL is closed (e.g., after each of the SL modes, when TS1 and TS2 are closed), the accumulated signal pulse sequence can be transferred from each of TG1 and TG2 to an output terminal Vout. Vout can be coupled to a current source I bias. Thus, the GS pixel array 800 can capture SL sensor data for generating SL depth information.

[0119] Figure 8BAn example circuit diagram is shown for a GS pixel array 850. The GS pixel array 850 can also be referred to herein as a NIR-GS imager. The GS pixel array 850 includes two shared GS photodiodes PD1 and PD2. Each of PD1 and PD2 can absorb photons (e.g., from light reflected back from a scene and / or object) during an SL sensing mode. Each of PD1 and PD2 is coupled to a CCD readout memory MEM1 and MEM2, respectively. MEM1 and MEM2 can operate as storage node elements for charge accumulation and readout from the photodiodes PD1 and PD2. Each of MEM1 and MEM2 is coupled to a transfer gate TG1 and TG2, respectively. TG1 and TG2 can be transistors with a relatively low voltage drop. Charge from PD1 and PD2 can flow to a transistor LOD1 and a transistor LOD2, respectively, which are each coupled to a power supply voltage Vddpix 868.

[0120] The GS pixel array 850 includes a capacitor FD for accumulating charge. The capacitor FD is coupled to a transistor TS1, which is coupled to, for example, a CCD readout memory MEM1. The capacitor FD is also coupled to a transistor TS2, which is coupled to, for example, a CCD readout memory MEM2. The capacitor FD is further coupled to a reset switch RST. When RST is closed, charge can flow to Vddpix 878. When RST is open, charge can flow to a source follower amplifier SF AMP. Since the source voltage of SF AMP is proportional to the gate voltage, SF AMP can convert charge to voltage, and a select switch SEL is toggled. When SEL is open (e.g., during an SL mode, when TS1 and TS2 are open), Vddpix 878 can be isolated, and a relatively small amount of charge from each signal pulse in a signal pulse train (e.g., a pulse string) can accumulate on the capacitor FD. When SEL is closed (e.g., after each of the SL modes, when TS1 and TS2 are closed), the accumulated signal pulse train can be transferred from each of TG1 and TG2 to an output terminal Vout. Vout can be coupled to a current source I_bias. Thus, the GS pixel array 850 can capture SL sensor data for generating SL depth information.

[0121] Figure 9 An example circuit diagram is shown for a rolling shutter (RS) pixel array 900. The pixel array 900 can also be referred to herein as a hybrid NIR-RS imager, and is capable of operating in a ToF sensing mode and an SL sensing mode. The RS pixel array 900 can be a Figure 4FIG. 9 illustrates an example embodiment of the sensor 402. The pixel array 900 can be configured to read out signals row-by-row and thus operate in a constant wave pattern (e.g., at a particular duty cycle) in order to expose each row of the RS for an equal amount of time. The RS pixel array 900 includes four shared RS photodiodes PD1-PD4. Each of PD1-PD4 can absorb photons (e.g., from light reflected back from a scene and / or object) during the SL sensing mode. PD1 is coupled to two transfer gates TG1 and TG2. PD2 is coupled to two transfer gates TG3 and TG4. PD3 is coupled to two transfer gates TG5 and TG6. PD4 is coupled to two transfer gates TG7 and TG8. Each of TG1-TG8 can be a transistor with a relatively low voltage drop.

[0122] The RS pixel array 900 includes capacitors FD1 and FD2 for accumulating charge from the reflected signal. Each of FD1 and FD2 is coupled to a reset switch RST1 and RS2, respectively. When either of RST1 and RS2 is closed, charge can flow to Vddpix 965 and Vddpix 975, respectively. When either of RST1 and RS2 is open, charge can flow to source follower amplifiers SF AMP1 and SF AMP2, respectively. Because the source voltage of SF AMP1 and SF AMP2 remains proportional to the gate voltage, SF AMP1 and SF AMP2 can convert the charge to a voltage and switch corresponding select switches SEL1 and SEL2, respectively.

[0123] During the ToF sensing mode, each of TG1-TG8 can be closed (activated), and the pixel array 900 can demodulate multiple phases of the reflected signal. SEL1 and SEL2 can be open during the ToF sensing mode, which can isolate Vddpix 965 and Vddpix 975, allowing a relatively small amount of charge from each signal pulse in a signal pulse sequence (e.g., a burst) to accumulate on FD1 and FD2. When SEL1 is closed, the accumulated signal pulse sequence can be transferred from TG1, TG3, TG5, and TG7 to an output terminal Vout1. When SEL2 is closed, the accumulated signal pulse sequence can be transferred from TG2, TG4, TG6, and TG8 to an output terminal Vout2. Vout1 can be coupled to a current source I bias1, and Vout2 can be coupled to a current source I bias2. Thus, the RS pixel array 900 can capture ToF sensor data for generating ToF depth information.

[0124] During the SL sensing mode, each of TG1, TG4, TG5, and TG8 (e.g., one half of the readout circuit) can be closed, while each of TG2, TG3, TG6, and TG7 (e.g., the other half of the readout circuit) can be open. In this way, the reflected signal can be captured at different time frames in a bi-phase (e.g., one on the left side and one on the right side). SEL1 and SEL2 can also be open during the SL sensing mode, which can isolate Vddpix 965 and Vddpix 975, allowing a relatively small amount of charge from each signal pulse in the signal pulse sequence (e.g., burst) to accumulate on FD1 and FD2.

[0125] In this way, the pixel array 900 can operate as a hybrid RS sensor for a hybrid mode ToF and SL system that uses sparse depth information from the SL mode as a reference to cancel multi-path effects from the ToF mode, generating high resolution and high accuracy depth information without MPI artifacts.

[0126] Figure 10 A flow diagram illustrating an example process 1000 for depth sensing that can be performed by a device in accordance with some implementations is shown. The process 1000 can be performed by a device such as the device 400 described above with reference to Figure 4 At block 1002, the device projects light from a first light projector of the device toward a second light projector of the device. At block 1004, the device projects light from the second light projector toward the first light projector. At block 1006, the device redirects the light projected by the first light projector onto a first portion of a scene and the light projected by the second light projector onto a second portion of the scene via a reflective component positioned between the first light projector and the second light projector, the first and second portions of the scene being adjacent to each other and non-overlapping relative to each other. At block 1008, the device detects reflections of the redirected light projected by the first and second light projectors.

[0127] Figure 11A A flow diagram illustrating an example process 1100 for depth sensing that can be performed by a device in accordance with some embodiments is shown. The process 1100 can be performed by a device such as the device 400 described above with reference to Figure 4 In some implementations, the process 1100 begins after the process 1000 described with reference to Figure 10 For example, after detecting reflections of the redirected light projected by the first and second light projectors in block 1008 of the process 1000, at block 1102, the device generates depth information based on the detected reflections of the redirected light, as described with reference to Figure 6Bdescribed. At block 1104, the device corrects the projection distortion in the generated depth information based at least in part on one or more refraction angles associated with the reflective component, as described with respect to Figure 6B described.

[0128] Figure 11B A flow diagram illustrating an example process 1110 for depth sensing that can be performed by a device in accordance with some embodiments is shown. The process 1110 can be performed by a device such as the device 400 described above with reference to Figure 4 described. In some implementations, the process 1110 begins after the process 1100 described with reference to Figure 11A described. For example, after correcting the projection distortion in block 1104 of the process 1100, at block 1112, the device identifies a face of a user of the device based at least in part on the generated depth information, as described with respect to Figure 6B described.

[0129] Figure 11C A flow diagram illustrating an example process 1120 for depth sensing performed by a device in accordance with some implementations is shown. The process 1120 can be performed by a device such as the device 400 described above with reference to Figure 4 described. In some implementations, the process 1120 begins after the depth information is generated in block 1102 of the process 1100. At block 1122, the device resolves multipath interference (MPI) in a first portion of the generated depth information based at least in part on a second portion of the generated depth information, as described with respect to Figure 5 and Figure 6B described.

[0130] Figure 12 A flow diagram illustrating an example process 1200 for depth sensing that can be performed by a device in accordance with some implementations is shown. The process 1200 can be performed by a device such as the device 400 described above with reference to Figure 4 described. In various aspects, Figure 12 the example process 1200 can be an implementation for redirecting light in block 1006 of the process 1000 described above with reference to Figure 10 described. At block 1202, the device refracts light projected by the first light projector and the second light projector symmetrically onto respective portions of the first portion and the second portion of the scene, as described with respect to Figure 6A described.

[0131] Figure 13A A flow diagram illustrating an example process 1300 for depth sensing that can be performed by a device in accordance with some implementations is shown. The process 1300 can be performed by a device such as the device 400 described above with reference to Figure 4 described. In various aspects, the example process 1300 of FIG. 13 can be an implementation for redirecting light in block 1006 of the process 1000 described above with reference toFigure 10 One implementation of redirecting light in block 1006 of process 1000. At block 1302, the device optically folds a first optical path of light projected from a first light projector via a first reflective element of the reflective assembly, as described with respect to Figure 6A At block 1304, the device optically folds a second optical path of light projected from a second light projector via a second reflective element of the reflective assembly, again as described with respect to Figure 6A

[0132] Figure 13B A flow diagram illustrating an example process 1310 that can be performed by a device for depth sensing, according to some implementations, is shown. Process 1310 can be performed by a device such as device 400 described above with reference to Figure 4 In some implementations, process 1310 begins after the first and second optical paths are optically folded in blocks 1302 and 1304 of process 1300 described with reference to Figure 13A At block 1312, the device refracts the light folded by the first reflective element onto a first portion of the scene based at least in part on the first folded optical path, as described with respect to Figure 6A At block 1314, the device refracts the light folded by the second reflective element onto a second portion of the scene based at least in part on the second folded optical path, again as described with respect to Figure 6A

[0133] Figure 13C A flow diagram illustrating an example process 1320 that can be performed by a device for depth sensing, according to some implementations, is shown. Process 1320 can be performed by a device such as device 400 described above with reference to Figure 4 In some implementations, process 1320 begins before the first and second optical paths are optically folded in blocks 1302 and 1304 of process 1300 described with reference to Figure 13A At block 1322, the device receives light projected from a first light projector via a first reflective face of a first prism of the first reflective element, as described with respect to Figure 6A At block 1324, the device receives light projected from a second light projector via a second reflective face of a second prism of the second reflective element, again as described with respect to Figure 6A

[0134] Figure 14 A flow diagram illustrating an example process 1400 that can be performed by a device for depth sensing, according to some implementations, is shown. Process 1400 can be performed by a device such as device 400 described above with reference to Figure 4 In various aspects, example process 1400 of Figure 14 may be for use in a device such as device 400 described above with reference to Figure 10 ​​​one or more of blocks 1002 and 1004 of process 1000. At block 1402, the device switches the respective light projector between a structured light (SL) sensing mode and a time-of-flight (ToF) sensing mode via an openable and closable diffuser provided within each of the first light projector and the second light projector, also as described with respect to Figure 6B

[0135] The techniques described herein can be implemented in hardware, software, firmware, or any combination thereof, unless specifically described otherwise. Any features described as modules or components can also be implemented together in an integrated logic device or separately as discrete but interoperable logic devices. If implemented in software, the techniques can be realized at least in part by a non-transitory processor-readable storage medium, such as Figure 4 the memory 406 in device 400, comprising instructions 408 that, when executed by the processor 404 (or the active depth controller 410), cause the device 400 to perform one or more of the methods described above. The non-transitory processor-readable data storage medium can form part of a computer program product, which can include packaging materials.

[0136] The non-transitory processor-readable storage medium can include random access memory (RAM), read only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read only memory (EEPROM), flash memory, other known storage media, and / or the like. Additionally or alternatively, the techniques can be implemented at least partly by a processor-readable communication medium that carries or communicates code or

[0137] The various illustrative logical blocks, modules, circuits, and instructions described in connection with the embodiments disclosed herein can be executed by one or more processors, such as Figure 4 ​such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), application specific instruction set processors (ASIPs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. The term "processor" as used herein can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functionality described herein can be provided within dedicated software modules or hardware modules configured as described herein. Also, the techniques could be fully implemented in one or more circuits or logic elements. A general purpose processor can be a microprocessor, but in the alternative, the processor can be any conventional processor, controller, microcontroller, or state machine. A processor can also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0138] While the present disclosure illustrates illustrative aspects, it should be noted that various changes and modifications could be made herein without departing from the scope of the appended claims. For example, while the projector is shown as including a lens for directing light to the diffractive element, the projector can not include a lens or can include multiple lenses. In another example, the voltage applied by the device or light projector in adjusting the projection can be alternating current (AC) or direct current (DC), and the voltage can be constant or non-constant. Thus, the electricity can be any suitable electricity for adjusting the projection. Additionally, the functions, steps or actions of the methods according to the aspects described herein need not be performed in any particular order. Furthermore, although elements can be described as being performed in a particular order or in succession, unless explicitly stated, these elements need not be performed in this order. Moreover, the duration during which different actions are performed can differ from that which is described or illustrated, which can in some instances depend on the implementation. Additionally, although elements can be described as being performed by a device, active depth controller, processor, and / or memory, unless explicitly stated, these elements need not be performed by this device. Therefore, the disclosure is not limited to the illustrative examples and includes any elements which would perform the function described herein.

Claims

1. A depth sensing device, comprising: A first light projector and a second light projector, wherein the first light projector is configured to project light toward the second light projector, and the second light projector is configured to project light toward the first light projector; A reflective component located between the first light projector and the second light projector, the reflective component being configured to: redirect light projected by the first light projector to a first portion of the scene, and redirect light projected by the second light projector to a second portion of the scene, wherein the first portion and the second portion of the scene are adjacent to each other and do not overlap with each other; A receiver configured to detect the reflection of redirected light projected by the first and second light projectors; and The controller is configured to generate depth information based on the detected reflections of redirected light.

2. The device of claim 1, further comprising a controller configured to perform the following operations: Projection distortion in the generated depth information is corrected at least in part based on one or more refraction angles associated with the reflective component.

3. The device according to claim 1, wherein, The device is at least one of a camera or a cellular phone, and the controller is further configured to: The user's face is identified, at least in part, based on the generated depth information.

4. The device according to claim 2, wherein, The controller is also configured to resolve multipath interference (MPI) in the first part of the generated depth information based at least in part on the second part of the generated depth information.

5. The device according to claim 1, wherein, The reflective component is further configured to symmetrically refract light projected by the first light projector and the second light projector onto corresponding portions of the first and second portions of the scene.

6. The device according to claim 1, wherein, The reflective component includes: A first reflective element, configured to optically fold the first optical path of light projected from the first light projector; and The second reflective element is configured to optically fold the second optical path of light projected from the second light projector.

7. The device according to claim 6, wherein, The first reflective element and the second reflective element have the same refractive index.

8. The device according to claim 6, wherein: The first reflective element is further configured to: refract light folded by the first reflective element onto the first portion of the scene, at least partially based on the first folded optical path; and The second reflective element is also configured to refract light folded by the second reflective element onto the second part of the scene, at least in part, based on the second folded optical path.

9. The device according to claim 6, wherein: The first reflective element includes a first prism, the first prism including a first reflective surface for receiving light projected from the first light projector; and The second reflective element includes a second prism, which includes a second reflective surface for receiving light projected from the second light projector.

10. The device according to claim 1, wherein, Each of the first and second light projectors further includes an on / off diffuser configured to switch between a structured light (SL) sensing mode and a time-of-flight (ToF) sensing mode.

11. The device according to claim 1, wherein, Each of the first and second parts of the scene has a field of view (FOV) greater than 70 degrees.

12. The device according to claim 1, wherein: The light projected by the first light projector is projected along the axis in a first direction; The light projected by the second light projector is projected along the axis in a second direction opposite to the first direction; The first light projector and the second light projector are physically separated by a distance; and The reflective component is positioned along the axis between the first light projector and the second light projector.

13. A depth sensing method, comprising: Light is projected from the first light projector of the device toward the second light projector of the device; Light is projected from the second light projector toward the first light projector; Light projected by the first light projector is redirected to a first part of the scene via a reflective component located between the first light projector and the second light projector, and light projected by the second light projector is redirected to a second part of the scene, wherein the first part and the second part of the scene are adjacent to each other and do not overlap with each other; Detecting the reflection of redirected light projected by the first light projector and the second light projector; and Depth information is generated based on the detected reflection of redirected light.

14. The method of claim 13, further comprising: Projection distortion in the generated depth information is corrected at least in part based on one or more refraction angles associated with the reflective component.

15. The method of claim 13, further comprising: The user's face is identified, at least in part, based on the generated depth information.

16. The method of claim 14, further comprising: Multipath interference (MPI) in the first part of the generated depth information is addressed at least in part based on the second part of the generated depth information.

17. The method of claim 13, further comprising: The light projected by the first light projector and the second light projector is symmetrically refracted onto corresponding portions of the first and second parts of the scene.

18. The method of claim 13, further comprising: The first optical path of light projected from the first light projector is optically folded via the first reflective element of the reflective assembly; as well as The second optical path of the light projected from the second light projector is optically folded through the second reflective element of the reflective assembly.

19. The method according to claim 18, wherein, The first reflective element and the second reflective element have the same refractive index.

20. The method of claim 18, further comprising: The light folded by the first reflective element is refracted onto the first part of the scene, at least in part, based on the first folded optical path. as well as The light folded by the second reflective element is refracted onto the second part of the scene, at least in part, based on the second folded optical path.

21. The method of claim 18, further comprising: The light projected from the first light projector is received via the first reflecting surface of the first prism of the first reflecting element; as well as The light projected from the second light projector is received via the second reflecting surface of the second prism of the second reflecting element.

22. The method of claim 13, further comprising: The corresponding light projector is switched between structured light (SL) sensing mode and time-of-flight (ToF) sensing mode via an on / off diffuser provided in each of the first and second light projectors.

23. The method according to claim 13, wherein, Each of the first and second parts of the scene has a field of view (FOV) greater than 70 degrees.

24. The method of claim 13, wherein: The light projected by the first light projector is projected along the axis in a first direction; The light projected by the second light projector is projected along the axis in a second direction opposite to the first direction; The first light projector and the second light projector are physically separated by a distance; and The reflective component is positioned along the axis between the first light projector and the second light projector.

25. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a device, cause the device to perform operations including: Light is projected from the first light projector of the device toward the second light projector of the device; Light is projected from the second light projector toward the first light projector; Light projected by the first light projector is redirected to a first part of the scene via a reflective component located between the first light projector and the second light projector, and light projected by the second light projector is redirected to a second part of the scene, wherein the first part and the second part of the scene are adjacent to each other and do not overlap with each other; Detecting the reflection of redirected light projected by the first light projector and the second light projector; and Depth information is generated based on the detected reflection of redirected light.

26. The non-transitory computer-readable medium according to claim 25, wherein, The execution of the instruction causes the device to perform further operations including the following: Projection distortion in the generated depth information is corrected at least in part based on one or more refraction angles associated with the reflective component.

27. The non-transitory computer-readable medium according to claim 25, wherein, The execution of the instruction causes the device to perform further operations including the following: The light projected by the first light projector and the second light projector is symmetrically refracted onto corresponding portions of the first and second parts of the scene.

28. The non-transitory computer-readable medium according to claim 25, wherein, The execution of the instruction causes the device to perform further operations including the following: The first optical path of light projected from the first light projector is optically folded via the first reflective element of the reflective assembly; and The second optical path of the light projected from the second light projector is optically folded through the second reflective element of the reflective assembly.

29. The non-transitory computer-readable medium according to claim 28, wherein, The execution of the instruction causes the device to perform further operations including the following: The light folded by the first reflective element is refracted onto the first part of the scene, at least in part, based on the first folded optical path. Light projected from the first light projector is received via the first reflecting surface of the first prism of the first reflecting element; The light folded by the second reflective element is refracted onto the second part of the scene, at least in part, based on the second folded optical path. as well as The light projected from the second light projector is received via the second reflective surface of the second prism of the second reflective element.

30. The non-transitory computer-readable medium according to claim 25, wherein, The execution of the instruction causes the device to perform further operations including the following: The corresponding light projector is switched between structured light (SL) sensing mode and time-of-flight (ToF) sensing mode via an on / off diffuser provided in each of the first and second light projectors.

Images