Synchronized image capture for electronic scanning LIDAR systems

Abstract

Embodiments describe an electronically scanned optical system comprising an emitter array configured to emit light into a field, a time-of-flight (TOF) sensor array configured to detect the emitted light reflected from the field, and an image sensor array configured to detect ambient light in the field, wherein a field of view of the emitter array corresponds to at least a subset of the field of view of the TOF sensor array and the field of view of the image sensor array. The optical system also includes an emitter controller configured to activate a subset of multiple light emitters at a time; a TOF sensor controller configured to synchronize readout of each TOF light sensor while the corresponding light emitters are excited; and an image sensor controller configured to capture an image representing the field during an emission period.

Description

Background Art

[0001] Light imaging, detection, and ranging (LIDAR) systems measure the distance to a target by illuminating it with a pulsed laser and measuring the reflected pulses with a sensor. Time-of-flight measurements can then be used to create a digital 3D representation of the target. LIDAR systems are used in a variety of applications requiring 3D depth imagery, including archaeology, geography, geology, forestry, mapping, construction, medical imaging, and military applications, among others. Autonomous vehicles also use LIDAR for obstacle detection and avoidance, as well as for vehicle navigation.

[0002] Some LIDAR systems include mechanical, moving parts that physically scan transmitting and receiving elements through a rotation angle of less than or equal to 360° to capture images of the scene in the field. One example of such a system that can be used for obstacle detection and avoidance in a vehicle is generally referred to as a rotating or pivoting LIDAR system. In a pivoting LIDAR system, the LIDAR sensor is typically mounted within a housing to a pole that rotates or pivots a full 360 degrees. The LIDAR sensor includes a coherent light emitter (e.g., a pulsed laser in the infrared or near-infrared spectrum) to illuminate the scene around the vehicle as the LIDAR sensor continuously rotates through the scene. As the coherent light emitters rotate, they send radiation pulses away from the LIDAR system in different directions in the scene. Some of the radiation incident on surrounding objects in the scene is reflected from these objects around the vehicle, and these reflections are then detected by the imaging system portion of the LIDAR sensor at different time intervals. The imaging system converts the detected light into an electrical signal.

[0003] In this way, information about objects surrounding the LIDAR system, including their distance and shape, is collected and processed. The LIDAR system's digital signal processing unit processes the electrical signals and reproduces information about the objects in a depth image or 3D point cloud, which can be used as an aid for obstacle detection and avoidance, vehicle navigation, and other purposes. Additionally, image processing and image stitching modules can acquire this information and assemble a display of objects surrounding the vehicle.

[0004] Solid-state LIDAR systems also exist that don't include any moving mechanical parts. Rather than rotating across a scene, some solid-state LIDAR systems flash entire portions of the scene with light, capturing and sensing reflected light. In such systems, the transmitter comprises an array of emitters, all of which emit light at once to illuminate the scene, and are therefore sometimes referred to as "flash" LIDAR systems. Due to the lack of moving parts, flash LIDAR systems are less complex to manufacture; however, since all emitters are activated at once, they can require significant power to operate and significant processing power to process the signals from all pixel detectors at once. Furthermore, existing LIDAR systems are designed to capture only the emitted light for ranging purposes, constructing a three-dimensional image of the scene. This image is monochromatic and does not reflect what the naked eye perceives in the visible spectrum. Summary of the Invention

[0005] Embodiments of the present disclosure relate to stationary, solid-state LIDAR systems without a rotating column that can capture two-dimensional images in addition to distance measurements used to generate a three-dimensional image of the scene. The embodiments can capture images of a scene at high resolution and low power consumption, and with improved accuracy, reliability, size, integration, and appearance compared to currently available rotating LIDAR systems.

[0006] According to some embodiments, a solid-state electronic scanning LIDAR system may include a light transmission module and a light sensing module, the operation of which is synchronized so that the firing sequence of the emitter array in the light transmission module corresponds to the capture sequence of the time-of-flight (TOF) sensor array in the light sensing module for time-of-flight measurement. The LIDAR system may also include an image capture module that includes an image sensor array that can capture an ambient light image from the same field of view as the emitter array and the TOF sensor array. In various embodiments, the image sensor array can operate as a rolling shutter camera or as a global shutter camera, and in some embodiments, the image sensor array can operate in a synchronous manner with the TOF sensor array or in an asynchronous manner with the TOF sensor array.

[0007] In some embodiments, the emitter array, the TOF sensor array, and the image sensor array can each be coupled with an image space telecentric optical device that is collimated in object space to the emitter, TOF sensor, and image sensor field of view, respectively. The emitter array can be an array of vertical cavity surface emitting lasers (VCSELs), and the TOF sensor array can include an array of TOF light sensors, wherein each TOF light sensor in the TOF sensor array includes a collection of light detectors, such as single photon avalanche diodes (SPADs), that detect light within a narrow spectrum, such as light within the infrared spectrum. However, unlike the TOF sensor array, the image sensor array can include an image light sensor array for capturing an image of a scene perceived within the visible spectrum.

[0008] During operation, the emitter array can sequentially fire one or more columns of light emitters in the emitter array to project light (e.g., a pulsed light beam in the near-infrared wavelength range) into the scene, and the reflected light can be received by one or more corresponding columns of TOF light sensors in the TOF sensor array. By synchronizing the firing and capture sequences, the solid-state scanning LIDAR system can effectively capture an image by illuminating only a certain amount of light from a group of emitters at a given point in time that can be effectively detected by the corresponding group of TOF light sensors, thereby minimizing over-illumination of the scene and concentrating energy in a manner that makes the best possible use of the system's available power.

[0009] Further synchronized with the excitation and capture sequences, the corresponding image light sensor groups can be activated to measure the ambient light in the scene. The field of view of the image sensor array can overlap with the field of view of the emitter array (which can be the same as the field of view of the TOF sensor array), so that the electronic scanning LIDAR system can capture a two-dimensional color image of objects in the scene while also capturing the distance to these objects in the scene. This allows the electronic scanning LIDAR system to provide a high-resolution color image of the scene in addition to the distance to the objects in the scene, so that the user can better understand which part of the scene the electronic scanning LIDAR system is scanning.

[0010] In some embodiments, the electronic scanning LIDAR systems herein may also utilize micro-optics to further improve the efficiency of time-of-flight measurements of captured scenes. Micro-optics can improve the brightness and intensity of light emitted from the emitter array and minimize crosstalk between sensor pixels of a TOF sensor array in the electronic scanning LIDAR system. For example, in some embodiments, an aperture layer can be located in front of a TOF light sensor in the TOF sensor array. Each light emitter can correspond to an aperture in the aperture layer, and each aperture can correspond to a TOF light sensor in a TOF sensor array element, such that each light emitter corresponds to a specific TOF light sensor. The aperture can reduce the exposure of stray light to adjacent TOF light sensors and narrow the field of view of the TOF light sensor to a single point in the field.

[0011] In some embodiments, an electronic scanning optical system includes an emitter array having a plurality of light emitters configured to emit light into a field external to the optical system, a time-of-flight (TOF) sensor array including a plurality of TOF light sensors configured to detect emitted light reflected back from the field, and an image sensor array including a plurality of image light sensors configured to detect ambient light in the field, wherein a field of view of the emitter array corresponds to a field of view of the TOF sensor array and at least a subset of a field of view of the image sensor array. The optical system also includes an emitter controller coupled to the emitter array and configured to activate a plurality of light emitters in each emission cycle by activating a subset of the plurality of light emitters at a time, a TOF sensor controller coupled to the TOF sensor array and configured to synchronize readout of individual TOF light sensors within the TOF sensor array concurrently with excitation of corresponding light emitters in the emitter array such that each light emitter in the emitter array can be activated and each TOF light sensor in the TOF sensor array can be readout during the emission cycle, and an image sensor controller coupled to the image light sensor array and configured to readout at least a portion of the image light sensor array whose field of view overlaps with the field of view of the entire emitter array to capture an image representative of the field during the emission cycle.

[0012] In some embodiments, a solid-state optical system is disclosed that includes: a time-of-flight (TOF) sensor array, an image sensor array, an emitter array, an emitter controller, a TOF sensor controller, and an image sensor controller. The TOF sensor array may include a plurality of TOF light sensors operable to detect light emitted from the emitter array and reflected from a field external to the solid-state optical system, wherein each of the plurality of TOF light sensors has a discrete field of view in the field that does not overlap with the field of view of other TOF light sensors in the plurality of TOF light sensors beyond a threshold distance from the optical system. The image sensor array may include a plurality of image light sensors configured to detect ambient light in the field, wherein the discrete field of view of each of the plurality of TOF light sensors defines a field of view of the TOF sensor array, and wherein the field of view of the image sensor array includes the field of view of the TOF sensor array. The emitter array may include a plurality of light emitters, each light emitter configured to emit a discrete light beam at an operating wavelength into a field external to the optical system, such that the emitter array outputs the plurality of discrete light beams according to an illumination pattern that substantially matches the field of view of the time-of-flight sensor array in size and geometry within a range of distances from the system. An emitter controller may be coupled to the emitter array and operable to activate the plurality of light emitters during each emission cycle by activating a subset of the plurality of light emitters at a time. A time-of-flight sensor controller may be coupled to the time-of-flight sensor array and operable to synchronize readout of individual time-of-flight (TOF) light sensors within the TOF sensor array with the activation of corresponding light emitters in the emitter array, such that each light emitter in the emitter array can be activated and each TOF light sensor in the TOF sensor array can be readout during the emission cycle. Furthermore, an image sensor controller may be coupled to the image light sensor array and operable to readout at least a portion of the image light sensor array whose field of view overlaps with the field of view of the emitter array to capture an image representative of the field during the emission cycle.

[0013] In other embodiments, a solid-state optical system is provided that includes a light detection system, an image capture system, and a light emission system. The light detection system may include a first bulk optical device having a focal length, an aperture layer including a plurality of apertures separated from the first bulk optical device by the focal length, a time-of-flight (TOF) sensor array including a plurality of TOF light sensors, a lens layer including a plurality of lenses disposed behind the aperture layer of the TOF sensor array, and an optical filter operable to pass a narrow light band centered at a first wavelength and disposed between the first bulk optical device and the TOF sensor array. The aperture layer, the lens layer, and the TOF sensor array may be arranged to form a plurality of TOF channels, each of the plurality of TOF channels including an aperture from the plurality of apertures, a lens from the plurality of lenses, and a TOF light sensor from the plurality of TOF light sensors. Each TOF light sensor in the plurality of TOF channels may include a plurality of single photon avalanche diodes (SPADs), and each TOF channel in the plurality of TOF channels may define a discrete field of view in a field that does not overlap beyond a threshold distance from the optical system before the optical system, and wherein each TOF channel transmits light incident on the first bulk optical device to the plurality of SPADs of the TOF channel. The image capture system may include a second bulk optical device and an image sensor array, the image sensor array including a plurality of image light sensors operable to detect ambient light received through the second bulk optical device in the field. The discrete field of view of each TOF light sensor in the plurality of TOF light sensors may define a field of view of the TOF sensor array, and wherein the field of view of the image sensor array may include the field of view of the TOF sensor array. The light emission system may include a third body optical device and an emitter array, the emitter array including a plurality of vertical cavity surface emitting lasers (VCSELs), each VCSEL configured to emit a discrete light beam at a first wavelength through the third body optical device into a field external to the optical system, such that the emitter array outputs the plurality of discrete light beams according to an illumination pattern that substantially matches the field of view of the TOF sensor array in size and geometry over a range of distances from the system.

[0014] The solid-state optical system may further include an emitter controller, a TOF sensor controller, and an image sensor controller. The emitter controller may be coupled to the emitter array and operable to activate a plurality of VCSELs in each emission cycle by activating a subset of the plurality of light emitters at a time. The TOF sensor controller may be coupled to the TOF sensor array and operable to synchronize readout of individual TOF light sensors within the TOF sensor array with excitation of corresponding light emitters in the emitter array, such that each light emitter in the emitter array may be activated and each TOF light sensor in the TOF sensor array may be readout over the emission cycle. The image sensor controller may be coupled to the image light sensor array and operable to readout at least a portion of the image light sensor array whose field of view overlaps with the field of view of the emitter array to capture an image representative of the field during the emission cycle.

[0015] The nature and advantages of the embodiments of the present disclosure may be better understood with reference to the following detailed description and accompanying drawings. However, it should be understood that each figure is provided for illustrative purposes only and is not intended to be a definition of the limitations of the scope of the present disclosure. Furthermore, as a general rule, and unless otherwise apparent from the description, where the same reference numerals are used for elements in different figures, these elements are generally the same or at least similar in function or purpose. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a block diagram of an exemplary solid-state electronic scanning LIDAR system according to some embodiments of the present disclosure.

[0017] Figure 2A is a simplified illustration of an emitter array, a TOF sensor array, and an image sensor array for an exemplary solid-state electronic scanning LIDAR system, according to some embodiments of the present disclosure.

[0018] Figures 2B-2D is a simplified diagram illustrating an exemplary excitation sequence for an emitter array and a sensor readout sequence for a TOF sensor array and an image sensor array according to some embodiments of the present disclosure.

[0019] Figure 3 is an illustrative example of light transmission and detection operations of an electronic scanning LIDAR system in one scenario, according to some embodiments of the present disclosure.

[0020] Figure 4 is a simplified illustration of overlapping fields of view for an emitter array, a TOF sensor array, and an image sensor array according to some embodiments of the present disclosure.

[0021] Figure 5Ais a simplified diagram illustrating the overlapping fields of view of an emitter array and an image sensor array according to some embodiments of the present disclosure.

[0022] Figure 5B is a simplified diagram illustrating overlapping fields of view of an emitter array and an image sensor array according to some embodiments of the present disclosure, where the corresponding emitter array and image sensor array are not perfectly aligned.

[0023] Figure 6 is a timing diagram illustrating synchronization of an image sensor array and a transmitter array according to some embodiments of the present disclosure.

[0024] Figures 7A-7C is a timing diagram illustrating synchronization of an image sensor array and an emitter array according to some embodiments of the present disclosure, wherein the activation rate of the image sensor array is much higher than the activation rate of the emitter array.

[0025] Figure 8 is a simplified diagram showing a detailed side view of an exemplary solid-state electronic scanning LIDAR system according to some embodiments of the present invention.

[0026] Figure 9 is a simplified cross-sectional view of an exemplary enhanced light emission system according to some embodiments of the present invention.

[0027] Figure 10 is a simplified illustration of a solid-state electronic scanning LIDAR system implemented at an exterior area of ​​a road vehicle in accordance with some embodiments of the present invention.

[0028] Figure 11 is a simplified illustration of a solid-state electronic scanning LIDAR system implemented atop a road vehicle in accordance with some embodiments of the present invention. DETAILED DESCRIPTION

[0029] Some embodiments of the present disclosure relate to stationary, solid-state LIDAR systems, where there is no rotating column. Embodiments can not only emit narrowband light into a field outside the LIDAR system and capture the emitted narrowband light after it reflects from objects in the field for time-of-flight measurements, but can also sense ambient visible light from the field. Embodiments can then use the captured emitted light to create a three-dimensional representation of the field and use the captured ambient visible light to create a high-resolution two-dimensional image of the field. The three-dimensional representation of the field can then be matched with the two-dimensional image of the scene, thereby improving the situational awareness capabilities of the solid-state LIDAR system.

[0030] A solid-state array electronic scanning LIDAR system according to some embodiments of the present disclosure may include a light transmission module and a light sensing module. The light transmission module may include an emitter layer including an array of individual emitters, and the light sensing module may include a TOF sensor layer including an array of TOF light sensors. Each emitter in the emitter array may be paired with a corresponding sensor (i.e., a TOF light sensor) in the TOF sensor array. In some embodiments, instead of flashing a scene with the entire emitter group, only a subset of emitters is activated at a time, and the corresponding subset of TOF light sensors is read out only while the emitters are excited. Then, different subsets of emitters are activated at different times, and the corresponding subsets of TOF light sensors are read out simultaneously, so that all emitters in the emitter array can be activated, and all TOF light sensors in the TOF sensor array can be read out through one emission cycle.

[0031] The electronically scanning LIDAR system may also include an image capture module that enables the system to capture high-resolution images of a scene as perceived within the visible spectrum from the scene's ambient light. The field of view of the image sensor array in the image capture module may overlap with the field of view of the emitter array (i.e., the field of view of the emitter array may be a subset of the field of view of the image sensor array, where the subset is not limited to a portion of the entire field of view but may be equal to the entire field of view), so that the electronically scanning LIDAR system can capture color images of objects in the scene while also capturing distances to those objects in the scene. In some embodiments, the image sensor array is formed from an array of image light sensors, each of which is sensitive to visible light.

[0032] In some embodiments, the image capture module can operate as a rolling shutter camera, such that subsets of the image light sensor array can be activated simultaneously and in the same order as the corresponding emitter groups, so that the image sensor array can sense ambient light of a scene also illuminated by the emitter array to measure distance. In some embodiments, the image capture module can operate as a global shutter camera, such that the entire image light sensor array as a whole can be activated at once to sense ambient light within its field of view each time a group of emitters is activated, in which case only the portion of the image sensor array that shares the field of view with the activated emitter group is read out and used to generate an image, as will be discussed in further detail herein.

[0033] I. Electronic Scanning LIDAR System

[0034] A better understanding of the solid-state electronic scanning LIDAR system according to some embodiments of the present disclosure can be referred to Figure 1, which shows a block diagram of an exemplary solid-state electronic scanning LIDAR system 100 according to some embodiments of the present disclosure. The solid-state electronic scanning LIDAR system 100 may include a range finding system controller 104, an optical transmission (Tx) module 106, an optical sensing (Rx) module 108, and an image capture (IRx) module 150.

[0035] The Tx module 106 and the Rx module 108 can operate together to generate ranging data representing the distance to objects in the field of view, while the IRx module 150 can synchronize with the operation of the Tx module 106 to simultaneously generate image data representing images of objects within the same field of view. The solid-state electronic scanning LIDAR system 100 can generate ranging data by transmitting one or more light pulses 110 from the light transmission module 106 to objects in the field of view surrounding the system 100. A reflected portion 112 of the transmitted light is then detected by the light sensing module 108 after a delay time. Based on the delay time, the distance to the reflecting surface can be determined. In addition to ranging data, the solid-state electronic scanning LIDAR system 100 can also generate image data by sensing ambient visible light from the same field of view in which the light pulses 110 were emitted and from which the reflected portion 112 of the transmitted light was reflected. The image data can be used to render a high-resolution image of objects in the field of view, so that the ranging data is matched with the corresponding image data to provide a more detailed image of the scene.

[0036] The optical transmission module 106 includes an emitter array 114 (e.g., a two-dimensional emitter array) and a Tx optical system 116, which, when taken together with the emitter array 114, can form a light transmission system 138. The Tx optical system 116 can include an image space telecentric bulk emitter optics 144. In some embodiments, the Tx optical system 116 can further include one or more Tx optical components 146, such as an aperture layer, a collimating lens layer, and an optical filter, which can be combined with the emitter array 114 to form an array of micro-optical emitter channels, wherein each micro-optical emitter channel can increase the brightness of a light beam emitted from the bulk emitter optics and / or be used for beam shaping, beam steering, etc., as described herein with respect to Figure 11 As discussed above, the emitter array 114 or individual emitters may be narrowband laser sources, such as vertical cavity surface emitting lasers (VCSELs), laser diodes, and the like. The Tx module 106 may also include an optional processor 118 and memory 120, although in some embodiments, these computing resources may be incorporated into the ranging system controller 104. In some embodiments, pulse coding techniques such as Barker codes may be used. In these cases, the memory 120 may store a pulse code indicating when light should be transmitted. In some embodiments, the pulse code is stored as a sequence of integers stored in the memory.

[0037] The light sensing module 108 may include a time-of-flight sensor array 126 (e.g., a two-dimensional time-of-flight sensor array), where each time-of-flight sensor in the time-of-flight sensor array 126 may correspond to a specific emitter in the emitter array 114, for example, as a result of the geometric configuration of the Rx module 108 and the Tx module 106. In some embodiments, each time-of-flight sensor (sometimes referred to herein as simply a "sensor," or a "pixel") may include a collection of light detectors, such as SPADs, while in other embodiments, the TOF light sensor may be a single-photon detector (e.g., an APD). The light sensing module 108 includes a receiver optical sensing system 128, which, when combined with the time-of-flight sensor array 126, may form a light detection system 136. In some embodiments, the receiver optical sensing system 128 may include a receiver body, receiver optics 140, and receiver optical components 142, such as aperture layers, lens layers, and filters, which may be combined with the time-of-flight sensor array 126 to form an array of micro-optical receiver channels, where each micro-optical receiver channel measures light corresponding to an image pixel in a different field of view of the surrounding field in which the optical ranging device 102 is located.

[0038] Further details of the Rx and Tx optical systems according to some embodiments of the present disclosure are incorporated herein by reference. Figure 10 and discussed in commonly assigned U.S. patent application 15 / 979,235, filed May 14, 2018, entitled “Optical Imaging Transmitter with Brightness Enhancement,” which is incorporated herein by reference in its entirety for all purposes.

[0039] According to some embodiments of the present disclosure, in addition to the Rx module 108 and the Tx module 106, the optical ranging device 102 may further include an IRx module 150. The IRx module 150 may include an image sensor array 152 (e.g., a two-dimensional array of image light sensors for detecting visible light), wherein, as a result of the geometric configuration of the IRx module 150 and the Tx module 106, at least a portion of the field of view of the image sensor array 152 overlaps the entire field of view of the emitter array 114. The image sensor array 152 may be any suitable visible light sensor, such as a charge-coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) image sensor. The IRx module 150 may further include an image optical sensing system 154, which, when used with the image sensor array 152, may form an image capture system 156. In some embodiments, the image optical sensing system 154 may include an imaging volume receiver optics 158, through which ambient light travels to be focused and exposed on the image sensor array 152.

[0040] As described herein, the optical ranging device 102 can be an electronic scanning LIDAR device that can capture an image based on the distance to objects in a scene by activating only one group of emitters at a time and by reading out only the corresponding group of time-of-flight (TOF) light sensors while the emitters are activated. Different groups of emitters can be activated at different times, and the corresponding groups of TOF light sensors read out simultaneously, so that ultimately all emitters can be activated and all TOF light sensors in the TOF sensor array can be read out during a single emission cycle. In addition to the TOF light sensor groups, corresponding image light sensor groups can also be read out simultaneously to capture an image of the scene by measuring ambient light. As an example, the emitter array can emit light by activating one group at a time during each emission cycle, sequentially from left to right, while both the TOF sensor array and the image sensor array can be configured to read out the corresponding TOF light sensor groups and the corresponding image light sensor groups, respectively, in the corresponding order. Thus, embodiments of the present disclosure may include one or more components to synchronize the emission and sensing of light.

[0041] For example, the light transmission module 106 may include an emitter controller 115 coupled to the emitter array 114. The emitter controller 115 is configured to control the operation of the emitter array 126 by, for example, selectively activating each group of emitters according to a desired activation sequence. The emitter controller 115 may include a suitable processor, such as an ASIC, a microcontroller, an FPGA, or other suitable processing element, and one or more driver components for operating the emitter array 114. Similarly, the light detection system 136 may include a time-of-flight sensor controller 125 coupled to the time-of-flight sensor array 126 and configured to control the operation of the time-of-flight sensor array 126. Furthermore, the image capture system 156 may include an image sensor controller 160 coupled to the image sensor array 152 and configured to control the operation of the image sensor array 152. The time-of-flight sensor controller 125 and the image sensor controller 160 may each be a suitable component or group of components capable of selecting one or more light sensors to sense light, such as an ASIC, a microcontroller, an FPGA, or other suitable processor coupled to a selection circuit (e.g., a multiplexer).

[0042] In some embodiments, the emitter controller 115, the TOF sensor controller 125, and the image sensor controller 160 are synchronized so that the sequence of light emissions in the emitter array 114 is synchronized with the sequence of activating and / or reading out the TOF light sensors in the TOF sensor array 126 and the image light sensors in the image sensor array 152. As an example, the emitter controller 115, the TOF sensor controller 125, and the image sensor controller 160 can all be coupled to a clock 117 so that all controllers can operate based on the same timing scheme. The clock 117 can be an electronic component that generates a specific signal that oscillates between a high state and a low state at a specific speed to coordinate the actions of digital circuits. Alternatively, the emitter controller 115, the TOF sensor controller 125, and the image sensor controller 160 can include their own clock circuits for coordinating their own actions. In such an embodiment, the emitter controller 115, the TOF sensor controller 125, and the image sensor controller 160 may be communicatively coupled together via the communication lines 119 and 162, such that the TOF sensor controller 125 and the image sensor controller 160 may synchronize their clocks with the emitter controller 115. In that way, the TOF sensor controller 125, the image sensor controller 160, and the emitter controller 115 may respectively synchronously operate the TOF sensor array 126, the image sensor array 152, and the emitter array 114 to achieve image capture.

[0043] In some embodiments, image sensor controller 160 can operate image sensor array 152 as a rolling shutter camera, in which an image of a scene is captured by sequentially activating and reading out portions of the entire image light sensor array. The order in which the portions of image sensor array 152 are activated and read out can correspond to the order in which emitters in the emitter array are excited and / or the TOF light sensors in the TOF sensor array are read.

[0044] In certain embodiments, image sensor controller 160 can operate image sensor array 152 as a global shutter camera, in which an image of a scene is captured by sequentially activating the entire image light sensor array all at once and at a specific activation rate, while only reading out a portion of the image light sensors. In some cases, the rate at which image sensor array 152 is activated corresponds to the rate at which emitter groups in the emitter array are activated. Instead of reading out image data from the entire image sensor array 152, only those portions of the image light sensors having a field of view corresponding to the field of view of the corresponding emitter group being activated can be read out to generate image data for generating a color image of the scene.

[0045] As described herein, the activation rate of the image sensor array 152 can be synchronized with the operation of the TOF sensor array 126. In this case, the rate at which the image sensor array 152 is activated can be equal to or greater than the rate at which the TOF sensor array 126 is read (and the rate at which the emitter array 114 is fired). For example, when the rates are the same, each time the image light sensor is activated, the TOF light sensor can be activated simultaneously but for the same or different durations, as will be described herein with respect to Figure 6 When the activation rate of the image sensor array 152 is greater than the activation rate of the TOF sensor array 126, for example, 2x, 4x, or 8x the rate of the TOF sensor array 126, the image sensor array 152 may be activated more times than the TOF sensor array 126 in a given time period, as will be discussed further herein. Figures 7A-7C Further discussion.

[0046] In some further embodiments, image sensor controller 160 may operate image sensor array 152 asynchronously with respect to the operation of emitter array 114 and TOF sensor array 126. For example, the activation rate of image sensor array 152 may be different from the activation rate of TOF sensor array 126 and emitter array 114, and not every activation of a TOF light sensor or firing of an emitter may correspond to a simultaneously activated image light sensor. In such embodiments, a two-dimensional color image representing a captured three-dimensional image may be generated by reading out image data captured from the activation of the TOF sensor light sensor group that is closest in time to the activation of TOF sensor array 126 or emitter array 114.

[0047] In some further embodiments, the ranging system controller 104 can be configured to synchronize the operation of the light sensing module 108, the image capture module 150, and the light transmission module 106 so that the light emission sequence of the emitter array 114 is synchronized with the light sensing sequence of the TOF sensor array 126 and the image sensor array 152. For example, the ranging system controller 104 can instruct the emitter array 114 of the light transmission module 106 to emit light by activating one group at a time and in a left-to-right sequence during each emission cycle, and correspondingly instruct the TOF sensor array 126 in the light sensing module 108 and the image sensor array 152 in the image capture module 150 to sense light from the corresponding TOF light sensor group and image light sensor group, respectively, at once in the same sequence. In such an embodiment, the ranging system controller 104 can have its own clock signal based on which it sequences instructions to the light sensing module 108, the image capture module 150, and the light transmission module 106. It should be understood that other forms of sequences for light detection are contemplated herein and that such sequences are not limiting, as will be further discussed herein.

[0048] To illustrate an example of sequential firing of an emitter array and sensing of a TOF sensor array and an image sensor array, refer to Figure 2A , which is a simplified illustration of an emitter array 210 and a TOF sensor array 220 for an exemplary solid-state electronic scanning LIDAR system 200, according to some embodiments of the present disclosure. The emitter array 210 can be a two-dimensional mxn array of emitters 212 having m columns and n rows, and the TOF sensor array 220 can correspond to the emitter array 210, such that each TOF light sensor 222 maps to a corresponding emitter 212 in the emitter array 210. Thus, the TOF sensor array 220 can be a corresponding two-dimensional mxn array of TOF light sensors 222. On the other hand, the image sensing array 230 can be a two-dimensional yxz array of image light sensors 232 having y columns and z rows. In some embodiments, the two-dimensional yxz array of image light sensors 232 can have a greater or significantly greater number of columns and rows than the mxn array of emitters 212, such that the resolution of the image sensing array is significantly greater than the resolution of the emitter array. For example, in some embodiments, image sensor array 230 may include 2x, 4x, 8x, 16x, or more columns and / or rows of image light sensors than the number of columns and / or rows of TOF light sensors included in TOF sensor array 126 .

[0049] In some embodiments, emitter array 210 and TOF sensor array 220 are typically large arrays that include more elements (i.e., more emitters and more TOF light sensors) than typically employed in rotating LIDAR systems. The size, i.e., overall physical size, of TOF sensor array 220 (and therefore the corresponding emitter array 210 corresponding to TOF sensor array 220 used to illuminate the field of view), along with the spacing of the TOF light sensors within TOF sensor array 220, can determine the field of view and the resolution of images that can be captured by TOF sensor array 220. Larger arrays generally result in a larger field of view, while smaller spacing generally results in captured images with higher resolution. In some embodiments, image sensor array 230 can have an overall size equal to or larger than that of emitter array 210, while having a spacing smaller than or substantially smaller than that of emitter array 210. Consequently, image sensor array 230 can have a field of view that is the same as or larger than that of emitter array 210 while achieving greater resolution. In some embodiments, the overall size of image sensor array 230 is larger than the overall size of emitter array 210, thereby increasing alignment tolerances and enabling compatibility with emitter arrays of different sizes, as will be discussed herein with respect to Figures 5A-5B Further discussion.

[0050] In some embodiments, emitter array 210, TOF sensor array 220, and image sensor array 230 are each formed from a single semiconductor die, while in other embodiments, one or more of arrays 210, 220, and 230 may be formed from multiple chips mounted on the same substrate. In some embodiments, each of emitter array 210, TOF sensor array 220, and image sensor array 230 is formed on a separate semiconductor die configured to have overlapping fields of view, as will be discussed herein with respect to Figure 4 Further discussion.

[0051] The emitter array 210 may be configured to operate so that one or more sets of emitters (where each set is referred to herein as a "group") may be excited simultaneously. Figure 2A In the illustrated embodiment, emitter array 210 is configured to include six groups 214(1 ... 214(6), where each group includes four columns of emitters. TOF sensor array 220 can be configured to have a similar geometry as emitter array 210, such that TOF light sensors 222 are arranged in similarly arranged groups. Image sensor array 230, on the other hand, can be configured to have a higher resolution than emitter array 210 and have certain image light sensor groups positioned within the field of view of the corresponding emitter groups to be assigned to those groups. Thus, in Figure 2A In the illustrated embodiment, the TOF sensor array 220 is further configured to include six groups 224(1...224(6), each of which includes four columns of TOF light sensors; and the image sensor array 230 can be divided into six groups 234(1)-234(6), each of which has the same field of view as the corresponding emitter group 214(1)-214(6).

[0052] Figures 2B-2D is a simplified diagram illustrating an excitation sequence of the emitter array 210 and a sensor readout sequence of the TOF sensor array 220 and the image sensor array 230 according to some embodiments of the present disclosure, wherein the image sensor array 230 operates with a rolling shutter technique. Figure 2BAs shown, a first phase of an image capture sequence may be initiated by activating emitter group 214(1) of emitter array 210 and simultaneously reading out TOF sensor array 220 and sensor group 224(1) of image sensor group 234(1) of image sensor array 220. During this first phase, a pulse of light emitted from each individual emitter in emitter group 214(1) is emitted into the field. The emitted light may then be reflected from one or more objects in the field and captured by a corresponding subset of TOF light sensors within sensor group 224(1) of TOF sensor array 220. Simultaneously, ambient light present in the scene within the field of view is captured by a corresponding group 234(1) of image light sensors 232.

[0053] Next, in the second stage of the sequence, emitters from a second group 214(2) of the emitter array may be activated to emit light pulses that may be read by TOF light sensors in a TOF sensor group 224(2) in the TOF sensor array while image light sensors in a group 234(2) in the image sensor array measure the ambient light at that location, e.g. Figure 2C The sequential activation of the emitter groups and the simultaneous readout of the TOF light sensors in the corresponding groups of image light sensors and the TOF light sensors in the groupings continue until the last group 214 (6) of emitters is activated and the last group of TOF light sensors 224 (6) and image light sensors 234 (6) is read, as shown. Figure 2D When a full cycle is completed (in Figures 2B-2D In the depicted example, six stages of an image capture sequence, each group of emitters in emitter array 210 will have been activated, each corresponding group of TOF light sensors in TOF sensor array 220 will have been read out to detect photons emitted from the corresponding group in emitter array 210, and each corresponding group of image light sensors in image sensor array 230 will have been read out to detect ambient light in the field of view that was emitted by emitter array 210. This cycle can then be repeated continuously while LIDAR system 200 is operating.

[0054] Although Figure 2A The illustration shows an array of emitters and two arrays of sensors divided into six different groups / groups, each group having a specific number of emitters or light sensors, but the embodiments are not limited to such a configuration. Other embodiments may have more or less than six groups / groups and more or fewer emitters or light sensors per group / group. For example, in some embodiments, k groups of emitters and k groups / groups of light sensors are employed, where k is more or less than Figure 2AAs a further example, in some embodiments, the LIDAR sensor 200 may be divided into 16, 32, 64, 128, or more groups, where each group includes 1, 2, 4, 8, or more columns of emitters, without departing from the spirit and scope of the present disclosure. Figure 2A While groups are discussed in terms of columns of emitters and columns of photosensors, in other embodiments, the emitter and photosensor arrays can be divided into groups having one or more rows of emitters and one or more rows of photosensors rather than columns, such that one or more rows of emitters are excited when one or more corresponding rows of photosensors are read simultaneously. In other embodiments, a group in emitter array 210 can include a subset of emitters that includes emitters in multiple columns and rows (e.g., emitters arranged in a square or rectangular pattern), and a group / group in TOF sensor array 220 / image sensor array 230 can include a subset of TOF photosensors / image photosensors arranged in a pattern corresponding to the emitter subset.

[0055] In addition, although Figures 2B to 2D An image capture sequence is shown in which the emitters excited advance one group / group per stage, but embodiments of the present disclosure are not limited to any particular sequence. For example, in some embodiments, the following sequence may be employed in a LIDAR system having k groups / groups: for the first stage, the first group of emitter array 210 is excited; for the second stage, the (k / 2+1)th group is excited; for the third stage, the second group is excited; for the fourth stage, the (k / 2+2)th group is excited, and so on, until the kth stage in which the kth group is excited. Such an embodiment may be beneficial in minimizing crosstalk within the TOF sensor array because adjacent sensor groups are not read out in consecutive stages. As another example, two or more adjacent emitter groups may be excited simultaneously, and corresponding two or more adjacent sensor groups / groups read out simultaneously. As an example in which two groups are excited and read out simultaneously, in the first stage of the image capture sequence, groups 214(1) and 214(2) of emitter array 210 may be excited, and in the second stage, groups 214(3) and 214(4) may be excited, and so on.

[0056] Although Figures 2B-2DEmbodiments are shown in which the image sensor array operates according to a rolling shutter technique, wherein specific subsets of image light sensors are activated and read out in a specific order, but the embodiments are not limited to such operation. For example, some embodiments are configured to operate the image sensor array using a global shutter technique, wherein the entire array of image light sensors of the image sensor array is activated at each stage of the emitter excitation sequence, but wherein only those image light sensors having a field of view corresponding to the field of view of the excited emitter group and / or the field of view of the activated TOF sensor group are read out to generate image data for capturing a two-dimensional color image of the scene. Figures 2B-2D The examples discussed are just some of the many different excitation and readout sequences that are possible, and in other embodiments, other excitation and readout sequences are possible.

[0057] Figure 3 is an illustrative example of light transmission and TOF detection operation of the electronic scanning LIDAR system 300 according to some embodiments of the present disclosure in certain aspects. Specifically, Figure 3 A solid-state electronic scanning LIDAR system 300 is shown, which may represent Figure 1 A LIDAR system 100 is shown that collects three-dimensional data and range data for a volume or scene surrounding the system. Figure 3 is an idealized diagram that highlights the relationship between the transmitter and sensor, so other components are not shown. Figure 3 The discussion of Figure 4 , to provide a better understanding of distance determination techniques using TOF methods. The image sensor array is not Figure 3 and Figure 4 5A and 5B are shown to simplify the discussion of how the LIDAR system in the embodiments performs ranging, but it should be understood that the image sensor array can be positioned near the TOF sensor array and / or the emitter array to sense ambient light from the scene in the same or overlapping field of view as the emitter and TOF sensor array, which will be discussed herein with respect to 5A and 5B.

[0058] refer to Figure 3, the electronic scanning LIDAR system 300 includes an emitter array 310 (e.g., emitter array 114) and a TOF sensor array 320 (e.g., TOF sensor array 126). The emitter array 310 can be an array of light emitters, such as a vertical cavity surface emitting laser (VCSEL) array, etc., which includes emitter groups 312 (1) to 312 (n). The TOF sensor array 320 can be an array of light sensors including TOF light sensor groups 322 (1) to 322 (n). The TOF light sensors can be pixelated light sensors that use a set of discrete light detectors, such as single photon avalanche diodes (SPADs), for each TOF light sensor. However, various embodiments may deploy other types of light sensors.

[0059] For ease of illustration, emitter array 310 is depicted as having seven groups of emitters, where each group includes a column of three emitters, and TOF sensor array 320 is depicted as having a corresponding arrangement of TOF light sensors. It should be understood that emitter groups 312(1) to 312(n) and TOF light sensor groups 322(1) to 322(n) may represent portions of larger groups of emitter array 310 and TOF sensor array 320, respectively. Thus, although for ease of illustration, Figure 3 Figure A shows only 21 different emitters and TOF light sensors, but it will be appreciated that other embodiments may have significantly more emitters, including groups with multiple columns of emitters, each column including more than three individual emitters. In other words, a denser array of emitters and a correspondingly denser array of TOF light sensors may achieve a denser sampling of points.

[0060] Each emitter can be spaced a distance apart from its neighboring emitters and can be configured to transmit light pulses from its neighboring emitters to a different field of view, thereby illuminating only the corresponding field of view associated with that emitter. For example, emitter group 312(1) emits illumination beams 314(1) (each formed of one or more light pulses) into region 315(1) of the field of view and thereby reflects off of trees 330 in the field. Similarly, emitter group 312(n) emits illumination beams 314(n) into region 315(n) of the field of view. It should be understood that in Figure 3 In the embodiment shown in A, the transmitter array 310 scans its groups in order from left to right. Figure 3 A shows the first time instance when transmitter group 312(1) is activated and the last time instance when the last group, transmitter group 312(n), is activated. Other groups can be stepped from left to right between groups 312(1) to 312(n). Although Figure 3A shows an embodiment in which the emitter array 310 and the TOF sensor array 320 are operated in vertically oriented groups and in sequence, but the embodiments are not limited to such a configuration. In other embodiments, the emitter and TOF sensor arrays 310 and 320 can be operated in vertically oriented groups in a non-sequential order to minimize crosstalk, or in horizontally oriented groups in a sequential or non-sequential order, or in any other suitable order to transmit and receive light, as discussed above and further herein.

[0061] Each field of view illuminated by an emitter can be thought of as a pixel or point of light in the corresponding 3D image generated from the ranging data. Each emitter can therefore be unique to the other emitters and non-overlapping with the other emitters, so that there is a one-to-one mapping between the group of emitters and the group of non-overlapping fields of view. In some embodiments, the emitter array 310 and the TOF sensor array 320 are each solid-state devices that can be very small and very close to each other. For example, according to an embodiment of the present invention, the size of the emitter or TOF sensor array can be in the range of a few millimeters to a few centimeters. In this way, the size of the two arrays and their separation distance (which can be less than a few centimeters) can be negligible compared to the distance to the objects in the scene. When this arrangement of the emitter and TOF sensor arrays is paired with respective bulk optics that can respectively calibrate the light emitted by the emitter arrays and focus the reflected light into the TOF sensor array, the fields of view of the TOF sensor array and the emitter array can be very similar beyond a threshold distance so that each emitter and the corresponding TOF sensor observes substantially the same light spot in the field of view. Reference Figure 4 to better understand the concept.

[0062] Figure 4 is a simplified illustration of overlapping fields of view for emitter array 310 and TOF sensor array 320 according to some embodiments of the present invention. Each emitter in emitter array 310 may emit in a cone 402. Figure 4 As shown, the cone of light pulses is collimated by the bulk transmitter optics 404 and output into the field as emitted light 406. The emitted light 406 can then reflect off one or more objects in the field and propagate back to the TOF sensor array 320 as reflected light 412, which first propagates through the bulk receiver optics 410, which focuses the reflected light 412 downward into a focal point, becoming a cone of pulsed light 408, which then impinges on a corresponding TOF light sensor in the TOF sensor array 320. Figure 4 It is understood that the distance between the body transmitter and receiver optics 184 and 410 can be, for example, between 1-3 cm, which is relatively small compared to the distance to the scene. Therefore, as the scene gets farther away, the field of view of the transmitter array and the field of view of the TOF sensor array increasingly overlap. For example, Figure 4 As shown, the overlapping regions 414, 416, and 418 of the fields of view of emitter array 310 and TOF sensor array 320 increase as the distance to the scene increases. Thus, at distances near the end of the scene, such as objects in the field of view, the field of view of emitter array 310 can substantially overlap with the field of view of TOF sensor array 320. Thus, even if the volume receiver and transmitter optics are one or more centimeters apart, each corresponding emitter and TOF light sensor can observe substantially the same point in the scene. That is, each illumination beam projected from volume transmitter optics 184 into the field of view in front of the system can have substantially the same size and geometry as the field of view of a corresponding TOF light sensor (or a micro-optical receiver channel of a corresponding TOF light sensor) at a distance from the system. In some embodiments, emitter array 310 can selectively project illumination beams into the field of view in front of system 300 according to an illumination pattern that substantially matches the field of view of an input channel in size and geometry across a range of distances to system 300. By having substantially overlapping fields of view between the emitter array and the TOF sensor array, the solid-state electronic scanning LIDAR system 300 can achieve a high signal-to-noise ratio (SNR).

[0063] In some embodiments, the emitter array and the TOF sensor array have matching geometries, with the emitter array's bulk optics being substantially identical to the TOF sensor array's bulk optics. In other embodiments, the dimensions and bulk optics of the TOF sensor array 320 may differ from those of the emitter array 310, but they may be selected so that corresponding groups of the emitter array 310 and the TOF sensor array 320 have substantially the same field of view. For example, the dimensions of the TOF sensor array 320 may be larger than those of the emitter array 310. This means that the bulk receiver optics 410 of the TOF sensor array 320 should be different from the bulk transmitter optics 184 of the emitter array 310, and both bulk optics should be carefully selected so that the fields of view of corresponding groups in the two arrays are substantially the same. For example, a similar bulk optic with lens elements that are twice as large as those of the emitter array 310 can be used. The resulting focal length of the bulk receiver optics will be twice that of the bulk transmitter optics. In this case, the height and width of the TOF sensor array 320 should be twice that of the emitter array 310, and its receiving aperture diameter should be twice that of the transmitting diameter, thereby ensuring that the viewing angles of each TOF light sensor and emitter are matched.

[0064] To ensure that corresponding groups of emitter array 310 and TOF sensor array 320 see the same field of view, an alignment process can be performed on the LIDAR system 300 before use in the field, such as by a manufacturer. Design features of some embodiments of the present invention (e.g., having a single semiconductor die or multi-chip module for the emitter array and a single semiconductor die of a multi-chip module for the TOF sensor array) allow the manufacturer to perform alignment only once, thereby simplifying the manufacturing and post-manufacturing maintenance of the LIDAR system 300. During optical alignment, the field of view of each pixel and each emitter can be measured to ensure that they are identical. The alignment process can take into account lens characteristics such as aberrations, distortion, and focal length, and adjust the position and orientation of lens elements relative to external components.

[0065] Because the field of view of an emitter overlaps with the field of view of its corresponding sensor, each TOF light sensor can ideally detect the reflected illumination beam originating from its corresponding emitter, ideally without crosstalk, i.e., without detecting reflected light from other illumination beams. Figure 3 A, emitter group 312(1) emits illumination beam 314(1) into region 315(1) of the field of view, and some of the illumination beam reflects off object 330 (i.e., a tree). Ideally, the reflected portion of light 324(1) is detected only by TOF light sensor group 322(1). Thus, emitter group 312(1) and TOF light sensor group 322(1) share the same field of view. Similarly, emitter group 312(n) and TOF light sensor group 322(n) may also share the same field of view, such that the reflected portion of light 324(n) is detected only by TOF light sensor 322(n). For example, during the last iteration of the emission cycle, emitter group 312(n) emits illumination beam 314(n) into region 315(n) of the field of view, and some of the illumination beam reflects off object 332 (i.e., a car parked next to object 330). In one cycle, Figure 3 The solid-state electronic scanning LIDAR system 350 in A can capture and generate an image representing a scene including a portion of the tree 330 and the car 332. Additional cycles can further capture other areas of the scene, especially when the system 300 is moving (e.g., when the system 300 is mounted on a car), which will be discussed further herein with reference to Figures 12 and 13. Although the corresponding emitter and TOF light sensor are in Figure 3 A are shown in the same relative positions in their respective arrays, but any emitter can be paired with any TOF light sensor depending on the design of the optics used in the system.

[0066] During ranging measurements, reflected light from different fields of view distributed around the volume surrounding the LIDAR system is collected and processed by various TOF light sensors to obtain distance information for any object in each corresponding field of view. As described above, time-of-flight technology can be used, in which a light emitter emits a precisely timed pulse, and the reflection of the pulse is detected by the corresponding TOF light sensor some time later. The time elapsed between emission and detection and the known speed of light are then used to calculate the distance to the reflecting surface. In some embodiments, the TOF light sensor can obtain additional information to determine other properties of the reflecting surface in addition to the distance. For example, the Doppler shift of the pulse can be measured by the sensor and used to calculate the relative velocity between the TOF light sensor and the reflecting surface. The pulse intensity can be used to estimate the target reflectivity, and the pulse shape can be used to determine whether the target is a hard or diffuse material.

[0067] According to some embodiments, LIDAR system 300 can transmit multiple pulses of light. In some embodiments, each coded pulse has an embedded positive pulse code formed by light intensity. The system can determine the temporal location and / or amplitude of the light pulses in the presence of background light by creating an intensity histogram of the reflected light detected at different time bins. For each time bin, the system adds a weighting value to the intensity histogram, which depends on the intensity of the detected light. The weighting value can be positive or negative and have different magnitudes.

[0068] By selecting different combinations of positive pulse codes and applying different weights, the system can detect both positive and negative codes suitable for standard digital signal processing algorithms. This approach provides a high signal-to-noise ratio while maintaining low uncertainty in the measured temporal position of the reflected light pulse.

[0069] II. Configuration and Operation of Image Sensor Arrays for Image Capture

[0070] As described herein, an emitter array and an image sensor array of an electronically scanning LIDAR system may have overlapping fields of view such that the image sensor array may capture images of a scene corresponding to the field of view of the emitter array.

[0071] A. Overlapping Fields of View

[0072] Figure 5A5 is a simplified diagram illustrating the overlapping fields of view of an emitter array field of view 500 and an image sensor array field of view 502, according to some embodiments of the present disclosure. The entire emitter array field of view 500 can overlap with at least a portion of the image sensor array field of view 502, such that the image sensor can capture an image of a scene corresponding to the entire field of view of the emitter array. That is, in some embodiments, the emitter array field of view 500 can be equal to a subset of the image sensor array field of view 502. The image sensor field of view 502 is shown with a plurality of rectangular blocks representing the respective fields of view of the respective image photosensors within the image sensor array.

[0073] The emitter array field of view 500 may include different field of view groups 504a-f, the number of which corresponds to the number of emitter groups in the emitter array. Figure 5A In the example shown, there are six. Each field of view group 504a-f can be a field of view area illuminated by light emitted from a corresponding emitter group, as described herein with respect to Figures 2A-2D and 3-4. The image sensor array field of view 502 may also include six field of view groups 506a-f corresponding to the number of groups of image light sensors used to sense ambient light. Each field of view group 506a-f may be a field of view area corresponding to the area where the corresponding emitter group emits light, as described herein with respect to Figures 2A-2D Thus, during operation, as the emitter group sequentially emits light to the field of view group 504a-f, the image light sensor group with corresponding fields of view 506a-f can simultaneously read out image light sensor data in the same sequence to capture an image of the scene.

[0074] In some embodiments where the image sensor array operates according to a rolling shutter technique, each image light sensor is individually addressable so that specific groups of image light sensors can be activated to sense light in sequence without having to activate every image light sensor in the image sensor array, for example, without having to activate image light sensors that do not have overlapping fields of view with emitter excitation groups and / or TOF light sensor activation groups. In this case, each image light sensor can be mapped to a corresponding emitter group so that the image light sensors can be activated by a controller using a control signal. The mapping between image light sensors and emitter groups can be recorded in a memory and used by the controller. For example, a control signal bus can be coupled from an image sensor controller (e.g., image sensor controller 160) to the image sensor array. A control signal can be sent to the image sensor array indicating which group of image light sensors to activate, and thereby which field of view groups 506a-f to measure. Continued Figure 5A In the example shown, the control signal may be 1-6, where 1 indicates that group 506a is to be measured, and 6 indicates that group 506f is to be measured. Thus, the image light sensor group associated with field of view group 506a may be configured to activate upon receiving control signal 1, as shown in FIG. Figure 5A The same can be said for the rest of the groups 506b-f. Configuring the image sensor array to be individually addressable can save power because only a subset of the image light sensors are activated to sense light, thereby minimizing power waste by illuminating all of the image light sensors at once.

[0075] In embodiments where the image sensor array operates according to global shutter techniques, the entire image light sensor array can be activated at a time. Thus, image data for the entire image sensor array field of view 502 can be captured, even if only a portion of the field of view corresponds to the field of view of an emitter excitation group (e.g., group 504a). In this case, image data will only be read from those image light sensors (e.g., image light sensor group 506a) that correspond to the field of view of emitter excitation group 504a, and all image light sensors with fields of view outside of group 504a will be ignored. Thus, those image light sensors with fields of view outside of emitter array field of view 500 can be completely ignored.

[0076] To assign the correct group control signal to each image light sensor, a calibration process can be performed during manufacturing or at startup of the LIDAR system. Each emitter group can be activated to emit light, while keeping track of which image light sensors receive light. Image light sensors that receive light when emitter group 1 is activated can be programmed to activate when emitter group 1 is activated. The same operation can be performed for the remaining emitter groups.

[0077] The methods and configurations discussed herein with respect to individually addressable configurations may also be implemented with image sensor arrays that are not individually addressable, for example, an image sensor array that is configured as a global shutter. In this case, each time the image sensor array is activated, the entire array of image light sensors is activated at once, rather than activating only a subset of them. Thus, some embodiments may configure the image sensor array to read out image data for only those image light sensors that correspond to a particular field of view group 506a-f, even though each image light sensor has been activated to sense light. Configuring the image sensor array as a global shutter may save design costs and ease of manufacturing, and may allow the LIDAR system to utilize existing high-quality image sensors, as discussed herein with respect to Figures 7A-7C discussed.

[0078] In some embodiments, the image sensor array field of view 502 may be larger than the emitter array field of view 500. Figure 5A As shown in Figure 1, the image sensor array can be physically larger than the emitter array. Having a larger image sensor array allows for greater compatibility with different emitter configurations of varying sizes. A larger image sensor array also allows for looser manufacturing tolerances, thereby improving manufacturing ease and saving manufacturing costs.

[0079] For example, Figure 5B is a simplified diagram illustrating the overlapping fields of view of an emitter array field of view 510 and an image sensor array field of view 512, where the corresponding emitter array and image sensor array are not perfectly aligned. Some embodiments of the present invention can correct for such misalignment as described herein. For example, by making the physical size of the image sensor array, and therefore the field of view 512, larger than the physical size of the emitter array, and therefore the field of view 510, the entire emitter array field of view 510 can still overlap with at least a portion of the image sensor array field of view 512 even if the image sensor array is positioned at a certain angle relative to the emitter array, as shown in FIG. Figure 5B Even if the image sensor array field of view 512 is angled, those image light sensors corresponding to the field of view of the emitter array can still be programmed to activate in sync with the emitter array. For example, the image light sensor group associated with field of view group 514a can be configured to activate upon receiving control signal 1, as shown in FIG. Figure 5B The angled arrangement may result in a set of image light sensors having jagged edges as an artifact of the angular mismatch between the image sensor array field of view 512 and the emitter array field of view 510 .

[0080] The larger size also allows for some tolerance for optical aberrations. As an example, imperfect volumetric imaging optics can result in some distortion of the image sensor array field of view 512, which can cause the edges of the field of view to collapse inward. However, because the image sensor array field of view 512 is larger than the emitter array field of view 510, the entire emitter array field of view 510 can still overlap with at least a portion of the image sensor array field of view 512, and the ability to capture an image of the emitter array field of view 510 may not be negatively impacted.

[0081] B. Image Light Sensor Synchronization and Timing

[0082] As discussed herein, activation of the image light sensors in the image sensor array can be synchronized with the excitation of the emitters in the emitter array and the activation of the TOF light sensors in the TOF sensor array. Furthermore, in some embodiments, the exposure of the synchronized image light sensors can be modified between activation cycles to achieve an optimal exposure time. Figure 6 6 is a timing diagram illustrating synchronization of an image sensor array with a TOF sensor array according to some embodiments of the present disclosure. The TOF exposure sequence 600 represents the timed activation of a group of TOF light sensors in the TOF sensor array (and the excitation of a corresponding group of emitters in the emitter array), and the image sensing sequence 602 represents the timed activation of a corresponding group of image light sensors in the image sensor array. Figure 6As shown, four groups of TOF light sensors can be activated at four respective times, for example at each time t(0)-t(3), and their activation durations can each be centered around their respective time t(0)-t(3). In some embodiments, the synchronized activation of the image sensor arrays can be configured such that their corresponding activations are also centered around time t(0)-t(3). Thus, the activation rate of the image sensor arrays can be equal to the activation rate of the TOF sensor arrays. However, it should be understood that the activation rate of the image sensor arrays can be significantly higher than the activation rate of the TOF sensor arrays, particularly in embodiments where the image sensor arrays have high shutter speeds, as will be discussed herein with respect to Figures 7A-7C Further discussion.

[0083] In some embodiments, the image sensor array can be controlled to modify the exposure time between each activation thereof, so that the exposure time can be adjusted so that the image light sensors receive the optimal amount of light. The exposure time can be defined by the duration of each activation. For example, the activation of the first, second, third, and fourth groups of image light sensors can have first, second, third, and fourth exposures 604, 606, 608, and 610, respectively. The brightness of the previous activation can be used to modify the exposure of subsequent activations. For example, if the previous activation was too bright, the exposure of the subsequent activation can be reduced, and vice versa. For example, if the previous activation was too bright, the exposure of the subsequent activation can be reduced, and vice versa. Figure 6 As shown, the image sensor controller may determine that the light sensed during activation of the first group of image light sensors was too bright; therefore, the exposure for the second group of image light sensors may be reduced 606. The image sensor controller may then determine that the light sensed during activation of the second group of image light sensors was too bright, and therefore increase the exposure for the third group of image light sensors 608. Thereafter, the image sensor controller may determine that the light sensed during activation of the third group of image light sensors was just right, and therefore maintain the exposure 610 the same for each subsequent activation until the entire image sensor array is activated.

[0084] As mentioned herein, embodiments are not limited to configurations in which the activation rate of the image sensor array is equal to the activation rate of the emitter array, and some embodiments may be configured with an image sensor array designed with a global shutter having a high shutter speed, where the activation rate of the image sensor array is greater than the activation rate of the emitter array. Figure 7A 7 is a timing diagram illustrating synchronization of an image sensor array with a TOF sensor array (and therefore with the firing of emitters of an emitter array) according to some embodiments of the present disclosure, wherein the activation rate of the image sensor array is much higher than the activation rate of the TOF sensor array. Activation sequence 700 represents the temporal activation of a group of TOF light sensors in the TOF sensor array, and image sensing sequence 702 represents the temporal activation of the entire image light sensor array in the image sensor array.

[0085] like Figure 6 As shown, the four groups of TOF light sensors can be activated at four independent times, for example at each time t(0)-t(3), and their activation durations can each be centered around the corresponding time t(0)-t(3). Figure 6 , the image sensing sequence 702 can have a significantly higher activation rate, such that the image sensor array is activated more times than the TOF sensor array in a given time period. Thus, not every activation of the image sensor array will be synchronized with a corresponding activation of the TOF sensor array. For example, the image sensor array may be activated four times between successive activations of the TOF sensor array, e.g. Figure 7A As shown (i.e. Figure 7A The frame rate of the image sensor array in the image sensor controller is five times faster than the frame rate of the TOF sensor array, which matches the rate at which the emitters are fired. The high activation rate of the image sensor array can result in a reduced exposure time per activation. However, the image sensor controller can be configured to utilize the additional activations of the image sensor array to improve the exposure of the captured image. For example, Figure 7B As shown, the image sensor controller can receive image data from three activations 706a-c to capture an image of the field of view during a single activation 704 of a group of TOF light sensors. The time at which the three activations 706a-c occur can be centered around the time t(1) at which the group of TOF light sensors is activated. Image data from a greater or lesser number of activations can be received to capture an image of the field of view. As an example, Figure 7C As shown, the image sensor controller can receive image data from two activations 708a-b and activations 706a-c, for a total of five activations, to capture an image of the field of view during a single activation 704 of a group of TOF light sensors. The time at which the five activations occur can be centered around the time t(1) at which the group of TOF light sensors is activated.

[0086] III. Enhanced Illumination System for Optical Distance Measurement

[0087] Some embodiments of the present disclosure relate to LIDAR sensors that can be used for obstacle detection and avoidance in autonomous vehicles, among other uses. Some specific embodiments relate to LIDAR sensors whose design features make the sensor manufacturing cost low enough and the reliability high enough, and achieve a small enough footprint for use in mass-market cars, trucks and other vehicles. For example, some embodiments include a group of vertical cavity surface emitting lasers (VCSELs) as an illumination source that emits radiation into a field, and include a single photon avalanche diode (SPAD) detector array as a group of TOF light sensors (detectors) that detect radiation reflected back from surfaces in the field. Using VCSELs as emitters and SPADs as detectors, multiple measurements can be performed simultaneously (i.e., the VCSELs can be excited at the same time), and a group of emitters and a group of TOF light sensors can be manufactured separately on a single chip using standard CMOS processes, which greatly simplifies the manufacturing and assembly process.

[0088] However, the use of VCSELs and SPADs in certain embodiments presents certain challenges that are overcome by various embodiments of the present disclosure. For example, VCSELs are much lower power than typical lasers used in existing LIDAR structures, and SPADs are much less efficient than typical detectors used in existing LIDAR structures. To address these challenges, as well as the challenges posed by simultaneously exciting multiple emitters, certain embodiments of the present disclosure include various optical components (e.g., lenses, filters, and aperture layers) that can work in conjunction with multiple SPAD arrays, each array corresponding to a different pixel (e.g., position within the field), as described herein. For example, as described herein with respect to Figure 1 As discussed, the optical system 128 of the light sensing module 108 may include a micro-light receiver layer ( Figure 1 ), for enhancing the light detected by the TOF sensor array 126, which may comprise an array of TOF light sensors, each of which may be a SPAD array.

[0089] Figure 8 8 is a simplified diagram showing a detailed side view of an exemplary solid-state electronic scanning LIDAR system 800 according to some embodiments of the present disclosure. Solid-state electronic scanning LIDAR system 800 may include a light detection system 801, a light emission system 803, and an image capture system 807. Light emission system 803 provides active illumination of at least a portion of a field in which system 800 is located using narrowband light 805. Light detection system 801 detects narrowband light emitted from light emission system 803 that has been reflected by objects within the field as reflected light 806. Image capture system 807 detects ambient light in the visible spectrum present in the portion of the field in which light emission system 803 emits light.

[0090] The light detection system 801 can represent the above reference Figure 1 The light detection system 136 discussed. The light detection system 801 may include an optical sensing system and a TOF sensor array. The optical sensing system may include a bulk receiver optics, an aperture layer, a collimating lens layer, and a filter layer; and the TOF sensor array may include an array of TOF light sensors, wherein each TOF light sensor may include one or more light detectors for measuring light. According to some embodiments, these components operate together to receive light from a field. For example, the light detection system 801 may include a bulk receiver optics 802 and a micro-optical receiver (Rx) layer 804. During operation, light rays 806 enter the bulk receiver optics 802 from multiple directions and are focused by the bulk receiver optics 802 to form a light cone 808. The micro-optical receiver layer 804 is positioned so that the aperture 810 coincides with the focal plane of the bulk receiver optics 802. In some embodiments, the micro-optical receiver layer 804 can be a two-dimensional array of micro-optical receiver channels 812, wherein each micro-optical receiver channel 812 is formed by a respective aperture 810, a collimating lens 814, and a TOF light sensor 816, which are positioned along the same axis in the direction of light transmission, for example, horizontally from left to right, as shown in FIG. Figure 8 As shown. In addition, each micro-optical receiver channel 812 can be configured in various ways to mitigate interference from stray light between TOF light sensors, which will be discussed further herein. During operation, each micro-optical receiver channel 812 measures light information of a different pixel (i.e., a position within the field).

[0091] At the focal point of the volume receiver optics 802, light rays 806 are focused and pass through apertures 810 in the aperture layer 811 and into corresponding collimating lenses 814. Each collimating lens 814 collimates the received light so that the rays all enter the filter at approximately the same angle, e.g., parallel to one another. The aperture and focal length of the volume receiver optics 802 determine the cone angle of the corresponding light rays focused at apertures 810. The aperture size and focal length of the collimating lenses 814 determine the degree of collimation allowed for the light, which in turn determines how narrow the bandpass can be achieved in the filter 818. During operation of the light detection system 800, the aperture layer can serve various functions. For example, (1) the aperture 810 can constrain the pixel field of view so that it has tight spatial selectivity despite large spacing on the TOF light sensor plane; (2) the aperture 810 can constrain the field of view to be similar or equal in size to the emitter field of view to efficiently utilize the emitter light; (3) the aperture can provide a small point source of light at the focal plane of the collimating lens to achieve tight collimation of the light before passing through the filter, where better collimation forms a tighter wavelength band that can pass through the filter, and (4) the aperture area of ​​the aperture layer surrounding each aperture can block stray light. In some embodiments, the collimating lens 814 is not included, and the passband of the bandpass filter is narrow.

[0092] Filter 818 blocks unwanted wavelengths of light. Interference-based filters often exhibit strong angular dependence in their performance. For example, a 1 nm wide bandpass filter with a center wavelength (CWL) of 900 nm at zero degrees of incidence might have a CWL of 898 nm at a fifteen-degree angle of incidence. Imaging systems typically use filters that are tens of nanometers wide to accommodate this effect, so the CWL shift is much smaller than the bandpass width. However, the use of micro-optical layer 804 allows all light to enter filter 818 at approximately the same angle of incidence, thereby minimizing CWL shift and allowing the use of very tight filters (e.g., less than 8 nm wide). TOF light sensor 816 generates a current or voltage in response to incident photons. In some embodiments, filter 818 is uniform across the entire array of micro-optical receiver channels 812, such that each individual micro-optical receiver channel 812 in the array receives light from the same wavelength range.

[0093] In some embodiments, the TOF light sensor 816 is located on the opposite side of the collimating lens 814 so that the light 806 first passes through the collimating lens 814 and the filter 818 before being exposed to the TOF light sensor 816. Each TOF light sensor 816 can be a plurality of light detectors, such as a micro-array of multiple single-photon avalanche detectors (SPADs). The micro-SPAD array can be manufactured on a single monolithic chip, thereby simplifying manufacturing. In some alternative embodiments, each TOF light sensor 816 can be a single light detector, such as a standard photodiode, an avalanche photodiode, a resonant cavity photodiode, or another type of light detector.

[0094] The light emitting system 803 may include a bulk emitter optics 820 and a light emitting layer 822 formed from a two-dimensional array of light emitters 824. Each light emitter 824 may be configured to generate a discrete beam of narrowband light. In some embodiments, the light emitting layer 822 is configured to selectively project the discrete beams of light through the bulk emitter optics 820 according to an illumination pattern that is sized and geometrically matched to the field of view of the receiver channels in the micro-optical receiver layer 804 across a range of distances from the light emitting system 803. The light emitters 824 may be any suitable light emitting device, such as a vertical cavity surface emitting laser (VCSEL) integrated on one or more monolithic chips, or any other type of laser diode. The light emitters 824 may generate narrowband cones of light 826 that are directed toward the bulk emitter optics 820, which may collimate the cones of light 826 and then output the collimated light as emission rays 805 toward a distant target within the field of view. In some embodiments, the bulk emitter optics 820 is image-space telecentric.

[0095] from Figure 8 As can be clearly seen from the illustration of parallel light rays 805 and 806, each micro-optic receiver channel 812 has a non-overlapping field of view beyond the threshold distance. Figure 8 As shown, each micro-optical receiver channel 812 includes an aperture from a plurality of apertures, a lens from a plurality of lenses, and a photodetector from a plurality of photodetectors, wherein the aperture of each channel defines a discrete field of view for pixels in the channel, the fields of view not overlapping beyond a threshold distance within the fields of view of other micro-optical receiver channels. In this way, each micro-optical receiver channel receives reflected light corresponding to a discrete location within the field that is not measured by any other micro-optical receiver channel in the micro-optical receiver layer 804.

[0096] In additional and alternative embodiments, light rays 805 from the light cone 826 are first focused by a micro-optical transmitter layer (not shown) onto a mid-plane in space before being directed by the bulk transmitter optics 820 toward a distant target to enhance the brightness and intensity of light emitted from the light emitting system 803. In such an embodiment, the light emitting system 803 and the light detection system 801 are configured such that each micro-optical transmitter channel (not shown) is paired with a corresponding micro-optical receiver layer 804, and their fields of view are aligned to overlap at a distance from the sensor, or their chief rays are parallelized. In yet other additional and alternative embodiments, the far-field light beam emitted by the light emitting system 803 has a size and divergence angle similar to the far-field field of view of each micro-optical receiver layer 804. Details of the light emitting system 803 with a micro-optical transmitter layer for enhancing the brightness and intensity of the output light will be discussed in detail below.

[0097] Because VCSELs are less powerful than typical lasers in existing LIDAR architectures, in some embodiments, the light emitting system 803 can be configured to improve the ability of the LIDAR system 800 to perform optical ranging functions. That is, the quality of the light emitted by the light emitting system 803 can be improved to improve optical ranging accuracy and efficiency. The quality of transmitted light for optical ranging and imaging purposes can be defined in terms of brightness and intensity. The brightness and intensity of the light emitted from the bulk transmitter optics 820 can be enhanced by modifying and / or implementing one or more optical transmitter layers.

[0098] Image capture system 807 may represent the above reference Figure 1 Image capture system 156 discussed. Image capture system 807 can include volumetric image receptor optics 830 and an image sensing layer 832 formed of a two-dimensional array of image light sensors 834. Each image light sensor 834 can be configured to detect ambient light in the visible wavelength spectrum present in a scene. Volumetric image receptor optics 830 can focus incoming ambient light 838 as optical code 836 onto image sensing layer 832 so that image light sensors 834 can detect ambient light 838. In some embodiments, image volumetric image receptor optics 830 is image-space telecentric. Image sensing layer 832 can be any suitable visible light sensor, such as a charge-coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) image sensor.

[0099] Figure 9is a simplified cross-sectional view of an exemplary enhanced light emission system 900 according to some embodiments of the present disclosure. The light emission system 900 may include a light emitter array 902 having light emitters 904, which may include, but are not limited to, any light emitting diode, laser diode, VCSEL, etc. for emitting light 913. A VCSEL is a semiconductor laser diode that emits a laser beam vertically from the top surface. Note that Figure 9 The linear array shown may be an array of emitters in any geometric form including but not limited to circular, rectangular, linear, or any other geometric shape.

[0100] The enhanced light emission system 900 may include an array of micro-optical transmitter channels 906 separated from the light emitter array 902 by an open space 918. Each micro-optical transmitter channel 908 may be paired with a corresponding receiver channel (e.g., receiver channel 512 in FIG. 5 ), and their field of view centers are aligned to overlap at a distance from the optical imager system. The array of micro-optical transmitter channels 906 may be formed by a substrate 919 sandwiched between a first optical surface 920 located on a side facing the light emitter array 902 and a second optical surface 921 located on an opposite side facing away from the light emitter array 902. The first optical surface 920 and the second optical surface 921 may both be configured as an array of convex micro-optical lenses, wherein each convex lens of the first optical surface 920 is configured to be optically aligned with a corresponding convex lens of the second optical surface 920 such that light transmitted through the first optical surface 920 may subsequently be transmitted through the second optical surface 921. As Figure 9 As shown, the corresponding convex lenses from the first optical surface 920 and the second optical surface 921 can face away from each other. In some embodiments, the convex lenses of the first optical surface 920 have a first optical power and the convex lenses of the second optical surface 921 have a second optical power different from the first optical power. For example, the second optical power can be greater than the first optical power, such that the focal length of the second optical power is shorter than the focal length of the first optical power. The substrate 919 can be formed of any suitable material that is transmissive within the wavelength range of the light emitter 904, such as silicon, silicon dioxide, borosilicate glass, polymers, etc. The first optical surface 920 and the second optical surface 921 can be formed of a transparent polymer printed on respective opposing surfaces of the substrate 919.

[0101] In some embodiments, micro-optical transmitter channel array 906 can be formed from a monolithic array of micro-optical transmitter channels 908. Each micro-optical transmitter channel 908 can include a first convex lens from first optical surface 920, a corresponding second convex lens from second optical surface 921, and a corresponding portion of substrate 919 located between the two convex lenses. Each micro-optical transmitter channel 908 can correspond to a respective light emitter 904, such that light output from light emitter 904, during operation, first passes through the first convex lens, passes through the corresponding area of ​​substrate 919, and then passes through the second convex lens.

[0102] Once the light exits the second convex lens of the second optical surface 921, it forms a micro-spot image 910, which is a true image of the corresponding light emitter 904, but a reduced-size image of the corresponding light emitter 904. In some embodiments, the micro-spot image 910 is located between the micro-optical transmitter channel array 906 and the bulk transmitter optics 914. For example, the micro-spot image 910 can be formed within a corresponding aperture of the aperture layer 909. Each aperture can be a pinhole in a reflective or opaque layer, where the emitted light is focused to form the micro-spot image 910. It should be understood that the aperture layer 909 is optional, and the light-enhancing capabilities of the micro-optical transmitter channel array 906 can be achieved without it. In such an embodiment, the micro-spot image 910 can be formed at the focal plane of the second convex lens of the second optical surface 921. From there, continuing away from the light emitters and micro-optical channels, the light forms a light cone 912 that leads to the bulk transmitter optics 914.

[0103] According to some embodiments of the present disclosure, the divergence of emitted light 913 can be less than the divergence of light cone 912. This difference in divergence can be generated by micro-optical transmitter channel 908, and in particular, by the optical power of second optical surface 921. Because the divergence of light exiting micro-optical transmitter channel 908 is greater than the divergence of light 913 emitted from light emitter 904, micro-spot image 910 can be a true image of light emitter 904, but much smaller than light emitter 904 and having the same number of photons as emitted light 913. The final light cone 912 formed after forming the true spot image is then projected into the field as a discrete beam 925 for each light emitter 904 after passing through bulk transmitter optics 914. The final light emitted from light emitting system 900 is a highly collimated beam 925 with a small cross-sectional area, enabling light emitting system 900 to output light with enhanced brightness and intensity. In contrast, a system without an array of micro-optical channels, but instead having an array of light emitters 902 at the focal plane of the bulk emitter optics 914, would produce light beams with much less collimation and therefore would have a larger cross-sectional area in the far field.

[0104] Note that the bulk transmitter optics 914 can include a single lens or a lens array, where two or more lenses together form the bulk transmitter optics 914. The use of multiple lenses in the bulk transmitter optics 914 can increase the numerical aperture, reduce the RMS spot size, flatten the image plane, increase telecentricity, or otherwise improve the performance of the bulk transmitter optics 914. Note also that for some embodiments, the light cones 912 can overlap to form a light cone overlap region 916.

[0105] The volume emitter optics 914 are located in front of the micro-optics and the emissive layer so that the focal plane of the volume imaging optics coincides with the micro-spot image 910. The volume emitter optics 914 accepts the diverging light cone 912 and outputs a collimated beam. Its numerical aperture can be at least large enough to capture the full range of angles in the diverging light cone. In addition, the volume emitter optics 914 can be telecentric in image space because the light cones 912 exiting the micro-optical layer can all be parallel (rather than having their central axes aligned with the center of the volume optics). In one embodiment, the light can exit the volume emitter optics 914 in a nearly collimated manner. Note that the quality of the beam collimation is related to the size of the "emitting object" (micro-spot image 910) at the focal plane. Because the size of this "emitting object" has been reduced by using the micro-optical stack, a better collimation angle is achieved than by simply imaging the emitting object directly.

[0106] although Figure 9 An enhanced light emission system is shown having an array of micro-optical channels formed by a substrate sandwiched between first and second optical surfaces and positioned a distance away from the light emitter array via an open space to increase the brightness and intensity of light output by the light emission system, but embodiments are not limited to this configuration. Rather, other embodiments do not necessarily require the implementation of an open space or two optical surfaces, as discussed in further detail in related U.S. patent application Ser. No. 15 / 979,235, filed May 14, 2018, entitled “Optical Imaging Transmitter with Brightness Enhancement,” which is incorporated herein by reference in its entirety for all purposes.

[0107] While the embodiments discuss an image capture module for an electronically scanned array of emitters, the embodiments are not limited to such a configuration. In some embodiments, the image capture module may be implemented in a LIDAR system that mechanically scans the light emission, such as using a mirror galvanometer. Such a system may use a mirror galvanometer to scan a two-dimensional array of laser beams along a predetermined scanning pattern. Thus, the light sensing module and the image capture module may be configured to capture an image of the scene that coincides with the scanning pattern of the laser beam.

[0108] In some embodiments, multiple electronically scanning LIDAR units according to the present disclosure can work together to provide a larger field of view than a single unit. For example, Figure 10 An implementation 1000 is shown in which a solid-state electronic scanning LIDAR system 1002a-d is implemented in an exterior area of ​​a road vehicle 1005, such as an automobile, according to some embodiments of the present disclosure; Figure 11 An embodiment 1100 is shown in which solid-state electronic scanning LIDAR systems 1102a-b are implemented on top of a road vehicle 1105, in accordance with some embodiments of the present disclosure. In each embodiment, the number of LIDAR systems, the placement of the LIDAR systems, and the field of view of each LIDAR system can be selected to obtain most, if not all, of a 360-degree field of view of the vehicle's surroundings. The automotive embodiment of the LIDAR system is chosen for illustration purposes only, and the sensors described herein can be used in other types of vehicles, such as ships, airplanes, trains, and the like, as well as in various other applications that use 3D depth images, such as medical imaging, mobile phones, augmented reality, geodesy, geomatics, archaeology, geography, geology, geomorphology, seismology, forestry, atmospheric physics, laser guidance, airborne laser swath mapping (ALSM), and laser altimetry.

[0109] Referring to FIG1000 , solid-state electronic scanning LIDAR systems 1002a-d can be mounted on the exterior areas of a vehicle, near the front and rear fenders. LIDAR systems 1002a-d can each be positioned at a respective corner of vehicle 1005, such that they are positioned near the outermost corners of vehicle 1005. This allows LIDAR systems 1002a-d to better measure the distances between vehicle 1005 and objects in the field at regions 1006a-d, as well as capture images of objects in the field at regions 1006a-d. Emitted light and sensed reflected emitted light for ranging / 3D imaging purposes are shown as solid lines, while reflected ambient light for imaging (i.e., 2D imaging) purposes is shown as dashed lines. Each solid-state LIDAR system can face a different direction (with partially overlapping and / or non-overlapping fields of view between units) to capture a larger composite field of view than each unit could capture on its own.

[0110] like Figure 10As shown, objects within the scene can reflect a portion of a light pulse 1010 emitted from LIDAR Tx module 1008. One or more reflected portions 1012 of light pulse 1010 then propagate back to LIDAR system 1002a and can be received by Rx module 1009. Additionally, ambient light 1014 reflected from objects within the scene can propagate to LIDAR system 1002a and be received by IRx module 1011. Rx module 1009 and IRx module 1011 can be housed in the same housing as Tx module 1008. As discussed herein, electronically scanning LIDAR systems 1002a-d can electronically scan a scene to capture an image of the scene. Thus, LIDAR system 1002a can scan between points 1020 and 1022 to capture objects in region 1006a, and similarly for systems 1002b-d and regions 1006b-d.

[0111] Although Figure 10 Four solid-state electronic scanning LIDAR systems are shown mounted at the four corners of the vehicle, but embodiments are not limited to this configuration. Other embodiments may have fewer or more solid-state electronic scanning LIDAR systems mounted in other areas of the vehicle. For example, the electronic scanning LIDAR system may be mounted on the roof of the vehicle, such as Figure 11 In such an embodiment, the electronic scanning LIDAR systems 1102a-b can have a higher vantage point to better observe the area 1107a-b surrounding the vehicle 1105. In some embodiments, scanning can be performed by other means, such as chip-based beam steering technology, for example, by using a microchip that employs one or more MEMS-based reflectors, such as a digital micromirror (DMD) device, a digital light processing (DLP) device, etc.

[0112] As mentioned herein, the number of LIDAR systems, the locations of the LIDAR systems, and the field of view of each LIDAR system can be selected to obtain a 360-degree field of view of most, if not all, of the vehicle's surroundings. Thus, each LIDAR system 1002a-d can be designed to have a field of view of approximately 90 degrees, so that when all four systems 1020a-d are implemented, all or most of the 360-degree field of view around the vehicle 1005 can be observed. In embodiments where each LIDAR system 1002a-d has a field of view of less than 90 degrees (e.g., a 45-degree field of view), additional LIDAR systems can be included as needed to expand the field of view to achieve a combined field of view, as may be required for a particular implementation.

[0113] Although the disclosure has been described with respect to specific embodiments, it should be understood that the disclosure is intended to cover all modifications and equivalents coming within the scope of the following claims.

Claims

1. A solid-state optical system comprising: an emitter array comprising a plurality of light emitters configured to emit light at an operating wavelength into a field external to the optical system; a first sensor array comprising a first plurality of light sensors configured to detect emitted light reflected from the field, the first sensor array having a field of view substantially the same as a field of view of the emitter array; an optical filter disposed between the field and the first sensor array and operable to pass a narrowband of light including the operating wavelength to the first sensor array; a second sensor array comprising a second plurality of light sensors configured to detect ambient light in the field, the second sensor array having a field of view larger than the field of view of the first sensor array; an emitter controller coupled to the emitter array and configured to activate the plurality of light emitters in each emission cycle by activating a subset of the plurality of light emitters at a time; a first sensor controller coupled to the first sensor array and configured to synchronize readout of individual light sensors within the first sensor array concurrently with excitation of corresponding light emitters in the emitter array such that each light emitter in the emitter array can be activated and each light sensor in the first sensor array can be readout during the emission cycle; and A second sensor controller is coupled to the second sensor array and is configured to read out at least a portion of the second sensor array's field of view that overlaps with the emitter array's field of view to capture an image representative of the field during the emission cycle.

2. The solid-state optical system of claim 1, wherein the operating wavelength is an infrared or near-infrared wavelength.

3. A solid-state optical system according to claim 1, wherein in each case where the emitter controller activates a subset of the plurality of light emitters, the second sensor controller activates and reads out a subset of the second plurality of light sensors that share the field of view of the activated subset of light emitters.

4. A solid-state optical system according to claim 1, wherein in each case where the emitter controller activates a subset of the plurality of light emitters, the second sensor controller activates the entire second sensor array and reads out a subset of the second plurality of light sensors that share the field of view of the activated subset of light emitters.

5. The solid-state optical system of claim 1 , wherein each photosensor in the second sensor array is configured to be individually activated, and the second sensor controller is configured to sequentially synchronize activation of groups of photosensors from the second plurality of photosensors with excitation of corresponding groups of light emitters.

6. The solid-state optical system of claim 5, wherein a duration of a first activation of a first group of light sensors from the second plurality of light sensors is different from a duration of a second activation of a second group of light sensors from the second plurality of light sensors.

7. A solid-state optical system according to claim 1, wherein the entire second sensor array is configured to be activated together, and the second sensor controller is configured to synchronize the activation of the entire second sensor array and the readout of corresponding groups of light sensors from the second plurality of light sensors with the excitation of corresponding groups of light emitters in sequence.

8. The solid-state optical system of claim 7, wherein the activation rate of the second sensor array is higher than the activation rate of the emitter array.

9. The solid-state optical system of claim 8, wherein the second sensor controller is configured to read out image data from more than one activation of the first sensor array and associate the readout image data with activation of corresponding light emitter groups.

10. The solid-state optical system of claim 1, wherein the field of view of the second sensor array is larger than the field of view of the emitter array.

11. The solid-state optical system of claim 1 , further comprising an aperture layer having a plurality of apertures, and wherein the aperture layer and the first sensor array are arranged to form a plurality of receiver channels, each receiver channel of the plurality of receiver channels comprising an aperture from the plurality of apertures and a light sensor from the first sensor array, the apertures defining a field of view of the light sensor in the receiver channel.

12. The solid-state optical system of claim 1, wherein activation of light sensors in the second plurality of light sensors is centered in time with respect to respective excitations of corresponding light emitters.

13. The solid-state optical system of claim 1 , wherein each of the first plurality of light sensors has a discrete field of view in the field that does not overlap with the fields of view of other of the first plurality of light sensors beyond a threshold distance from the optical system, and wherein the emitter array is configured to emit a light beam into the field according to an illumination pattern that substantially matches the field of view of the first sensor array in size and geometry over a range of distances from the system.

14. A solid-state optical system comprising: body transmitter optics; an emitter array comprising a plurality of light emitters configured to emit discrete light beams at an operating wavelength through bulk emitter optics into a field external to the optical system according to an illumination pattern; first body receiver optics; a first sensor array comprising a first plurality of light sensors operable to detect light emitted from the emitter array and reflected back from a field external to the solid-state optical system by the first bulk receiver optics, wherein each light sensor of the first plurality of light sensors has a discrete field of view within the field that does not overlap with the field of view of other light sensors of the first plurality of light sensors beyond a threshold distance from the optical system, and wherein the field of view of the first sensor array substantially matches in size and geometry the illumination pattern of the emitter array over a range of distances from the system; an optical filter disposed between the field and the first sensor array and operable to pass a narrowband of light including the operating wavelength to the first sensor array; second body receiver optics; a second sensor array comprising a second plurality of light sensors configured to detect ambient light in the visible spectrum received from the field through the second body receiver optics, wherein the second sensor array has a field of view that is larger than a field of view of the first sensor array; an emitter controller coupled to the emitter array and configured to activate each emitter of the plurality of light emitters in each emission cycle by activating a subset of the plurality of light emitters at a time; a lidar sensor controller coupled to the first sensor array and configured to synchronize readout of individual light sensors within the first plurality of light sensors concurrently with excitation of corresponding light emitters in the emitter array such that each light emitter in the emitter array can be activated and each light sensor in the first sensor array can be readout through the emission cycle; and An image sensor controller coupled to the second sensor array and configured to read out at least a portion of the second sensor array's field of view that overlaps with the emitter array's field of view to capture an image representative of the field during the emission cycle.

15. A solid-state optical system according to claim 14, wherein in each case where the emitter controller activates a subset of the plurality of light emitters, the image sensor controller activates and reads out a subset of the second plurality of light sensors that share the field of view of the activated subset of light emitters.

16. The solid-state optical system of claim 14 , wherein in each instance in which the emitter controller activates a subset of the plurality of light emitters, the image sensor controller activates all of the second plurality of light sensors but reads out only a subset of the second plurality of light sensors that share a field of view of the activated subset of light emitters.

17. The solid-state optical system of claim 14 , wherein groups of light sensors in the second plurality of light sensors are capable of being independently activated, and the image sensor controller is operable to simultaneously activate and synchronize the groups of light sensors in the second plurality of light sensors with the excitation of corresponding groups of light emitters in a predetermined sequence.

18. The solid-state optical system of claim 14 , wherein all of the second plurality of photosensors are activated simultaneously, the image sensor controller is configured to synchronize activation of the entire second photosensor array and readout of corresponding groups of photosensors in the second plurality of photosensors sequentially and simultaneously with excitation of corresponding groups of light emitters, and the activation rate of the second sensor array is higher than the activation rate of the emitter array.

19. The solid-state optical system of any one of claims 1 to 18, wherein the emitter array comprises a plurality of vertical cavity surface emitting lasers (VCSELs), and each photosensor in the first sensor array comprises a plurality of single photon avalanche diodes (SPADs).

20. A solid-state optical ranging device comprising: an emitter array comprising a plurality of optical emitters aligned to emit electromagnetic radiation at an operating wavelength according to an illumination pattern into a field external to the optical ranging device; a plurality of depth light sensors arranged in a first sensor array having a first resolution and aligned to detect emitted electromagnetic radiation within the illumination pattern reflected from the field, wherein each optical emitter in the emitter array is paired with a depth light sensor in the first sensor array, and the emitter array and the first sensor array have substantially similar fields of view beyond a threshold distance such that each optical emitter and corresponding paired depth light sensor look toward substantially the same point in the field, and wherein each depth light sensor includes a plurality of single photon avalanche diodes (SPADs) operable to detect photons emitted from its corresponding optical emitter and after reflection from a surface within the field; an optical filter disposed between the field and the first sensor array and operable to pass narrowband electromagnetic radiation including the operating wavelength of the emitter array to the plurality of depth light sensors in the first sensor array; a plurality of image light sensors arranged in a second sensor array having a second resolution higher than the first resolution, the second sensor array configured to detect ambient light in the visible spectrum from within the field, wherein the second sensor array has a field of view greater than the field of view of the first sensor array; a first sensor controller coupled to the first sensor array and configured to activate individual depth light sensors within the first sensor array in synchronization with excitation of corresponding optical emitters in the emitter array; and a second sensor controller coupled to the second sensor array and configured to activate image light sensors in the second sensor array to capture an image of the field including the illumination pattern of the emitter array, wherein activation of the image light sensors in the second sensor array is synchronized with excitation of optical emitters in the emitter array and activation of depth light sensors in the first sensor array.

Images