Accurate Optical Detector Measurements for LIDAR

Abstract

The present disclosure relates to accurate light detector measurements for LIDAR. An optical ranging system may include a laser device and an imaging device having a light sensor. The laser device illuminates a scene with laser pulse radiation, which is reflected from objects in the scene. The reflection can vary greatly depending on the shape and reflectivity of the reflecting surface. The signal measured by the light sensor can be filtered using several matched filters designed based on the curves of different reflected signals. The optimal matched filter can be identified, and thus information about the reflecting surface and accurate ranging information can be obtained. The laser pulse radiation can be emitted in decoded pulses by allowing weights for different detection intervals. Other enhancements include interleaving the laser pulses and changing the operating state of the light detectors of the pixel sensor, as well as efficient signal processing using a sensor chip that includes processing circuitry and the light sensor.

Description

[0001] This application is a divisional application of the Chinese patent application with application date of March 1, 2018, application number 201880026495.1, and invention name “Accurate Light Detector Measurement for LIDAR”. Background Art

[0002] Light Detection And Ranging (LIDAR) systems are used for object detection and ranging, for example, in vehicles such as cars, trucks, and boats. LIDAR systems are also used in mobile applications (e.g., for facial recognition), home entertainment (e.g., capturing gestures for video game input), and augmented reality. LIDAR systems measure the distance to an object by illuminating a landscape with pulses from a laser and then measuring the time it takes for photons to travel to the object and return after reflection, as measured by the LIDAR system's receiver. The detected signal is analyzed to detect the presence of reflected signal pulses amidst background light. The distance to the object can be determined based on the time of flight from the emission of the pulse to the reception of the corresponding reflected pulse.

[0003] Providing robust distance accuracy to a few centimeters under all conditions can be difficult, especially at the economic cost of LIDAR systems. Promising new detector technologies such as single photon avalanche diodes (SPADs) are attractive but have significant drawbacks when used to measure time of flight and other signal characteristics, particularly over a wide range of environmental conditions and target distances due to their limited dynamic range.

[0004] LIDAR systems would benefit from more accurate methods of detecting reflected laser pulses and measuring their time of flight under varying real-world conditions. SPAD-based LIDAR systems require novel approaches to overcome their inherent limitations before they can become viable options for economical, long-range, accurate 3D imaging. It would also be desirable to have two or more LIDAR devices operating in close proximity without interfering with each other. It would also be desirable to have LIDAR systems operate in an energy-efficient manner without sacrificing accuracy. Summary of the Invention

[0005] Various embodiments can address the aforementioned issues of LIDAR systems. For example, a set of matched filters in a light detection and ranging receiver can be tuned so that each filter matches one of a set of expected signal profiles (pulse shapes), so that for each signal profile analyzed, a best-fit filter can be identified. Each of the expected signal profiles can nominally be derived from the same template signal profile, but differ due to distortion imposed by the digitization process, external environmental factors, or both. In an active illumination sensor such as light detection and ranging, the template signal can correspond to the temporal shape of the outgoing illumination (generally a pulse or pulse train) from the sensor. Applying an appropriate matched filter to the return signal can improve the probability of correctly detecting the reflected signal, the accuracy of temporally locating the reflected signal, or determining other properties of the reflected signal, which in turn can yield information about the target from which the signal was reflected.

[0006] Therefore, some embodiments can correct for SPAD-induced distortion while providing a better understanding of signal and target properties. Single-photon avalanche diodes (SPADs) can impose different levels of distortion on optical signals depending on signal power and time distribution, an effect known as "pile-up". In a pile-up scenario, many SPADs, all acting as a single pixel, can be triggered on the leading edge of a powerful reflected signal pulse, thereby reducing the number of SPADs available for triggering on the trailing edge of the reflected signal because an increasing percentage of SPADs within a pixel remain in a dead-time state after their initial triggering. In weaker signal scenarios, a more uniform number of SPADs are triggered over the duration of the reflected signal pulse, and the reflected pulse shape is more accurately digitized. By running multiple matched filter curves, each tuned to a different degree of signal pile-up, the filter curve that best matches the received signal can achieve more consistent signal detection and estimate a more accurate reception time of the reflected pulse. More accurate time estimates can directly improve the accuracy of distance (ranging) measurements. Further accuracy can be achieved by using a second set of interpolation filters with a curve that best matches the received signal.

[0007] As another example, the transmitted pulses can be coded so that the cumulative signal (e.g., a histogram of the triggered light detector) has a pattern with desired properties (e.g., autocorrelation properties). The cumulative signal can correspond to multiple pulse trains, each having one or more pulses, where each pulse train can correspond to a different time interval (e.g., a pulse train is transmitted and detected before the next pulse train is transmitted). Decoding can be achieved by assigning different weights (e.g., positive and negative) to the pulse trains sent during different time intervals. Such weighting can result in the cumulative signal being a Barker code or a more complex orthogonal code. Such decoded pulses can reduce interference from adjacent optical ranging systems because each pulse can use a different code. Furthermore, certain codes can provide high accuracy in detecting the temporal position of the received pulses, such as using a matched filter that provides a positive peak with a negative sidelobe.

[0008] As another example, different pulse trains can be offset from each other. For example, a pulse train can be offset relative to a previous pulse train (e.g., less than the temporal resolution of the cumulative signal) (e.g., for interleaving), thereby providing increased temporal resolution. When the cumulative signal is a digitized signal consisting of a histogram spanning different pulse trains, the offset can interleave the pulses so that they are recorded in different time intervals of the histogram. This interleaving allows distinguishing between pulses detected at the beginning of a time interval and pulses detected at the end of the time interval, thereby obtaining increased temporal resolution.

[0009] As another example, the operation of a set of photodetectors (e.g., SPADs) of a light sensor can be altered in response to a determined intensity level of previously detected photons. The determined intensity level can be used as an estimate for certain future measurements (e.g., at similar angular positions near or rotating in time for a LIDAR system). In various embodiments, the change in operating state can improve power usage (e.g., by disconnecting or reducing the power of photodetectors that are not expected to provide useful signals) or improve the dynamic range of the photodetectors (e.g., by changing the attenuation level so that the photodetectors provide useful signals). For example, different photodetectors may have different sensitivity levels for detecting photons; for example, some photodetectors may detect photons at a higher frequency than others. If a strong signal is received (e.g., from a background light source), highly sensitive photodetectors (e.g., those that provide strong signal levels, such as absolute responsivity or higher responsivity relative to other nearby photodetectors) can be disconnected; such photodetectors will always trigger, and therefore their signals will not correspond to any detected pulses reflected from an object. Similarly, photodetectors with weak sensitivity can be disconnected when the signal is weak, for example, because they will not trigger for weak signals. In this way, energy can be saved by reducing the operation of photodetectors that will not provide a signal meaningful for a particular time and / or location, and the detector can extend its distortion-free dynamic range.

[0010] As another example, a single integrated circuit may include a light sensor and signal processing components. For example, the timing circuit of the integrated circuit may determine when a photon is detected, and the histogram circuit may accumulate the number of detected photons over a plurality of detection time intervals measured.

[0011] These and other embodiments of the present invention are described in detail below.For example, other embodiments are directed to systems, apparatus, and computer-readable media associated with the methods described herein.

[0012] A better understanding of the nature and advantages of embodiments of the present invention may be obtained by reference to the following detailed description and accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1A and 1B An automotive light ranging device, also referred to herein as a LIDAR system, is shown in accordance with some embodiments.

[0014] Figure 2 A block diagram of an exemplary LIDAR device for implementing various embodiments is shown.

[0015] Figure 3 The operation of a typical LIDAR system that may be improved by embodiments is described.

[0016] Figure 4 An illustrative example of a light transmission and detection process for an optical ranging system is shown in accordance with some embodiments.

[0017] Figure 5 Various stages of a sensor array and associated electronics are shown according to an embodiment of the present invention.

[0018] Figure 6 A histogram according to an embodiment of the present invention is shown.

[0019] Figure 7 An accumulation of histograms over multiple bursts for a selected pixel is shown in accordance with an embodiment of the present invention.

[0020] Figure 8 Three different types of digitized signal curves output from a light sensor according to an embodiment of the present invention are shown.

[0021] Figure 9 A series of positions for applying a matched filter to an original histogram is shown according to an embodiment of the present invention.

[0022] Figures 10A-10C Shown are the types of matched filter responses (filtered outputs) produced by different decoded pulse types according to an embodiment of the invention.

[0023] Figure 11 is an exemplary schematic diagram illustrating the operation of a coded pulse optical system (CPOS) according to an embodiment of the present invention.

[0024] Figure 12 Two binary decoded pulse intensities and their difference are shown providing decoded pulse trains with positive and negative values according to an embodiment of the present invention.

[0025] Figure 13 A coded pulse optical system (CPOS) according to an embodiment of the present invention is shown.

[0026] Figure 14 is a flow chart illustrating a method 1400 of using decoded pulses in an optical measurement system according to an embodiment of the invention.

[0027] Figure 15 A plurality of curve filters applied to an original histogram according to an embodiment of the present invention are shown.

[0028] Figure 16 The application of different second-order curve filters to the filtered output from the first-order filter is shown according to an embodiment of the present invention.

[0029] Figure 17is a process flow illustrating the use of a two-stage filter applied to a moderately piled-up signal according to an embodiment of the present invention.

[0030] Figure 18 The application of different interpolation filters to the original histogram according to an embodiment of the present invention is shown.

[0031] Figure 19 is a diagram illustrating a two-stage filtering scheme using multiple coarse filters according to an embodiment of the present invention.

[0032] Figure 20 Shown are the raw histogram, matched filter, and corresponding filter output resulting from a decoded pulse, according to an embodiment of the present invention.

[0033] Figure 21 Shown are maximum windower results according to an embodiment of the present invention.

[0034] Figure 22 The application of multiple coarse curve filters with different widths according to an embodiment of the present invention is shown.

[0035] Figure 23 A filtering optical system according to an embodiment of the present invention is shown.

[0036] Figure 24 is a flowchart illustrating a method for performing ranging using a curve filter of an optical ranging system according to an embodiment of the present invention.

[0037] Figure 25A A single rectangular pulse is shown that is commonly used in LIDAR systems to illuminate a scene. Figure 25B The reflected pulse is shown with some noise. Figure 25C A high pile-up signal is shown detected at the rising edge of a reflected pulse in accordance with an embodiment of the present invention.

[0038] Figure 26 The SPAD signal is shown in the resulting histogram according to an embodiment of the present invention.

[0039] Figure 27A and 27B Two examples of pulses in different pulse trains being delayed relative to each other are shown, the delay resulting in a histogram where only one bin has significant values.

[0040] Figure 28A and 28B An example is shown of staggering the emitted pulses of different pulse trains so that the detected high pile-up pulses span multiple time intervals according to an embodiment of the present invention.

[0041] Figure 29is a flowchart illustrating a method for performing ranging using staggered pulses in an optical ranging system according to an embodiment of the present invention.

[0042] Figure 30A A conventional arrangement of 16 light detectors 3002 (eg, SPADs) forming a single pixel light sensor is shown in accordance with an embodiment of the present invention. Figure 30B An arrangement of 16 photodetectors with different attenuation levels is shown according to an embodiment of the present invention. Figure 30C An arrangement of 16 photodetectors with different attenuation levels and different effective operations is shown according to an embodiment of the present invention.

[0043] Figure 31 Diagram showing different detector arrangements for a pixel sensor under different lighting conditions at different angles according to an embodiment of the invention.

[0044] Figure 32 An arrangement of light detectors at the periphery having different operating states compared to light detectors in the central area is shown according to an embodiment of the present invention.

[0045] Figure 33 A configurable optical system according to an embodiment of the present invention is shown.

[0046] Figure 34 is a flow chart illustrating a method for performing ranging using a configurable optical ranging system according to an embodiment of the present invention.

[0047] Figure 35 A compact optical system of an optical ranging system according to an embodiment of the present invention is shown.

[0048] the term

[0049] The term "ranging," particularly when used in the context of methods and apparatus for measuring the environment or assisting in vehicle operation, may refer to determining the distance or distance vector from one location or position to another. "Optical ranging" may refer to a class of ranging methods that utilize electromagnetic waves to perform a ranging method or function. Thus, an "optical ranging device" may refer to a device for performing an optical ranging method or function. "Light detection and ranging" or "LIDAR" may refer to a class of optical ranging methods that measures the distance to a target by illuminating the target with a pulsed laser and thereafter measuring the reflected pulses with a sensor. Thus, a "light detection and ranging device" or "LIDAR system" may refer to a class of optical ranging devices for performing an optical detection and ranging method or function. An "optical ranging system" may refer to a system that includes at least one optical ranging device (e.g., an optical detection and ranging device). The system may also include one or more other devices or components in various arrangements.

[0050] A "pulse train" may refer to one or more pulses emitted together. The emission and detection of a pulse train may be referred to as a "shot." A shot may occur during a "detection time interval" (or "detection interval").

[0051] A "measurement" may comprise N pulse trains emitted and detected over N excitations, each excitation lasting a detection interval. The entire measurement may be over a measurement interval (or just a "measurement interval"), which may be equal to N detection intervals of the measurement or longer, for example when a pause occurs between detection intervals.

[0052] A light sensor can convert light into an electrical signal. A light sensor can include multiple light detectors, such as single-photon avalanche diodes (SPADs). A light sensor can correspond to a specific resolution pixel in distance measurement.

[0053] "Histogram" can refer to any data structure that represents a series of values over time, such as discrete values over a time interval. A histogram can have values assigned to each time interval. For example, a histogram can store a counter of the number of photodetectors that are activated during a specific time interval in each of one or more detection intervals. As another example, a histogram can correspond to the digitization of an analog signal at different times. A photodetector can be in "active operation" when it generates a signal and the signal is used to generate a histogram. A histogram can contain a signal (e.g., a pulse) and noise. Therefore, a histogram can be viewed as a combination of signal and noise as a photon time series or photon flux. A raw / digitized histogram (or cumulative photon time series) can contain the signal and noise digitized in a memory without filtering. A "filtered histogram" can refer to the output after the raw histogram has passed through a filter.

[0054] The emitted signal / pulse may refer to an undistorted "nominal," "ideal," or "template" pulse or pulse train. The reflected signal / pulse may refer to a reflected laser pulse from an object and may be distorted. The digitized signal / pulse (or raw signal) may refer to the digitized result of the detection of one or more pulse trains from a detection interval stored in a memory, and may therefore be equivalent to a portion of a histogram. The detected signal / pulse may refer to the location in the memory where the signal was detected. The detected pulse train may refer to the actual pulse train found by the matched filter. The expected signal curve may refer to the shape of the digitized signal resulting from a specific emitted signal with specific distortion in the reflected signal. DETAILED DESCRIPTION

[0055] CROSS-REFERENCE TO RELATED APPLICATIONS

[0056] This application claims priority to and is a non-provisional application of U.S. Provisional Application No. 62 / 465,310, filed on March 1, 2017, entitled “System And Method Of Object Detection Using Coded Optical Pulses,” and U.S. Provisional Application No. 62 / 596,002, filed on December 7, 2017, entitled “Accurate Photo Detector Measurements For Lidar,” the entire contents of which are incorporated herein by reference for all purposes.

[0057] The present disclosure relates generally to the field of object detection and ranging, and more specifically to the use of time-of-flight optical receiver systems for applications such as real-time 3D mapping and object detection, tracking, and / or classification. Various improvements can be achieved with various embodiments of the present invention. Such improvements can include increased accuracy, reduced noise, and increased energy efficiency.

[0058] To increase accuracy, some embodiments may account for nonlinear distortion in the measured data due to fundamental operating characteristics (e.g., dead time and after-pulse). For example, embodiments may account for varying rates of photon detection in consecutive time intervals. For example, after detecting a photon, a single-photon avalanche diode (SPAD) has a period of dead time (e.g., 1-100 ns), after which the SPAD cannot detect new photons. Therefore, a strong pulse may cause many SPADs in a photosensor to activate immediately, but the number of activated SPADs subsequently decreases during the dead time, resulting in a digitized signal profile that differs from a normal pulse. This change in the photon detection rate can cause errors in determining the precise time of receipt of a reflected pulse. For example, determining the time of receipt of a light pulse can be difficult because the signal is distorted from a roughly rectangular pulse, making it difficult to determine the distance to an object. Such varying rates of photon detection also occur with other types of photodetectors. To account for such varying profiles, embodiments may use filters with different profiles and select the filter that best matches the profile (e.g., for determining the precise time of receipt of the reflected pulse, rather than just the peak of the digitized signal).

[0059] To reduce noise (e.g., from background light or interference from nearby LIDAR devices), embodiments may transmit different decoded pulse patterns (pulse trains) at different detection time intervals. Different decoded pulse patterns may be assigned different weights, thereby providing easily identifiable patterns of accumulated signals that may have desirable properties. For example, a first transmitted pulse train may have a pulse pattern of {1, 1, 0, 0, 1} over five time units (time intervals). A second transmitted pulse train may have a pulse pattern of {0, 0, 1, 1, 0}. The first pulse train may be weighted by +1, and the second pulse train may be weighted by -1, thereby providing an accumulated signal having a pattern of {1, 1, -1, -1, 1} for a given measurement. More complex weighting schemes may be used, such as non-integer weights and weighting in different dimensions. In addition, different pulse patterns may be implemented by adding delays, for example, {1, 1, 0, 0} may become a different pulse pattern of {0, 1, 1, 0}.

[0060] To further increase accuracy, particularly when a signal pulse occupies only a single histogram bin (e.g., to achieve a higher resolution than the time bin), the measurement process can interleave consecutive pulse trains (e.g., with the same offset for each consecutive pulse train), thereby ensuring that the cumulative histogram of digitized pulses from different pulse trains spans more than one time bin. In this way, it can be determined whether the first pulse was received at the beginning, middle, or end of a time bin. For example, the measurement resolution (e.g., the width of a time bin) can be 1 ns, and ten consecutive pulse trains can each be delayed by 100 picoseconds relative to the previous pulse train. Thus, if two consecutive time bins have approximately the same detected value (e.g., 5 pulses detected in a first time bin and 5 pulses detected in a second time bin), the first pulse will have a leading edge that arrives in the middle of the first time bin.

[0061] In order to increase energy efficiency and reduce pile-up induced distortion, the operation of a set of photodetectors can be changed based on the intensity level of previously detected photons. It can be estimated that the determined intensity level will occur again for certain future measurements (e.g., at similar angular positions of a LIDAR system close in time or rotated). For example, the change of operating state can improve power usage (e.g., by disconnecting or reducing the power of photodetectors that are not expected to provide useful signals) or improve the dynamic range of the photodetectors (e.g., by changing the attenuation level so that the photodetectors provide useful signals). For example, different photodetectors (e.g., SPADs) of a light sensor can operate differently in different lighting environments. For example, in a direction with significant background light or perpendicular to a highly reflective surface that will reflect a strong signal, sensitive photodetectors can have their power levels reduced (e.g., disconnected), otherwise they will start quickly, thereby providing little ranging information or consuming a lot of power. Changes in the operating state of photodetectors with low sensitivity can be made in other environments. For example, in directions with weak signals, such weak light detectors may never activate, and thus the attenuation level of such weak light detectors may be reduced, thereby enabling the light detectors to detect photons in low light flux environments.

[0062] Further benefits (e.g., cost and size) can be achieved by having the light sensor (e.g., each a series of SPADs) and signal processing components on the same integrated circuit. For example, the timing circuit of the integrated circuit can determine the time of the detected photon, and the histogram circuit can accumulate the value of the pulse detected over multiple detection time intervals measured. The use of SPADs or similar light detectors can provide the ability to combine all of these circuits on a single integrated circuit.

[0063] The following sections introduce an illustrative automotive LIDAR system, followed by a description of example techniques for detecting signals via the light ranging system, and then describe various embodiments in greater detail.

[0064] I. Illustrative Automotive LIDAR System

[0065] Figures 1A-1BAn automotive optical ranging device, also referred to herein as a LIDAR system, is shown according to some embodiments. The automotive application of the LIDAR system is chosen here for illustration purposes only, and the sensors described herein can be used in other types of vehicles, such as ships, airplanes, trains, etc., as well as in a variety of other applications where 3D depth images are useful, such as medical imaging, mobile phones, augmented reality, geodesy, geomatics, archaeology, topography, geology, geomorphology, seismology, forestry, atmospheric physics, laser guidance, airborne laser swath mapping (ALSM), and laser altimetry. According to some embodiments, a LIDAR system, such as a scanning LIDAR system 101 and / or a solid-state LIDAR system 103, can be mounted on the roof of a vehicle 105, such as Figure 1A and 1B As shown in .

[0066] Figure 1A The scanning LIDAR system 101 shown in FIG. 1 can employ a scanning architecture, wherein the orientation of the LIDAR light source 107 and / or detector circuitry 109 can be scanned around one or more fields of view 110 within an external field or scene outside the vehicle 105. In the case of a scanning architecture, emitted light 111 can be scanned over the surrounding environment as shown. For example, the output beam of one or more light sources (e.g., infrared or near-infrared pulsed IR lasers, not shown) positioned in the LIDAR system 101 can be scanned (e.g., rotated) to illuminate the scene surrounding the vehicle. In some embodiments, the scanning, represented by the rotation arrow 115, can be implemented mechanically, such as by mounting the light emitter to a rotating column or platform. In some embodiments, the scanning can be implemented by other mechanical means, such as by using a galvanometer. Chip-based steering techniques can also be employed, such as by using a microchip employing one or more MEMS-based reflectors, such as a digital micromirror (DMD) device, a digital light processing (DLP) device, and the like. In some embodiments, the scanning can be achieved by non-mechanical means, such as by using electronic signals to steer one or more optical phased arrays.

[0067] For fixed architectures, e.g. Figure 1B 1 , one or more solid-state LIDAR subsystems (e.g., 103a and 103b) may be mounted to a vehicle 105. Each solid-state LIDAR unit may face a different direction (possibly with partially overlapping and / or non-overlapping fields of view between the units) in order to capture a larger field of view than each unit could capture on its own.

[0068] In either a scanning or fixed architecture, objects within the scene may reflect portions of a light pulse emitted from a LIDAR light source. One or more reflected portions then travel back to the LIDAR system and may be detected by the detector circuitry. For example, reflected portion 117 may be detected by the detector circuitry 109. The detector circuitry may be housed in the same housing as the emitter. Aspects of scanning and fixed systems are not mutually exclusive and may therefore be used in combination. For example, Figure 1B The individual LIDAR subsystems 103a and 103b may employ steerable emitters, such as optical phased arrays, or the entire composite unit may be rotated by mechanical means to scan the entire scene in front of the LIDAR system, for example, from field of view 119 to field of view 121.

[0069] Figure 2 A more detailed block diagram illustrating a rotating LIDAR system 200 according to some embodiments. More specifically, Figure 2 Optionally described is a rotating LIDAR system that may employ a rotary actuator on a rotating circuit board, which may receive power and data (and transmit) from a fixed circuit board.

[0070] LIDAR system 200 can interact with one or more examples of a user interface 215. Different examples of user interface 215 can vary and can include, for example, a computer system with a monitor, keyboard, mouse, CPU, and memory; a touch screen in an automobile; a handheld device with a touch screen; or any other suitable user interface. User interface 215 can be local to the object on which LIDAR system 200 is installed, but can also be a remotely operated system. For example, commands and data to and from LIDAR system 200 can be transmitted via a cellular network (LTE, etc.), a personal area network (Bluetooth, Zigbee, etc.), a local area network (WiFi, IR, etc.), or a wide area network such as the Internet.

[0071] The user interface 215 of the hardware and software can present LIDAR data from the device to the user, but can also allow the user to control the LIDAR system 200 with one or more commands. Example commands may include commands to activate or deactivate the LIDAR system, specify photodetector exposure level, bias, sampling duration, and other operating parameters (e.g., pulse pattern and signal processing), and specify light emitter parameters such as brightness. In addition, commands can allow the user to select a method for displaying results. The user interface can display LIDAR system results, which can include, for example, a single-frame snapshot image, a constantly updated video image, and / or a display of other light measurements of some or all pixels. In some embodiments, the user interface 215 can track the distance (proximity) of objects from the vehicle and potentially provide warnings to the driver or provide such tracking information for analysis of driver performance.

[0072] In some embodiments, the LIDAR system can communicate with the vehicle control unit 217 and can modify one or more parameters associated with the control of the vehicle based on the received LIDAR data. For example, in a fully autonomous vehicle, the LIDAR system can provide real-time 3D images of the car's surroundings to assist in navigation. In other cases, the LIDAR system can be used as part of an advanced driver assistance system (ADAS) or as part of a safety system, which can, for example, provide 3D image data to any number of different systems, such as adaptive cruise control, automatic parking, driver drowsiness monitoring, blind spot monitoring, collision avoidance systems, etc. When the vehicle control unit 217 is communicatively coupled to the optical ranging device 210, a warning can be provided to the driver or tracking of the proximity of an object can be tracked.

[0073] Figure 2 The illustrated LIDAR system 200 includes an optical ranging device 210. The optical ranging device 210 includes a ranging system controller 250, an optical transmission (Tx) module 240, and an optical sensing (Rx) module 230. The optical ranging device can generate ranging data by transmitting one or more light pulses 249 from the optical transmission module 240 to objects in the field of view surrounding the optical ranging device. A reflected portion 239 of the transmitted light is then detected by the optical sensing module 230 after some delay time. Based on the delay time, the distance to the reflecting surface can be determined. Other ranging methods, such as continuous wave, Doppler, and the like, may also be employed.

[0074] Tx module 240 includes an emitter array 242, which can be a one-dimensional or two-dimensional array of emitters, and Tx optics 244, which, when combined, can form an array of micro-optical emitter channels. Emitter array 242 or individual emitters are examples of laser sources. Tx module 240 also includes a processor 245 and memory 246. In some embodiments, pulse coding techniques such as Barker codes and the like can be used. In these cases, memory 246 can store a pulse code indicating when light should be emitted. In one embodiment, the pulse code is stored as a sequence of integers stored in the memory.

[0075] The Rx module 230 may include a sensor array 236, which may be, for example, a one-dimensional or two-dimensional array of light sensors. Each light sensor (also referred to simply as a sensor) may include an array of light detectors, such as SPADs or the like, or the sensor may be a single-photon detector (e.g., an APD). Similar to the Tx module 240, the Rx module 230 includes an Rx optics system 237. Combined, the Rx optics system 237 and the sensor array 236 may form an array of micro-optical receiver channels. Each micro-optical receiver channel measures light corresponding to an image pixel in a distinct field of view of the surrounding volume. For example, due to the geometric configuration of the light sensing module 230 and the light transmission module 240, each sensor (e.g., an array of SPADs) of the sensor array 236 may correspond to a specific emitter of the emitter array 242.

[0076] In one embodiment, the sensor array 236 of the Rx module 230 is fabricated as part of a monolithic device on a single substrate (using, for example, CMOS technology) and includes an array of photon detectors and an ASIC 231 for signal processing raw histograms from individual photon detectors (or groups of detectors) in the array. As an example of signal processing, for each photon detector or group of photon detectors, the memory 234 (e.g., SRAM) of the ASIC 231 can accumulate counts of photons detected over consecutive time intervals, and these time intervals, when combined, can be used to recreate a time series of reflected light pulses (i.e., photon counts versus time). This time series of aggregated photon counts is referred to herein as an intensity histogram (or simply histogram). The ASIC 231 can implement matched filters and peak detection processing to identify return signals in a timely manner. Additionally, the ASIC 231 can implement certain signal processing techniques (e.g., via the processor 238), such as multi-curve matched filtering, to help recover a photon time series that is less susceptible to pulse shape distortion due to SPAD saturation and quenching. In some embodiments, all or part of this filtering may be performed by processor 258, which may be implemented in an FPGA.

[0077] In some embodiments, the Rx optics system 237 can also be part of the same monolithic structure as the ASIC, with separate substrate layers for each receiver channel layer. For example, the aperture layer, collimating lens layer, filter layer, and photodetector layer can be stacked and bonded at the wafer level before dicing. The aperture layer can be formed by placing an opaque substrate on top of a transparent substrate or by coating a transparent substrate with an opaque film. In still other embodiments, one or more components of the Rx module 230 can be external to the monolithic structure. For example, the aperture layer can be implemented as a separate metal sheet with pinholes.

[0078] In some embodiments, the photon time series output from the ASIC is sent to the ranging system controller 250 for further processing. For example, the data can be encoded by one or more encoders of the ranging system controller 250 and then sent as data packets to the user interface 215. The ranging system controller 250 can be implemented in a variety of ways, including, for example, by using a programmable logic device such as an FPGA, an ASIC or a portion of an ASIC, using a processor 258 with a memory 254, or a combination thereof. The ranging system controller 250 can operate in conjunction with a fixed base controller or independently of the base controller (via pre-programmed instructions) to control the light sensing module 230 by sending commands, including starting and stopping light detection and adjusting light detector parameters. Similarly, the ranging system controller 250 can control the light transmission module 240 by sending commands, or relaying commands from the base controller, including starting and stopping light emission control and adjusting other light emitter parameters (e.g., pulse code). In some embodiments, the ranging system controller 250 has one or more wired interfaces or connectors for exchanging data with the light sensing module 230 and the light transmission module 240. In other embodiments, the ranging system controller 250 communicates with the light sensing module 230 and the light transmission module 240 over a wireless interconnect, such as an optical communication link.

[0079] The motor 260 is an optional component required when system components such as the Tx module 240 and / or the Rx module 230 need to rotate. The system controller 250 controls the motor 260 and can start the rotation, stop the rotation, and change the rotation speed.

[0080] II. Detection of Reflected Pulses

[0081] The light sensors can be arranged in a variety of ways for detecting the reflected pulses. For example, the light sensors can be arranged in an array, and each light sensor can include an array of light detectors (e.g., SPADs). Different patterns of pulses (pulse trains) emitted during the detection interval are also described below.

[0082] A. Time-of-Flight Measurement and Detector

[0083] Figure 3 The operation of a typical LIDAR system that can be improved by embodiments is illustrated. A laser generates a short-duration light pulse 310. The horizontal axis is time and the vertical axis is power. Example laser pulse durations, characterized by full width at half maximum (FWHM), are a few nanoseconds, with peak powers of a single emitter on the order of a few watts, as seen in the lower figure. Embodiments using side-emitter lasers or fiber lasers can have much higher peak powers, while embodiments with small-diameter VCSELs can have peak powers in the tens to hundreds of milliwatts.

[0084] The start time 315 of pulse emission need not coincide with the leading edge of the pulse. As shown, the leading edge of light pulse 310 is after the start time 315. It may be desirable that the leading edges be different in situations where pulses of different modes are emitted at different times, which is described in more detail below for decoded pulses.

[0085] The optical receiver system can begin detecting received light simultaneously with the activation of the laser, i.e., at the start time. In other embodiments, the optical receiver system can begin detecting received light at a later time, a known time after the start time of the pulse. The optical receiver system initially detects background light 330 and, some time later, detects laser pulse reflection 320. The optical receiver system can compare the detected light intensity to a threshold value to identify laser pulse reflection 320. The threshold value can distinguish between background light and light corresponding to laser pulse reflection 320.

[0086] Time of flight 340 is the time difference between the transmitted pulse and the received pulse. This time difference can be measured by subtracting the emission time of the pulse (e.g., as measured relative to the start time) from the reception time of the laser pulse reflection 320 (e.g., also measured relative to the start time). The distance to the target can be determined as half the product of the time of flight and the speed of light.

[0087] Pulses from the laser device reflect off objects in the scene at different times, and the pixel array detects the pulses of radiation reflection.

[0088] B. Object Detection Using Array Lasers and Light Sensor Arrays

[0089] Figure 4 An illustrative example of a light transmission and detection process for an optical ranging system is shown in accordance with some embodiments. Figure 4 An optical ranging system (eg, solid-state and / or scanning) is shown that collects three-dimensional distance data for a volume or scene surrounding the system. Figure 4This is an idealized diagram to highlight the relationship between the emitter and the sensor, and therefore other components are not shown.

[0090] Optical ranging system 400 includes a light emitter array 402 and a light sensor array 404. Light emitter array 402 includes an array of light emitters, such as an array of VCSELs and the like, such as emitter 403 and emitter 409. Light sensor array 404 includes an array of light sensors, such as sensors 413 and 415. The light sensors can be pixelated light sensors that employ a set of discrete light detectors, such as single photon avalanche diodes (SPADs) and the like, for each pixel. However, various embodiments may employ any type of photon sensor.

[0091] Each emitter can be slightly offset from its neighbors and can be configured to emit light pulses into a different field of view than its neighbors, thereby illuminating only the corresponding field of view associated with that emitter. For example, emitter 403 emits an illumination beam 405 (formed by one or more light pulses) into a circular field of view 407 (the size of which is exaggerated for clarity). Similarly, emitter 409 emits an illumination beam 406 (also referred to as an emitter channel) into a circular field of view 410. Although Figure 4 Not shown in to avoid complexity, each emitter emits a corresponding illumination beam into its corresponding field of view, resulting in a 2D array of illuminated fields of view (21 distinct fields of view in this example).

[0092] Each field of view illuminated by an emitter can be considered as a pixel or light point in the corresponding 3D image generated from the ranging data. Each emitter channel can be distinct for each emitter and non-overlapping with other emitter channels, i.e., there is a one-to-one mapping between a set of emitters and a set of non-overlapping fields or viewing angles. Thus, in Figure 4 In the example of , the system can sample 21 distinct points in 3D space. A denser sampling of points can be achieved by having a denser array of emitters or by scanning the angular position of the emitter beam over time so that one emitter can sample several points in space. As described above, scanning can be achieved by rotating the entire emitter / sensor assembly.

[0093] Each sensor can be slightly offset from its neighbors, and similar to the emitters described above, each sensor can see a different field of view of the scene in front of the sensor. In addition, the field of view of each sensor is substantially consistent with the field of view of the corresponding emitter channel, for example, overlapping with it and having the same size.

[0094] Figure 4In the example, the distances between corresponding emitter-sensor channels are magnified relative to the distances to objects in the field of view. In reality, the distances to objects in the field of view are much greater than the distances between corresponding emitter-sensor channels, and thus the path of light from the emitter to the object is approximately parallel to the path of light reflected from the object back to the sensor (i.e., it is almost "reflected back"). Consequently, there is a range of distances ahead of system 400 where the fields of view of individual sensors and emitters overlap.

[0095] Because the field of view of an emitter overlaps with the field of view of its corresponding sensor, each sensor channel can ideally detect the reflected illumination beam originating from its corresponding emitter channel, ideally without crosstalk, i.e., without detecting reflected light from other illumination beams. Thus, each light sensor can correspond to a respective light source. For example, emitter 403 emits illumination beam 405 into a circular field of view 407, and a portion of the illumination beam reflects from object 408. Ideally, reflected beam 411 is detected only by sensor 413. Thus, emitter 403 and sensor 413 share the same field of view, e.g., field of view 407, and form an emitter-sensor pair. Similarly, emitter 409 and sensor 415 form an emitter-sensor pair, sharing field of view 410. Although emitter-sensor pairs are Figure 4 are shown in the same relative positions in their respective arrays, but any emitter can be paired with any sensor depending on the design of the optics used in the system.

[0096] During ranging measurements, reflected light from different fields of view distributed around the volume surrounding the LIDAR system is collected and processed by various sensors to derive distance information for any object in each respective field of view. As described above, time-of-flight techniques can be used, in which a light emitter emits a precisely timed pulse, and after some elapsed time, the reflection of the pulse is detected by a respective sensor. The time elapsed between emission and detection, along with the known speed of light, is then used to calculate the distance to the reflecting surface. In some embodiments, additional information can be obtained from the sensor to determine other properties of the reflecting surface in addition to the distance. For example, the Doppler shift of the pulse can be measured by the sensor and used to calculate the relative velocity between the sensor and the reflecting surface. The pulse intensity can be used to estimate the target reflectivity, and the pulse shape can be used to determine whether the target is a hard or diffuse material.

[0097] In some embodiments, the LIDAR system may include a relatively large 2D array of emitter and sensor channels and operate as a solid-state LIDAR, i.e., it can obtain frames of range data without scanning the orientation of the emitter and / or sensor. In other embodiments, the emitter and sensor can scan, for example, rotate around an axis, to ensure that the field of view of the emitter and sensor set samples a full 360-degree area (or a useful portion of the 360-degree area) of the surrounding volume. For example, the range data collected from the scanning system over a predefined time period can then be post-processed into one or more data frames, which can then be further processed into one or more depth images or 3D point clouds. The depth images and / or 3D point clouds can be further processed into map tiles for use in 3D mapping and navigation applications.

[0098] C. Multiple light detectors in a light sensor

[0099] Figure 5 The various stages of a sensor array and associated electronics according to an embodiment of the present invention are shown. Array 510 shows light sensors 515, each corresponding to a different pixel. Array 510 can be a staggered array. In this particular example, array 510 is an 18x4 light sensor. Array 510 can be used to achieve high resolution (e.g., 72x1024) because the embodiment is suitable for scanning.

[0100] Array 520 shows an enlarged view of a portion of array 510. As can be seen, each photosensor 515 includes a plurality of photodetectors 525. The signals from the photodetectors of a pixel collectively contribute to the measurement for that pixel.

[0101] In some embodiments, each pixel has a large number of single-photon avalanche diode (SPAD) cells, which increases the dynamic range of the pixel itself. Each SPAD can have analog front-end circuits for biasing, quenching, and recharging. SPADs are typically biased at a bias voltage higher than the breakdown voltage. Suitable circuitry senses the leading edge of the avalanche current, generates a standard output pulse synchronized with the avalanche buildup, quenches the avalanche by reducing the bias below the breakdown voltage, and restores the photodiode to operating levels.

[0102] SPADs can be positioned to maximize the fill factor in their local area, or a microlens array can be used, which allows for a high optical fill factor at the pixel level. Thus, an imager pixel can contain an array of SPADs to increase the efficiency of the pixel detector. A diffuser can be used to diffuse the rays passing through the aperture and collimated by the microlenses. A can diffuser is used to diffuse the collimated light in such a way that all SPADs belonging to the same pixel receive some radiation.

[0103] Figure 5Also shown is a specific photodetector 530 (e.g., a SPAD) that detects photons 532. In response to the detection, the photodetector 530 generates an avalanche current 534 of charge carriers (electrons or holes). A threshold circuit 540 regulates the avalanche current by comparing the avalanche current 534 with a threshold. When a photon is detected and the photodetector 530 is functioning normally, the avalanche current 534 rises above the comparator threshold, and the threshold circuit 540 generates a time-accurate binary signal 545 indicating the exact time of the SPAD current avalanche, which in turn is an accurate measurement of the arrival of the photon. The correlation of the current avalanche with the arrival of the photon can occur with a resolution of nanoseconds, thereby providing high timing resolution. The rising edge of the binary signal 545 can be latched by a pixel counter 550.

[0104] Binary signal 545, avalanche current 534, and pixel counter 550 are examples of data values that can be provided by a light sensor including one or more SPADs. The data value can be determined from the corresponding signal from each of the plurality of light detectors. Each of the corresponding signals can be compared to a threshold value to determine whether the corresponding light detector is triggered. Avalanche current 534 is an example of an analog signal, and thus the corresponding signal can be an analog signal.

[0105] The pixel counter 550 can use the binary signal 545 to count the number of photodetectors for a given pixel that have been triggered by one or more photons during a specific time interval (e.g., a 1, 2, 3, etc. nanosecond time window) controlled by the periodic signal 560. The pixel counter 550 can store a counter for each of multiple time intervals for a given measurement. The value of the counter for each time interval can start at zero and increment based on the binary signal 545 indicating that a photon was detected. The counter can increment when any photodetector of the pixel provides this signal.

[0106] The periodic signal 560 can be generated by a phase-locked loop (PLL) or a delay-locked loop (DLL) or any other method for generating a clock signal. The coordination of the periodic signal 560 and the pixel counter 550 can act as a time-to-digital converter (TDC), which is a device for identifying events and providing a digital representation of the time at which they occur. For example, the TDC can output the arrival time of each detected photon or optical pulse. The measured time can be the time elapsed between two events (e.g., the start time and the detected photon or optical pulse) rather than the absolute time. The periodic signal 560 can be a relatively fast clock that switches between a group of memories including the pixel counter 550. Each register in the memory can correspond to a histogram interval, and the clock can switch between them at sampling intervals. Therefore, a binary value can indicate the triggering of the histogram circuit when the corresponding signal is greater than a threshold. The histogram circuit can aggregate the binary values across the multiple photodetectors to determine the number of photodetectors triggered during a specific time interval.

[0107] The time interval can be measured relative to the start signal, e.g. Figure 3 The start time 315 of the start signal. Therefore, the counter of the time interval just after the start signal can have a low value corresponding to the background signal, such as the background light 330. The last time interval can correspond to the end of the detection time interval (also known as excitation) of a given pulse train, which is further described in the next section. The number of cycles of the periodic signal 560 since the start time can serve as a timestamp when the rising edge of the avalanche current 534 indicates the detected photon. The timestamp corresponds to the time interval for a specific counter in the pixel counter 550. This operation is different from a simple analog-to-digital converter (ADC) following a photodiode (e.g., for an avalanche diode (APD)). Each of the counters of the time interval can correspond to a histogram, which is described in more detail below. Therefore, while the APD is a linear amplifier for the input optical signal with a finite gain, the SPAD is a trigger device that provides a binary output of yes / no for a trigger event that occurs in a time window.

[0108] D. Pulse train

[0109] Ranging can also be achieved by using a pulse train, which is defined as containing one or more pulses. Within a pulse train, the number of pulses, the width of the pulses, and the duration between pulses (collectively referred to as the pulse pattern) can be selected based on several factors, some of which include:

[0110] 1 - Maximum laser duty cycle. The duty cycle is the fraction of time the laser is on. For pulsed lasers, this can be determined by the FWHM as explained above and the number of pulses emitted during a given cycle.

[0111] 2- Eye safety limit. This is determined by the maximum amount of radiation that can be emitted by the device without harming the eyes of a bystander who happens to be looking in the direction of the LIDAR system.

[0112] 3- Power consumption. This is the power consumed by the emitter to illuminate the scene.

[0113] For example, the spacing between pulses in a pulse train may be on the order of a few single digits or tens of nanoseconds.

[0114] Multiple pulse trains may be emitted during the time span of one measurement. Each pulse train may correspond to a different time interval, for example a subsequent pulse train is not emitted before a time limit for detecting a reflected pulse of a previous pulse train expires.

[0115] For a given emitter or laser device, the time between pulse train emissions determines the maximum detectable range. For example, if pulse train A is emitted at time t0 = 0 ns, and pulse train B is emitted at time t1 = 1000 ns, then reflected pulse trains detected after t1 should not be assigned to pulse train A because they are more likely to be reflections from pulse train B. Therefore, the time between pulse trains and the speed of light define the maximum limit on the system range:

[0116] R max =c×(t1-t0) / 2

[0117] The time between excitation (emission and detection of the pulse train) may be approximately 1 μ8 to allow sufficient time for the entire pulse train to travel to a distant object approximately 150 meters away, and then back.

[0118] III. Histogram Signal from Photodetector

[0119] One mode of operation of a LIDAR system is time correlated single photon counting (TCSPC), which is based on counting single photons in a periodic signal. This technique works well for low levels of periodic radiation, which is suitable in LIDAR systems. This time correlated counting can be controlled by Figure 5 The periodic signal 560 controls and uses the time interval, such as for Figure 5 Discussed.

[0120] A. Histogram Generation

[0121] The frequency of a periodic signal can specify the temporal resolution at which data values of the measured signal are obtained. For example, a measured value can be obtained for each light sensor during each cycle of the periodic signal. In some embodiments, the measured value can be the number of light detectors triggered during the cycle. The time period of the periodic signal corresponds to a time interval, where each cycle is a different time interval.

[0122] Figure 6 A histogram 600 is shown according to an embodiment of the present invention. The horizontal axis corresponds to a time interval as measured relative to a start time 615. As described above, the start time 615 can correspond to the start time of the pulse train. Any offset between the rising edge of the first pulse of the pulse train and the start time of either or both of the pulse train and the detection time interval can be taken into account, wherein determining the reception time will be used for the time of flight measurement. The vertical axis corresponds to the number of SPADs triggered. In some embodiments, the vertical axis can correspond to the output of an ADC following the APD. For example, an APD can exhibit traditional saturation effects, such as a constant maximum signal, rather than the dead time-based effects of a SPAD. Some effects can occur for both SPADs and APDs, such as pulse tailing of extremely tilted surfaces can occur for both SPADs and APDs.

[0123] The counter for each of the time intervals corresponds to a different bar in histogram 600. The counter in the early time intervals is relatively low and corresponds to background noise 630. At some point, a reflected pulse 620 is detected. The corresponding counter is much larger and may be above the threshold for distinguishing between background and detected pulses. Reflected pulse 620 (after digitization) is shown as corresponding to four time intervals, which may be caused by laser pulses of similar width, such as 4 ns pulses when the time intervals are each 1 ns. However, as described in more detail below, the number of time intervals can vary, for example, based on the properties of the particular object in the incident angle of the laser pulse.

[0124] The temporal position of the time interval corresponding to the reflected pulse 620 can be used, for example, to determine the time of reception relative to the start time 615. As described in more detail below, a matched filter can be used to identify pulse patterns, thereby effectively increasing the signal-to-noise ratio and more accurately determining the time of reception. In some embodiments, the accuracy of determining the time of reception can be less than the temporal resolution of a single time interval. For example, for a 1 ns time interval, the resolution would correspond to approximately 15 cm. However, it may be desirable to have an accuracy of only a few centimeters.

[0125] Thus, the detected photons can cause a particular time interval of the histogram to be incremented based on their arrival time relative to the start signal, for example, indicated by start time 615. The start signal can be periodic so that multiple pulse trains are sent during the measurement. Each start signal can be synchronized to a laser pulse train, where multiple start signals cause multiple pulse trains to be emitted over multiple detection intervals. Thus, a time interval (e.g., from 200 to 201 ns after the start signal) will occur for each detection interval. The histogram can accumulate counts, where the count for a particular time interval corresponds to the sum of the measured data values that occurred in the particular time interval across multiple excitations. When the detected photons are histogrammed based on this technique, it results in a return signal with a signal-to-noise ratio that is greater than that of a single pulse train by the square root of the number of excitations made.

[0126] Figure 7 An accumulation of histograms over multiple bursts for a selected pixel is shown in accordance with an embodiment of the present invention. Figure 7 Three detected pulse trains 710, 720, and 730 are shown. Each detected pulse train corresponds to a transmitted pulse train with the same pattern of two pulses separated by the same amount of time. Thus, each detected pulse train has the same pulse pattern, as shown by the two time intervals having distinct values. Counters for the other time intervals are not shown for ease of illustration, but the other time intervals may have relatively low non-zero values.

[0127] In the first detected pulse train 710, the counters for time intervals 712 and 714 are identical. This may be due to the same number of photodetectors detecting photons during the two time intervals. Or, in other embodiments, approximately the same number of photons were detected during the two time intervals. In other embodiments, more than one consecutive time interval may have consecutive non-zero values; however, for ease of illustration, a few non-zero time intervals are shown.

[0128] Time intervals 712 and 714 occur 458ns and 478ns after start time 715, respectively. The counters shown for the other detected pulse trains occur at the same time intervals relative to their respective start times. In this example, start time 715 is identified as occurring at time 0, but the actual time is arbitrary. The first detection interval for the first detected pulse train may be 1 μs. Thus, the number of time intervals measured from start time 715 may be 1,000. This first detection interval then ends and a new pulse train may be transmitted and detected. The start and end of the different time intervals may be controlled by a clock signal, which may be part of a circuit acting as a time-to-digital converter (TDC), such as in a Figure 5 described in .

[0129] For the second detected pulse train 720, the start time 725 is 1 μs, for example, at which time the second pulse train can be emitted. This separate detection interval can occur so that any pulses emitted at the beginning of the first detection interval will have been detected, and therefore will not cause confusion with the pulses detected in the second time interval. For example, if there is no additional time between excitations, the circuit may confuse a back-reflected stop sign at 200 meters with a much less reflective object at 50 meters (assuming an excitation period of approximately 1 μs). The two detection time intervals for pulse trains 710 and 720 can be the same length and have the same relationship to the corresponding start times. Time intervals 722 and 724 occur at the same relative time of 458 ns and 478 ns as time intervals 712 and 714. Therefore, when the accumulation step occurs, the corresponding counter can be added. For example, the counter values at time intervals 712 and 722 can be added.

[0130] For the third detected pulse train 730, the start time 735 is 2 μs, for example, at which the third pulse train may be emitted. Time intervals 732 and 734 also occur at 458 ns and 478 ns relative to their respective start times 735. Even if the emitted pulses have the same power, the counter values at different time intervals may have different values, for example, due to the random nature of the scattering process of the light pulses leaving the object.

[0131] Histogram 740 shows the accumulation of counters from three detected bursts at time intervals 742 and 744, which also correspond to 458 ns and 478 ns. Histogram 740 may have a smaller number of time intervals measured during the corresponding detection interval, for example, due to discarding time intervals at the beginning or end, or having values less than a threshold. In some embodiments, depending on the pattern of the bursts, approximately 10-30 time intervals may have significant values.

[0132] For example, the number of pulse trains emitted during a measurement to generate a single histogram can be about 1-40 (e.g., 24), but can also be much higher, such as 50, 100, or 500. Once a measurement is completed, the counter for the histogram can be reset, and a set of pulse trains can be emitted to perform a new measurement. In various embodiments, and depending on the number of detection intervals in the corresponding duration, measurements can be performed every 25, 50, 100, or 500 μs. In some embodiments, the measurement intervals can overlap, for example, so that a given histogram corresponds to a specific sliding window of pulse trains. In this example, a memory can be present for storing multiple histograms, each corresponding to a different time window. The weights applied to the detected pulses can be the same for each histogram, or such weights can be controlled independently.

[0133] B. Example signal curve from a pixel detector

[0134] Under various conditions, different levels of reflected or ambient radiation may reach a photodetector (e.g., a SPAD). This can affect the efficiency and accuracy of the photodetector in sensing the reflected radiation, and therefore affect the performance of the LIDAR system in detecting objects in the scene and reconstructing the scene. Under normal conditions and for reflections from many surfaces, the probability of detecting a photon in a given time period is much smaller than one. Therefore, in a LIDAR system, there are no photons during some time intervals, and there is a small signal in some other intervals.

[0135] However, when a large amount of radiation impinges on a photosensor (e.g., an array of photodetectors, such as SPADs), the pulses digitized as a histogram from the optical pulse may initially be extremely high and then decrease, rather than having a more uniform value over the duration of the pulse (assuming a rectangular shape). For example, for a given optical pulse with a width of five time bins, almost all photodetectors may activate in the first time bin. This effect, which can be referred to as pile-up, can occur in binary counting photodetectors, such as SPADs, which have dead times in which the photodetectors cannot detect another photon.

[0136] High levels of pile-up occur when a large number of photons hit a pixel, causing most of the SPADs in the pixel to fire and enter their dead time within a small fraction of the pulse width. Consequently, the remainder of the photons in the pulse are not captured by the pixel, and the SPADs do not recover in time to account for the true optical profile and magnitude of the signal. In these situations, the true amount and profile of reflected radiation in the LIDAR system is unknown. These problems can arise from large amounts of radiation reflecting off highly reflective objects in the scene or from high levels of background radiation.

[0137] According to some embodiments, the raw histogram (e.g., a counter of the histogram on one or more pulse trains) output by a series of light detectors in a pixel is filtered according to one or more expected or possible curves of the digitized signal. Different curve filters can be used for different levels of accumulation. The level of accumulation can correspond to the different rates at which photons are detected by the light sensor on a continuous time interval. In some embodiments, the curve filter can be stored and used to identify the curve type. The type can be used to more accurately determine time, and then more accurately determine distance or be used to determine other signal properties such as pulse width.

[0138] Figure 8Three different types of digitized signal curves output from a light sensor according to an embodiment of the present invention are shown. In some embodiments, the light sensor can be a series of SPADs. The curves can be stored in a histogram, which can be determined as described above. In the three curves, the horizontal axis corresponds to the time interval, and the vertical axis corresponds to the data value output from the light sensor during a specific time interval (e.g., the number of activated SPADs). These example types include signals with no pile-up (or low pile-up), signals with moderate pile-up, and signals with high pile-up. In practice, there may be many (K) different curves.

[0139] Different types of digitized signal curves have different rates for detecting photons. A high rate for detecting photons can correspond to a higher number of photodetectors being triggered during an initial time interval than during a later time interval. A low rate can correspond to the same number of photodetectors being triggered during the initial time interval as during a later time interval.

[0140] The low pile-up curve 810 has relatively sharp rises and falls and remains fairly flat between these two times, except for some observed noise levels. This is characteristic of a small number of photons reflected from a surface. A low pile-up curve corresponds to normal activity. For example, the number of photons reflected back from an object may be sufficiently small over several time intervals that the same SPAD does not attempt to trigger multiple times while it is in its dead time. In this case, the reflected rectangular pulse is digitized by a series of SPADs (e.g., 16 or 32), resulting in a square curve in the histogram. For example, when there are 32 SPADs per pixel, four SPADs can be activated at any given time, with approximately four of them being activated in unison. Therefore, even if a SPAD that has already been activated is about to be in its dead time, the other SPADs will still be available. Therefore, over the duration of a 5-ns pulse, fewer than half of the SPADs may be activated.

[0141] The moderate accumulation curve 820 has a sharp rise followed by a steady decline over time as it returns to the background level. The number of SPADs that are not stagnant and therefore available for detecting photons decreases relatively quickly because the light flux is high. The moderate accumulation curve 820 can be caused by a relatively large number of photons impinging on the SPADs in a short duration within the time interval. This may come from background radiation or, more commonly, due to laser reflection from a relatively reflective surface. As an example, if 10 SPADs are triggered in the first nanosecond, then 10 fewer SPADs can detect photons in the next time interval. Therefore, the digitized pulse slopes downward in the histogram for the pixel, even though the actual light flux 850 on the pixel is a square pulse shown in dashed lines.

[0142] High pile-up curve 830 indicates a very sharp rise in the signal followed by a rapid fall. High pile-up can occur when all SPADs are triggered in the initial nanosecond or less. This results in a large spike; immediately thereafter, no further firing occurs even if the pulse continues for another 5 ns. High levels of pile-up can result from large levels of reflected signal from extremely reflective surfaces, particularly those close to and perpendicular to the axis of radiation emitted from the laser source, resulting in reflections that point straight back to the imaging device.

[0143] The scale of the vertical axes of the different signals can vary between the different curves and has been shown at approximately the same scale for ease of illustration. As mentioned above, more curves may be considered according to embodiments of the present invention. Furthermore, photodiodes other than SPADs can skew the shape. For example, a retroreflector can saturate the APD. The APD can attenuate strong pulses from the retroreflector, which has two effects: providing a flat top for the digitized signal in memory and having the secondary effect of increasing the effective full width at half maximum of the signal, since the true maximum value of the signal is not measurable.

[0144] C. Problems with detecting different signal curves

[0145] A matched filter can be used to detect the most accurate position of a detected pulse, for example after calculating the convolution of the histogram and the matched filter, where the maximum value of the convolution corresponds to the position of the pulse in the histogram. A matched filter is an optimal filter for detecting a known signal in the presence of uncorrelated noise. For example, refer back to Figure 3 , the background light 330 can be optimally suppressed by a matched filter having a shape matched to the digitized laser pulse reflection 320 .

[0146] The low pile-up curve most closely resembles the shape of the emitted laser pulse. Therefore, it is natural to use a matched filter that matches the shape of the laser pulse. This filter will provide the highest accuracy when the digitized signal has a low pile-up curve. However, the accuracy decreases when the digitized signal has a different pile-up (detection rate).

[0147] In cases where pile-up is present, the filtered signal is temporally shifted, for example, to an earlier time. For example, convolution of a filter applied to a high pile-up curve will result in the center of the pulse being at a peak, but the peak is at the leading edge of the pulse. In other words, in a high pile-up situation, all digitized signals are to the left of the leading edge of the pulse. If a rectangle of similar width to the emitted pulse is matched to the detected high pile-up pulse, then the optimal match of the rectangle occurs earlier than if it were matched to the actual light flux 850.

[0148] A shift of a few intervals (e.g., 2.5 ns to the left from the center of a 5 ns pulse) can result in an error of approximately 37 cm, an unacceptable error for most 3D sensing applications. This can occur for light reflected from a car's license plate, where light reflected from the rest of the car may have low pile-up. This would make the license plate appear closer than the rest of the car, which could cause problems for autonomous vehicle decision-making or alerts / warnings to the vehicle.

[0149] In the embodiment described below, different curve filters can be used to compensate for the distortion imposed by pile-up or other expected factors. For example, instead of using a matched filter similar to that emitting a pulse to only analyze the original histogram, multiple matched filters with different curves can be used, as long as the filter power is all normalized to the same filter "power" (defined as the root mean square of the filter tap) so that its output can all be directly compared. For example, only one or two time interval wide matched filters can be used to determine whether there is a high pile-up curve. If the maximum value provided by the output of this high pile-up filter (for example, determined by the convolution of the filter and the original histogram) is higher than the low pile-up filter, the high pile-up filter will be used to calculate the reception time. Additional details are provided in a later section.

[0150] D. Limitations on the Accuracy of Matched Filters and Certain Pulse Patterns

[0151] As mentioned above, a matched filter can be used to determine the temporal position (receive time) of a detected pulse. The receive time can then be used to determine the total flight time of the pulse, which can then be converted to distance. However, the measurement accuracy of the digitized signal from the light sensor (e.g., the resolution of the time interval) limits the accuracy of the measured distance. It is desirable to have a resolution that is less than the width of the time interval. To achieve this goal, it is first necessary to identify the best match within the time resolution of the time interval. However, due to noise, the best match may occur just before or just after.

[0152] To address such issues and reduce interference, various pulse patterns are preferred over others in terms of sidelobe response and temporal "sharpness" of the filter response for a given pulse power.One question is how to achieve such coded pulses, but first the benefits of coded pulses are described.

[0153] Figure 91. A series of positions for applying a matched filter to an original histogram according to an embodiment of the present invention is shown. The series of positions can be considered as sliding positions because the filter slides (moves) on the original histogram. The original histogram 902 depicts a relatively small time window around the detected pulse of a single pixel. The filter 904 corresponds to the shape of the digitized pulse. Both the histogram 902 and the filter 904 have idealized shapes for ease of presentation. Figure 9 The series of plots to the left of shows different positions of the filter 904 relative to the histogram 902. The filter 904 is shifted by one time bin in each successive plot.

[0154] The filtered output resulting from the application of filter 904 to histogram 902 is Figure 9 is displayed on the right side of . The bar graph shows the level of overlap between the filter 904 and the histogram 902 at each position. The filtered output 910 corresponds to a first position where the overlap between the filter 904 and the histogram 902 is only one time interval. The vertical axis of the filtered output 910 is an arbitrary unit of overlap, for example, the sum of the multiplication products of the corresponding values of the filter 904 and the histogram 902. The values of the filtered output 910 are shown at a time interval corresponding to the center of the filter 904. This can be done when the center of the pulse is to be detected. In other embodiments, the values of the filtered output can be shown at a time corresponding to the leftmost interval of the filter. This can be done when the rising edge of the pulse is to be detected. The values of the center and the rising edge can be derived from one another. For example, it can be easier to define the position of the first filter tap (essentially the rising edge) when the pulses in the pulse train are of different widths. The different widths can be caused by different emission widths or due to different detected intensities, for example as Figure 8 Described in .

[0155] Filtered output 920 corresponds to the second position, where the overlap is two time intervals, and therefore the resulting value is twice that of filtered output 910. The values shown show different time intervals, followed by filtered output 910, because filter 904 has been shifted one time interval to the right. Filtered output 930 corresponds to the third position, where the overlap is three time intervals. Filtered output 940 corresponds to the fourth position, where the overlap is four time intervals. Filtered output 950 corresponds to the safe position, where the overlap is five time intervals. It is easy to see that the fifth position is the highest, corresponding to a perfect overlap between filter 904 and histogram 902.

[0156] The final filtered output 990 shows the value at each of the nine locations where there is some level of overlap between the filter 904 and the histogram 902. This filtered output can be analyzed to identify the maximum corresponding to the reception time of the detected pulse. In various embodiments, this time can be recorded directly or modified (e.g., to identify where the leading edge will be) as part of the time of flight measurement.

[0157] Figures 10A-10C 1 shows the types of matched filter responses (filtered outputs) produced by different decoded pulse types according to an embodiment of the present invention. Figure 10A In FIG, the decoded pulse 1010 is formed by turning on the optical transmitter for 5 time intervals. The matched filter response 1015 is wide (spanning multiple time intervals) and has a gradual gradient, making it difficult to detect the match, measure the amplitude of the match, and accurately determine the time. The matched filter response 1015 has a Figure 9 The final filtered output 990 is of a similar form.

[0158] exist Figure 10B In FIG, the decoded pulse 1020 is formed by turning the optical transmitter on and off five times and has a higher frequency component, thereby allowing for more accurate distance determination. Figure 10A and 10B The decoded pulses of have the same power because both are on for 5 time intervals. Figure 10B The matched filter response 1025 in has multiple side lobes, making it difficult to use in the presence of noise.

[0159] Figure 10C Decoded pulses 1030 are shown corresponding to the sequence {1, 1, 1, -1, 1} as an example of a Barker code, which is used in radar to produce a matched filter response with minimal or even purely negative sidelobes while preserving time-critical filter peaks. Figure 10C , the matched filter response 1035 shows a peak at the time interval that best matches the decoded pulse. The time interval with the peak magnitude of the matched filter response 1035 can be used to calculate the reception time of the reflected decoded pulse.

[0160] The peak magnitude has further uses. For example, the peak magnitude of the matched filter response 1035 can be used to calculate the amplitude of the reflected decoded pulse. The peak magnitude of the matched filter response 1035 depends on (1) the sum of the squares of the amplitudes of the reflected decoded pulse and (2) the width and magnitude of the filter. If all filters have the same power, they can be directly compared without scaling. For a given filter, the relative amplitude is given by the square root of the peak magnitude. The absolute value of the amplitude of the reflected decoded pulse is useful for distinguishing different types of reflecting objects or estimating the target reflectivity.

[0161] While such codes are known to have desirable properties, it's unclear how to implement such decoding for optical pulses. In electrical applications, negative voltages exist, but negative photons for optical pulses don't. In radio frequency (RF), techniques use the negative phase of one signal relative to a reference signal, such as with binary phase-shift keying (BPSK). However, such phase differences aren't a practical option for optical pulses. As described below, some embodiments can assign weights to different pulses based on the time interval between detections, such as which pulse train it is. For example, when accumulated into a histogram, the first pulse train might have a different weight than the second pulse train.

[0162] IV. Decoding pulses based on detection intervals

[0163] An optical ranging system (also known as a decoded pulse optical receiver system) can transmit multiple light pulses, where each decoded pulse has an embedded positive pulse code formed by light intensity. The system can determine the temporal position and / or amplitude of the light pulses in the presence of background light by generating an intensity histogram of the reflected light detected at different time intervals. For each time interval, the system adds a weighting value to the intensity histogram that depends on the intensity of the detected light. The weighting value can be positive or negative and have varying magnitudes.

[0164] By selecting different combinations of positive pulse codes and applying different weights, the system can detect both positive and negative codes suitable for standard digital signal processing algorithms. This approach gives a high signal-to-noise ratio while maintaining low uncertainty in the measured time position of the reflected light pulse.

[0165] A. Different weights for different pulse trains

[0166] Figure 111 is an exemplary schematic diagram 1100 illustrating the operation of a coded pulse optical system (CPOS) according to an embodiment of the present invention. Decoded pulse 1110 is first emitted during pulse time interval 1160 (also known as a detection interval), and decoded pulse 1120 is emitted after pulse time interval 1160 during pulse time interval 1165. Decoded pulses are formed by turning light transmission from an optical transmitter on or off. Decoded pulse 1110 can be represented as the sequence 0, 1, 0, 1, 1, 0, where 1 means the optical transmitter is on (i.e., transmitting light) and 0 means the optical transmitter is off. The sequence digits give the on / off values at consecutive light sampling intervals. Decoded pulse 1120 has a different pulse code than decoded pulse 1110. Decoded pulse 1120 can be represented by the sequence 0, 0, 1, 0, 0, 1. Decoded pulses are binary in that they can have either an on state or an off state.

[0167] The optical transmission module (optical transmitter) and the optical sensing module (optical receiver) can be synchronized to start at the same time and be active during the same pulse time interval. For decoded pulse 1110, synchronization can occur due to start signal 1101. For decoded pulse 1110, synchronization can occur due to start signal 1102. Start signals 1101 and 1102 can be considered a common start signal from which time is measured.

[0168] The light sensing module detects background light 1150 and reflected decoded pulses 1130 and 1140. Received decoded pulse 1130 is a reflection of transmitted pulse code 1110. Received decoded pulse 1140 is a reflection of transmitted pulse code 1120. The light sensing module digitizes the received light intensity and generates a light intensity value for each light sampling interval, referred to as a time bin or histogram bin. In this particular example, the light transmission interval between pulse trains is the same as the light sampling interval of the light sensing module used, for example, to generate the histogram. However, the light transmission interval at the light transmission module can differ from the light sampling interval of the optical receiver system.

[0169] Time interval 1155 corresponds to the light sampling interval when the optical receiver system is detecting background light 1150. Time interval 1135 corresponds to the light sampling interval when the optical receiver system first detects decoded pulse 1110. Time interval 1145 corresponds to the light sampling interval when the optical receiver system first detects decoded pulse 1120. CPOS applies a pulse weight to the received digitized light intensity (via multiplication) to obtain a weighted data value 1175. In this example, CPOS applies a pulse weight of +1 during pulse time interval 1160 and a pulse weight of -1 during pulse time interval 1165. Therefore, the weighted light intensity value for time interval 1135 is positive, and the weighted light intensity value for time interval 1145 is negative.

[0170] As described earlier, the light sensing module can maintain an intensity histogram with one cumulative value per time interval. The light sensing module initially sets the intensity histogram cumulative value to zero. During pulse time intervals 1160 and 1165, the light sensing module adds the weighted light intensity value to the existing value in the corresponding time interval of the intensity histogram. Therefore, for the first decoded pulse in pulse time interval 1160, the intensity histogram value is set equal to the weighted light intensity value because all values in the histogram start at zero. For the second decoded pulse in pulse time interval 1165, the weighted light intensity value is subtracted from the existing intensity histogram value. During pulse time interval 1165, the background light intensities of the two pulse time intervals tend to cancel each other and reduce the magnitude of the values in the intensity histogram. The result is histogram 1170.

[0171] CPOS can improve its detection and ranging accuracy by repeating the decoded pulses as needed. For example, the light intensity histogram can accumulate the results from two decoded pulses of type 1 and two decoded pulses of type 2. The order of the decoded pulse types (i.e., type 1, 2, 1, 2 or 1, 1, 2, 2) often has little effect on detection and ranging accuracy. In one embodiment, the decoded pulse types are alternating.

[0172] Figure 12 Two binary decoded pulse intensities 1210 and 1220 are shown, providing a decoded pulse train 1230 having positive and negative values, and their difference, according to an embodiment of the present invention. When the time series function of pulse train 1220 is subtracted from the time series function of pulse train 1210, time series function 1230 is generated. CPOS can use a matched filter to detect digitized pulses in the intensity histogram.

[0173] B. Reduced interference and higher levels of orthogonality

[0174] Figure 11 and12 A simple example is shown in which two types of binary coded pulses are transmitted, with pulse weights of (+1, -1) and their detected reflections are combined to simulate the effect of sending positive and negative coded pulses, which we call matching codes. After selecting the appropriate matching code, CPOS splits the positive and negative matching codes into two positive coded pulses, where the first coded pulse has the positive component of the matching code and the second coded pulse has the negative component of the matching code.

[0175] In some applications, there may be multiple CPOS terminals with different optical ranging systems operating in close proximity, for example, different emitter / sensor pairs in corresponding arrays. One CPOS terminal may detect the reflection of a decoded pulse emitted by a second CPOS terminal and report an incorrect result. In this case, it is advantageous to assign a different matching code combination to each CPOS terminal to avoid interference. In some embodiments, a pseudo-random pulse train can be used instead, which also provides a low probability of crosstalk between different ranging devices or between different pixel emitters in the same ranging device.

[0176] Furthermore, codes can be defined in multiple dimensions beyond just positive and negative weighting. For example, weights can be assigned to different excitations in 2D vector space, with either a Y-direction or an X-direction component of the weight. Instead of storing signed numbers, a histogram can be adjusted to store 2D vectors in each time interval, and vector addition can be used to accumulate vector-weighted codes into each time interval. For example, a pulse train can be sent with a positive Y weight, and the next pulse train can be sent with a negative Y weight, but both have zero weight in the X direction. Thus, the weights can be in higher dimensions and can have non-zero values in both directions. For example, a code could be {+Y, -Y, +Y, +X, -X}, and corresponding matched filters could be used. Background noise can be substantially eliminated (e.g., zero mean) in this multidimensional example in a similar manner as described above for the one-dimensional example, e.g., as long as an equal number of pulse trains have positive and negative values in the same dimension.

[0177] For higher dimensions, the decoded pulses of one system can be unique (or at least less likely to be used by another system) and therefore more easily distinguishable from other decoded pulses, thereby reducing crosstalk from other CPOS systems. Therefore, to reduce interference, embodiments can support advanced codes by combining multiple types of positive-valued decoded pulses with multidimensional vector pulse weights to generate multidimensional matching codes. For example, a quadrature phase code is a code with a phase difference of 90 degrees. An earlier code example with pulse weights of (+1, -1) can be considered as two codes with a phase difference of 180 degrees in one dimension. For a quadrature phase code, the pulse weights can be ([+1, 0], [-1, 0], [0, +1], [0, -1]), and the weighted light intensity values and intensity histogram values are 2-dimensional vectors. For a magnitude 1 pulse weight, the desired 2-dimensional matching code is decomposed into four positive-valued pulse codes by selecting positive and negative components for each vector dimension. In some embodiments, pulse weights can be in more than two dimensions. Various higher dimensional pulse codes may be used, making it easier to assign unique pulse codes to multiple CPOS.Thus, the weights may be vectors in two or more dimensions.

[0178] In addition to different dimensions, the coded pulses can have different values, such as other than just 0 or 1. Figure 11 and 12 In the example of , the transmitted pulse code value consists of 0 and 1 values corresponding to the optical transmitter being on or off. CPOS also supports multi-valued non-integer pulse codes with values such as 0, 1 The values of 1, 2, and 1 can be used. Such non-integer pulse codes can be generated, for example, by: a) varying the intensity of the optical transmitter light; and b) transmitting on / off optical pulses at a frequency faster than the light sampling interval. Therefore, the matching code derived from the pulse code can include multi-valued positive or negative integers or non-integer numbers. For example, CPOS can apply a weight of +1 to the pulse codes (1, 1, 1), (1, 1, 0), and (1, 0, 0) to generate a matching code of (3, 2, 1).

[0179] Additional techniques can be used to reduce interference from other optical ranging systems. For example, different systems can emit light pulses in different frequency / wavelength ranges. The receiver optical system can use filters (e.g., as part of a micro-optical device containing one aperture per pixel) that pass light in a relatively narrow range, such as spectral widths of 2 nm, 1 nm, 0.5 nm, 0.25 nm, or less. Further details on such micro-optics can be found in U.S. Patent Publications 2017 / 0289524 and 2017 / 0219426.

[0180] If a different system emits light outside the spectral range of the system's filters, then such other light will not reach the system's sensors. The number of different possible types of emitters (e.g., lasers) and corresponding filters can depend on how narrow the emitter's emission spectrum is and how narrow the filter's spectral range is. In some embodiments, different models or families of devices (e.g., vehicles, phones, survey equipment) can be assigned different spectral ranges, thereby reducing the likelihood or number of interfering devices in the vicinity of the system.

[0181] Alternatively or in addition, other techniques may have different channels of the same system with different spectral ranges. For example, Figure 4 Emitter 403 in the sensor 403 may emit light in a different spectral range than emitters adjacent to emitter 403. Also, the optical filter for sensor 413 may pass light in a range corresponding to the light emitted by emitter 403 but different from the optical filter of the adjacent sensor. In this way, crosstalk from other channels may be reduced.

[0182] C. System Components

[0183] Figure 13 A coded pulse optical system (CPOS) 1310 is shown according to an embodiment of the present invention. The CPOS 1310 may be Figure 2 The optical distance measuring device 210, wherein Figure 13 The corresponding device potentially has the same Figure 2 CPOS 1310 may interact with a user interface 1315, which may be Figure 2 The user interface 1315 (eg, hardware and software) may specify the match code and other operating parameters. The user interface 1315 may display the CPOS results, which may include a 3-dimensional image of the detected objects and distance values for specific objects.

[0184] The system controller 1320 can control the light sensing module 1330 (also referred to as the optical receiver system) by sending commands including: a) clear the intensity histogram accumulation value; b) specify pulse weight and other operating parameters (e.g., pulse time interval and light sampling interval); c) start light detection; d) send the intensity histogram accumulation value. The system controller 1320 can use the command bus to send decoded commands and the data bus to send and receive data values. In some embodiments, the system controller 1320 can send a 0 or 1 start signal to indicate when light detection should begin, and a 0 or 1 polarity signal to indicate the weight (positive or negative one) to be applied to the intensity value.

[0185] The system controller 1320 can control the optical transmission module 1340 (also referred to as an optical transmitter) by sending commands containing the following: a) specifying a pulse code and other operating parameters; b) starting the transmission of a pulse train with the specified pulse code.

[0186] System controller 1320 may include a digital signal processor (DSP) 1322 for executing a matched filter using matching codes 1324, which may be stored in Figure 2 1324 in memory 254. More than one matching code (filter) and filters of varying levels can be stored, as described in more detail in a later section. DSP 1322 can be implemented in a variety of ways, including: a) as a processor dedicated to DSP operations; b) using programmable logic blocks within an FPGA; and c) as logic within an ASIC. In alternative embodiments, DSP 1322 and matching code 1324 can be part of light sensing module 1330. Matching code 1324 defines the template time series function to be matched. In one embodiment, matching code 1324 is a series of values stored in memory. Matching code 1324 can be stored in conjugate time-reversed format for use by the matched filter.

[0187] Light sensing module 1330 contains pulse weights 1332, a histogram 1334, a sensor array 1336, and an arithmetic logic unit (ALU) 1338. Pulse weights 1332 can be stored as a sequence of integers or floating point numbers in a register or other memory location. In an alternative embodiment, the pulse weights are constrained to have positive or negative unit values (i.e., +1 or -1) in each dimension, and the pulse weights are implemented as bit patterns in a register or memory. The intensity histogram 1334 can be implemented as a memory with one or more memory cells per time interval (e.g., Figure 2 Memory 234). Sensor array 1336 detects photons of light and generates a digitized intensity value for each time interval.

[0188] In some embodiments, ALU 1338 multiplies the digitized light intensity from sensor array 1336 by the appropriate pulse weight 1332 and adds the result to the appropriate memory cell of histogram 1334. In other embodiments, light sensing module 1330 may add or subtract the detected light intensity to or from the appropriate memory cell of the intensity histogram. In such embodiments, ALU 1338 may perform addition and subtraction without the need for a multiplier.

[0189] The light transmission module 1340 contains a pulse code 1344 and an array of emitters 1342 (e.g., one or more laser diodes) capable of emitting light. The pulse code 1344 indicates when light should be emitted. The pulse code 1344 can be stored in a memory as a sequence of integers (e.g., Figure 2 in the memory 246).

[0190] The dimensions of histogram 1334 (e.g., the number of time bins) can be tailored to the application's needs. In various embodiments, when the pulse time interval (detection interval) is 1024 ns, the light sampling interval is 1 ns, and both types of pulse codes are transmitted 12 times each, histogram 1334 can contain 1024 12-bit memory cells. Light travels 30 cm in 1 ns, so a 1 ns light sampling interval indicates a nominal distance accuracy of 30 cm, or 15 cm when round-trip time is considered. By repeating the decoded pulse 12 times and accumulating the results over a longer period, improved distance accuracy is achieved in addition to improved detection accuracy.

[0191] Detecting 1024ns of light means that the CPOS can detect reflected light that has traveled up to 307.2m. If the CPOS were to transmit a second decoded pulse immediately after the first, a distant reflection from the first could be misinterpreted as a reflection from the second. The CPOS can avoid this problem by pausing between decoded pulses to allow the energy of the decoded pulses to dissipate. Various pauses can be implemented, such as the same amount as the detection interval, or more or less. Transmitting two different types of pulse codes 12 times with a 1024ns pause between decoded pulses takes 48 x 1024ns, approximately 50μs. If the object reflecting the light has moved a significant distance in that 50μs, distance accuracy may be affected, but this is not a problem for most objects. Even a car moving at 100km / h moves only 1.4mm in 50μs.

[0192] D. Methods of decoding pulses

[0193] Figure 14 14 is a flow chart illustrating a method 1400 for using decoded pulses in an optical measurement system according to an embodiment of the present invention. The optical measurement system may be an optical ranging system. Method 1400 may use multiple decoded pulses to detect the temporal position of a reflected pulse from a target. In real-time 3D applications, method 1400 may be repeated continuously for multiple directions. Method 1400 may be implemented by any of the optical measurement systems described herein.

[0194] At 1410, the coded pulse optical system (CPOS) performs initialization. For example, the CPOS can respond to user interface commands for starting, stopping, and changing parameters. The CPOS can initialize the optical transmitter to indicate parameters such as pulse code, optical power level, and various time intervals (e.g., detection interval, interval for pausing between detection intervals, and overall measurement time interval). The CPOS can also initialize the light sensing module to indicate parameters such as pulse time interval and light sampling interval. The CPOS can also clear histogram values, such as in histogram 1334.

[0195] At 1420, a pulse train is emitted from a light source (e.g., a laser) as part of an optical measurement. The pulse train may be emitted as part of N pulse trains emitted for the measurement. The N pulse trains may be reflected from an object, thereby allowing a range measurement to the object. Each of the N pulse trains may include one or more pulses from a light source (e.g., a VCSEL) and correspond to a different time interval triggered by a start signal. For example, Figure 11 Start signals 1101 and 1102 are shown triggering time intervals 1160 and 1165, respectively.

[0196] In some embodiments, the CPOS may wait a specified time to allow the previous pulse train (decoded pulse transmission) to dissipate. The CPOS may then transmit the next pulse train of the N pulse trains measured, where the N pulse trains form a code. Once the measurement is complete, e.g., the last of the N pulse trains has dissipated (e.g., after a predetermined time to allow for any reflections), the CPOS may then begin transmitting the first / next decoded pulse using the appropriate pulse code. N may be an integer greater than one, e.g., 2, 3, 4, 5, or more.

[0197] At 1430, optical detection can be initiated, for example, in response to a start signal that triggers the emission of a pulse train. Thus, the CPOS can begin optical detection at the same time as it begins emitting decoded pulses. As part of the optical detection, the pulse train can be detected by a light sensor (e.g., corresponding to a pixel) of the optical measurement system, thereby generating data values at multiple time points. In some embodiments, the light sensor is a series of light detectors (e.g., SPADs). The data values can take various forms, such as a count of the number of SPADs triggered at a time point (e.g., within a time interval of a histogram). As another example, the data value can be a digitized value from an ADC following an analog light sensor (e.g., an APD). Both examples can correspond to intensity. A total of N pulse trains can be detected. Furthermore, the process can be performed separately for each light sensor of the optical measurement device.

[0198] At 1440, weights are assigned to the data values at the time points within the time interval corresponding to the pulse train, thereby obtaining weighted values. The assigned weights may be from the pulse weights 1332. A weight may be assigned to each of the N pulse trains. Some of such weights for different pulse trains may be the same as for other pulse trains. In some embodiments, at least two of the N pulse trains are assigned different weights and have different pulse patterns. For example, Figure 12 Two pulse patterns with different weights are shown. The two pulse trains may have some similarities (e.g., portions of the pulses may overlap), but there may be at least some time when one pulse train is on and the other is off. Such different pulse patterns may have similar shapes but different delays, e.g., {1,0,1,1,0} has a similar non-zero value shape as {0,1,0,1,1}, but they are different pulse patterns due to the offset, which can be achieved by delaying the second signal relative to the first.

[0199] Thus, the CPOS can detect light and generate a digitized intensity value for each light sampling interval.For each light sampling interval, the CPOS can apply a pulse weight to the digitized intensity value and add the result to the appropriate time bin of the intensity histogram.

[0200] The CPOS tests whether it has sent the required number of decoded pulses at 1450. If the CPOS has sent the required number of decoded pulses, it continues at block 1460, otherwise it loops back to block 1420.

[0201] At 1460, a histogram corresponding to weighted values in a plurality of time intervals is determined. As described above, a counter for a particular time interval of the histogram can be determined by accumulating weighted values at time points within the particular time interval spanning a plurality of time intervals. For example, histogram 1170 is accumulated over two time intervals and includes a time interval of weighted data value 1175.

[0202] At 1470, the signal corresponding to the N pulse trains is detected using the histogram. For example, the CPOS can determine whether the histogram has a sequence of values that matches a matching code (filter). The CPOS can report whether a matching code was found and the matching amplitude. The match can allow the desired signal to be detected relative to noise or interference from other light sources.

[0203] As an example, you can use Figure 9The technique detects a signal. The filter may include a set of values to be applied to a time interval window of the histogram. The filter can slide on the histogram to calculate a filtered histogram, which has counters corresponding to different sliding positions of the curve filter relative to the histogram. Each of the counters of the filtered histogram can correspond to the overlap of the curve filter with the histogram at a specific sliding position. The maximum value of the counter of the filtered histogram can be identified, thereby allowing detection, for example, when the maximum value is above a threshold. The specific sliding position for the maximum value of the counter can correspond to the reception time, which can be used for ranging measurement. Additional details and techniques for using the filter are described herein and can be used with method 1400.

[0204] In some embodiments, the signal may be a reflection signal, such as that caused by N pulse trains reflecting off an object when the optical measurement system is configured to perform ranging measurements. In other embodiments, such as when the light source is in one location and the light sensor is in a different location, the signal may be a communication signal. This configuration can be used for communication purposes. For example, a microwave transmission tower can transmit data to a receiving tower. The transmitted data may include decoded pulses, which can help reduce errors in data reception that may be caused by noise or interference from other sources. The receiving tower can identify the pulse trains and generate a histogram by selecting any time between two pulse trains as the start time of the first time interval. A matched filter can then be applied (e.g., by sliding it over the histogram); and if a sufficient match is found, a communication signal can be detected. A sufficient match can be measured by the maximum value obtained from the filtered histogram. As another example, the system can detect an interfering signal from another CPOS in a manner similar to that used to detect communication signals. If interference is measured, some embodiments may change the transmission code, for example, so that the interfering code is similar to the currently used code.

[0205] At 1480, the distance to the object can be determined. For example, the reception time corresponding to the N pulse trains relative to the start signal can be determined. The reception time can be used to determine the distance to the object. The reception time may be offset from the transmission time of the pulse train, but this offset can be accounted for. Therefore, the CPOS can report the time at which the offset was detected. The distance can correspond to the round-trip time between the reception time and the start time of the start signal, and thus the distance can be expressed in terms of time.

[0206] The detected signal can be used for other purposes besides ranging. For example, the quality of the detected signal can be used to measure the reflectivity of an object. For example, if the detected signal has a high intensity, the system can determine that the object has high reflectivity. The above discusses embodiments for communication and interference measurement. For detection of interference from another light source, the detected signal will come from another set of pulses emitted by the interfering light source.

[0207] As a generalization, an embodiment may transmit N+1 unique codes with N+1 unique weights to generate an N-dimensional vector space histogram. For example, instead of holding intervals of signed numbers, an interval may hold a 1-D vector (e.g., equivalent to a signed number) by transmitting at least two unique codes: one positive and one negative. To store a 2-D vector (e.g., in polar or Cartesian coordinates), the system may transmit at least three unique codes, which may be weighted with three different polar angles and summed to form a single 2-D vector. An N-dimensional vector (defined as N individual numbers all held within a single "interval") would require N+1 different codes, each weighted at a different angle when the vector is summed (in other words, having a component with its weight that is orthogonal to all other weights). By increasing the dimensionality, more advanced decoding techniques may be used, such as quadrature phase decoding or code division multiple access (CDMA) used in RF communications. An N-dimensional matched filter may be used in this context.

[0208] As mentioned above, method 1400 can be used to reduce interference between channels. For example, method 1400 can be repeated for multiple channels of a light source and a light sensor as part of multiple optical measurements. The multiple optical measurements can overlap in time, for example, performed substantially simultaneously. Thus, each channel can perform a measurement simultaneously. To reduce interference, the codes can be different for at least some of the channels. For example, the pulse patterns of the N pulse trains of at least two of the multiple channels can be different, thereby resulting in different histogram patterns for the different channels. Alternatively or in fact, the weights assigned to the N pulse trains of at least two of the multiple channels can be different, thereby resulting in different histogram patterns for the different channels.

[0209] V. Matched Filtering Using Different Curve Filters

[0210] As mentioned above Figure 8 As described, the signal generated from a detected pulse may vary depending on the type of object from which the pulse is reflected. Figure 8As shown, the portion of the resulting signal (e.g., a histogram accumulated over multiple detection intervals) corresponding to a pulse from a highly reflective object (e.g., the first pulse of a decoded pulse pattern) may have a spike at the beginning and then rapidly decrease. This behavior can be achieved when using a SPAD because there is a dead time (e.g., 10-20ns) after the SPAD detects a photon before it can detect another photon. In contrast, a portion of the histogram of a pulse reflected from a standard object may be relatively square, similar to an actual transmitted pulse. Therefore, even if a decoded pulse pattern is used to increase the ability to distinguish reflected pulses from background, other problems may arise due to the different shapes of the detected signals, such as due to the detection of pulses of different intensities. The detected signal may be longer than the transmitted pulse width, such as when a pulse is reflected from an angled surface.

[0211] This behavior can be mitigated by using more SPADs or other photodetectors per pixel (i.e., per pixel sensor), so that even at moderate light fluxes, some SPADs will be on, but there will always be some active. However, at very high light fluxes, there will be situations where most or all of the SPADs are on, and the pixel sensor will not be able to detect many photons at once because most or all of the SPADs are off.

[0212] As a further illustration, assume that the SPAD has a dead time of 20ns, but the high light flux from the reflected pulse is 1 photon every 10ns. The output of the SPAD will indicate only one photon every 20ns. As another example, for a strong 50ns pulse of reflected light, each SPAD can start (register) and then turn off for 20ns. After 20ns, the SPAD will turn back on, register again, turn off, and turn on again. Therefore, the 50ns pulse will appear as three short spikes. This change in the detected signal may cause the decoded pulse to be missed completely, causing confusion about the number of objects, or at least incorrectly determining the reception time of the 50ns pulse. To solve this problem, different matched filters can be used to identify the different attenuation / response / pile-up of the SPADs.

[0213] Thus, embodiments can use filters corresponding to different types of signal profiles (e.g., no / low pile-up, moderate pile-up, and high pile-up). The matching process can determine the type of signal from the light detector of the pixel sensor, thereby allowing for better determination of the reception time for performing the time-of-flight calculation used to determine the distance to the object. In some embodiments, multiple filtering layers can be performed, for example, a coarse filter can identify a value window for performing a more refined analysis, which can be accomplished by interpolating the signal in the window using a subset of interpolation filters (e.g., corresponding to the matched coarse filter). Such refinements are described later.

[0214] A. Application of filter curve to original histogram (and normalization)

[0215] In some embodiments, a curve filter may be applied to the raw histogram. The curve filter may generally correspond to a nominal laser pulse, but the curve filter may be modified to be shorter, longer, or have a different shape or distortion applied based on the expected effect of the target or detector on the nominal pulse, e.g., as Figure 8 The shorter curves may correspond to higher pile-up situations, and the longer curves may correspond to situations where the pulse hits an angled surface, causing the reflected pulse to be smeared in time (e.g., one side of the laser pulse (e.g., left) is closer than the other side of the laser pulse (e.g., right) due to the angled surface). Due to flux smearing, the overall height of the original histogram may be reduced. As other examples, clouds or foliage may also cause smeared pulses. Generating the filter curve may be achieved by simulating different distortion sources and applying their effects to the nominal pulse curve. Alternatively, the filter curve may be derived empirically by performing histogram measurements in the field and saving the digitized pulse curve that appears in the histogram as the filter curve.

[0216] Thus, different curves can take into account different pulse widths due to the orientation of the object, as well as the intensity of the reflected light (e.g. due to the effect of dead time, as for Figure 8 Both effects cause the light sensor to detect photons at different rates over successive time intervals.

[0217] In some embodiments, the curve filters are designed based on a technique of normalized cross-correlation, whereby the highest value will be returned by the filter that most closely matches the shape of the input signal. Thus, in some embodiments, three different filter curves are designed based on curves of digitized signals with no pile-up, with moderate pile-up, and with high pile-up, as shown below. Figure 8 and Figure 15 As seen in. Figure 15 A smearing filter is also shown. To illustrate the application of different curve filters, a single pulse is assumed for one or more pulse trains in the examples below.

[0218] Figure 15 A plurality of curve filters are shown applied to the original histogram 1510 according to an embodiment of the present invention. Figure 15 Various filters that can be applied to an unknown signal to determine the shape of the detected signal, its location, and the time of reception of the pulse that produced the detected signal are shown. In this example, the emitted pulse has a width of 5 time bins. The original histogram 1510 can be much larger, and only a portion is shown for ease of illustration.

[0219] Raw histogram 1510 has one time bin 1515 with a significant value, specifically counter 1512. This shape corresponds to a high pile-up. The type of actual signal shape is unknown to the system, and therefore various filter curves can be applied. Reflected pulse 1517 shows that actual pulses are being received (five time bins), but time bin 1515 is the only time bin with a measurable signal due to the dead time in the last four time bins. Reflected pulse 1517 corresponds to the actual light flux hitting the light sensor, but not all of it is actually detected. In other examples, reflected pulse 1517 can be longer (e.g., due to an angled surface or an increase in the relative distance to the object) or shorter (e.g., a decrease in the relative distance to the object).

[0220] The high pile-up filter 1520 has a time interval with a non-zero value and is similar to Figure 8 830. The high pile-up filter 1520 can be applied to the original histogram 1510 to produce a high filter output 1525 having a single time bin with a significant value, where the time bin corresponds to the time in 1515 as shown by the alignment of the time bins and the original histogram 1510 and the high filter output 1525. As would be expected, the high pile-up filter 1520 provides the best match, as evidenced by the peak 1522 (i.e., the maximum value of the high filter output 1525) being higher than the maximum values of the other filter outputs.

[0221] Moderate pile-up filter 1530 has a similar Figure 8 820 . A moderate pile-up filter 1530 can be applied to the original histogram 1510 to produce a moderate filter output 1535 having multiple time bins with distinct values. The value of the filtered output at a given time bin can be shown at the time corresponding to the leftmost (first) bin of the filter. This can be done when the rising edge of the pulse is detected. Moderate filter output 1535 has a shape that is inverted relative to moderate pile-up filter 1530, as the rightmost tap 1531 will initially overlap with counter 1512. Each shift to the right provides more overlap as the tap filter increases. However, peak 1532 of moderate filter output 1535 is lower than peak 1522.

[0222] The low pile-up filter 1540 has a similar Figure 8The shape of the low pile-up curve 810 is similar to the emitted pulse of 5 time bins. The low pile-up filter 1540 can be applied to the original histogram 1510 to produce a low filter output 1545, which has multiple time bins with significant values. The maximum value in the low filter output 1545 is lower than the peak 1522. The peak is difficult to identify because the filtered output is flat. However, as shown later ( Figure 16 ) will describe that, for example, additional filter layers may be applied when the first layer is a coarse filter layer. Thus, there are some embodiments that use an initial coarse filter that may be similar to the low pile-up filter 1540, and then apply a second stage filter to the coarse filter output.

[0223] The tailing filter 1550 has a shape corresponding to a reflected pulse that is longer than the transmitted pulse (9 time bins versus 5 time bins). Like the other filters, the tailing filter 1550 can be applied to the raw histogram 1510 to produce a tailing filter output 1555. This can be used to detect targets with temporally trailing pulse widths, due to high angles of incidence to the target, or because the target is a diffuse medium such as shrubs or, in extreme cases, a patch of fog. Filters 1520, 1530, 1540, and 1550 can each have an expected signal profile corresponding to a different digitized signal profile.

[0224] Since the peak 1522 of the high filter output 1525 is the global maximum from all the various filter outputs, time bin 1515 is selected as the time location of the reflected pulse. The matched filter circuit may not know that only one time bin has a significant value, and it just happens that time bin 1515 of the high filter output 1525 has the highest value of any other filter. If desired, the width of the outgoing pulse can be used to determine the center of the outgoing pulse, for example by adding half the width to the time of time bin 1515.

[0225] If the digitized signal has moderate pile-up, the moderate filter output will have the highest value because most of the overlap will occur when the filter exactly overlaps the original histogram. Also, the peak of the moderate filter output will also be in time interval 1515. Similarly, if the digitized signal has low pile-up, the low filter output 1545 will have the highest value, and similarly for the trailing filter 1550 for the trailing digitized signal.

[0226] In order to guarantee that global maximum corresponds to the peak of correct filter output, the power of various filters can be through normalization.For instance, as shown in the figure, a tap in high pile-up filter 1520 has the value higher than any other filter.Similarly, the highest tap in appropriate pile-up filter 1530 is higher than the tap in low pile-up filter 1540, and the tap in low pile-up filter is higher than the tap in tailing filter 1550.In certain embodiments, the root mean square of each group of filter taps is equal.In another embodiment, the total integrated area of each filter can equate, and then forces value to reduce when adding more taps.

[0227] Therefore, before comparing the individual filtered signals, these individual signals can be normalized. In one embodiment (as shown), the normalized comparison is accomplished by performing a cross-correlation (matched filter) using normalized filters. To generate a normalized filter, the root mean square of each set of filter taps can be calculated, and then the entire filter is divided by the square root of the energy. Other functions besides square root can be used.

[0228] In other embodiments, the normalized comparison is performed by first applying a non-normalized filter and then dividing the result (i.e., the filter output) by the square root of the filter energy. This latter embodiment can be computationally simpler. After the normalized comparison of multiple filtered signals, the maximum value corresponds to the best match. In this way, according to embodiments of the present invention, information about the properties of the reflective surface (e.g., the angle of the surface, how reflective it is, and the relative speed of the object) can be obtained.

[0229] As part of generating the filter output, the sliding step can be a time interval, such as Figure 15 In other embodiments, a smaller step size may be used. For example, an overlap value may be determined every 0.1, 0.2, or 0.5 time intervals (e.g., by convolving a filter with the original histogram).

[0230] B. Application of Curve Filter to Filtered Histogram

[0231] In some embodiments, different curve filters may be applied to the filtered signal, for example, produced by an initial coarse filter. This initial coarse filter may be applied to the original histogram to identify the portion of the histogram corresponding to the reflected pulse (i.e., distinct from the background light). The resulting filter output may have different curves for different digitized signal curves.

[0232] 1. Comparison between different curve filters

[0233] Figure 16The application of different second-stage curve filters to the filtered output from the first-stage filter is shown according to an embodiment of the present invention. Figure 16 There are three columns and five rows. The three columns are no pile-up 1610, moderate pile-up 1620, and high pile-up 1630. The rows correspond to: (A) the original histogram, (B) the coarse filter output (e.g., from a single square filter) applied at the first filter stage, (C) the no pile-up filter output applied at the second filter stage, (D) the moderate pile-up filter output applied at the second filter stage, and (E) the high pile-up filter output applied at the second filter stage.

[0234] The no-pile-up signal 1611, the moderate-pile-up signal 1621, and the high-pile-up signal 1631 correspond to the low-pile-up curve 810, the moderate-pile-up curve 820, and the high-pile-up curve 830, respectively. The actual light flux 1650 on the pixel is a square pulse shown in dashed lines.

[0235] exist Figure 16 In

[15] , a single square filter is applied to each of the raw histograms. The square filter may have a pulse width. This square filter can be considered coarse because it is applied at the first level (i.e., directly to the raw histogram) and the resolution may be the size of a single time interval (e.g., the sliding step is one time interval). Different filter outputs are derived from different raw histogram inputs. The filter output is determined by determining the overlap of the square filters and the signal at different positions of the square filters.

[0236] The coarse non-piling filter output 1612 has a triangular shape, for example similar to Figure 9 The final filtered output 990 is plotted as follows. The horizontal axis on the filtered histogram corresponds to a given position of the square filter. But the actual time position of the filter output 1612 relative to the no-pile-up signal 611 does not necessarily relate to its relative position on the horizontal axis. If the filter output is generated to detect the rising edge of the reflected pulse (e.g., the filtered value at the time interval corresponds to the leftmost position of the filter), then the peak of the filter output 1612 will correspond to the rising edge of the no-pile-up signal 1611. In addition, the width of the filter output 1612 is greater than the width of the no-pile-up signal 1611 (e.g., 9 intervals versus 5 intervals). Therefore, the scale on the horizontal axis is different for the two plots, but line 1617 can correspond to the center of the pulse detected when the value of the time interval is disconnected from the center of the coarse filter, as shown in FIG. Figure 9 Like in.

[0237] Coarse moderate pile-up filter output 1622 increases with each time step of the first portion, e.g., similar to moderate filter output 1535. The decrease at time 1627 (e.g., corresponding to the center of the detected pulse) can result from the leading edge of the square filter passing a peak in moderate pile-up signal 1621.

[0238] The coarse high-pileup filter output 1632 resembles a square pulse, similar to the low-pileup filter output 1545, for example, because the square filter can be considered a low-pileup filter at this first stage. The high-pileup signal 1631 can be approximated as a single time interval with a significant value above the threshold, similar to the original histogram 1510. Line 1637 illustrates the ambiguity associated with a flat-top filter response. The maximum value of the filter response can easily occur anywhere along the flat top of the coarse high-pileup filter output 1632, where noise is added.

[0239] Rows CE correspond to second-order filters applied to the outputs of the square filters in row B. Each row corresponds to a different second-order filter applied to the output of the corresponding square filter in row B. The three different second-order filters have curves that roughly correspond to the curves of the outputs in row B. Thus, the second-order filters can be considered to be different curve filters having the curves shown in row B. The second-order filter that corresponds to the underlying digitized signal (i.e., is of a similar shape to the corresponding filter output) will provide the best match and, therefore, be used to determine the reception time.

[0240] In row C, a curve filter output 1613 is generated by applying a no-pile-up curve filter to filter output 1612. In this example, the no-pile-up curve filter is similar in shape to filter output 1612 itself (but with a more exponential rise and decay, e.g., Gaussian), and is not similar to low-pile-up filter 1540. These curve filters differ in that they are applied at different stages: one directly to the raw histogram and the other to the first-stage filter output. Since filter output 1612 is essentially convolved with itself, curve filter output 1613 is symmetrical and may have a peak at substantially the same time as filter output 1612.

[0241] Curve filter output 1623 is produced by applying a no-pile-up curve filter (essentially filter output 1612) to coarse fit filter output 1622. The asymmetric shape of coarse fit filter output 1622 results in the asymmetric shape in curve filter output 1623. Also, because the no-pile-up curve filter does not exactly match coarse fit filter output 1622, the peak of curve filter output 1623 is smaller than the maximum value 1616 of curve filter output 1613.

[0242] Curve filter output 1633 is produced by applying a no-pile-up curve filter to coarse high-pile-up filter output 1632. The generally symmetrical shape of coarse high-pile-up filter output 1632 provides a symmetrical shape in curve filter output 1633. Also, because the no-pile-up curve filter does not exactly match the coarse high-pile-up filter output 1632, the peak of curve filter output 1633 is smaller than maximum value 1616.

[0243] In row D, a moderate pile-up curve filter is applied to the various filter outputs in row B. The moderate pile-up curve filter has a similar shape to the coarse moderate filter output 1622. Thus, filter output 1624 has a maximum value relative to filter output 1614 (no pile-up) and filter output 1634 (high pile-up).

[0244] In row E, a high pile-up curve filter is applied to the various filter outputs in row B. The high pile-up curve filter has a shape similar to the coarse high pile-up filter output 1632 (i.e., a substantially square filter). Thus, the filter output 1635 has a maximum relative to the filter output 1615 (no pile-up) and the moderate output 1625 (moderate pile-up).

[0245] 2. Maximum Window Finder - Moderate Stacking Example

[0246] In some embodiments, the first-order filter can identify a specific window for performing the second-order filtering. The first-order filter can be applied to the entire original histogram because it is not known exactly where the reflected pulse will be detected. The location of the maximum value of the filter output can be identified, and the window around the maximum value can be analyzed using a second-order curve filter. Memory, time, and computational workload can be saved by applying the curve filter only in this maximum value window. Therefore, by applying each of these filters to the maximum value window results and comparing them to find which one has the highest value, it is possible to find which accumulation situation best approximates the true signal.

[0247] Figure 17 17 is a diagram illustrating a process flow for using a two-stage filter applied to a moderately piled-up signal according to an embodiment of the present invention. A first-stage filter (e.g., a square filter) can be applied to the moderately piled-up signal to obtain a moderately piled-up filter output 1710, which can appear as a rough moderately piled-up filter output 1622. As part of obtaining the moderately piled-up filter output 1710, the first-stage filter can be slid across the original histogram and a maximum value in the filter output can be identified. A window around this maximum value can be selected, wherein the window will contain a relevant portion of the moderately piled-up filter output 1710, thereby obtaining a selected maximum window 1720.

[0248] Once the maximum window 1720 is selected, a plurality of second-order curve filters may be selected. The selection of which second-order curve filters may depend on the properties of the filtered output (e.g., the pattern of non-zero values, such as their width), and the properties of the first-order filter used (e.g., the pattern of non-zero values, such as their width). The selected second-order curve filter may then be applied to the filter output data within the selected maximum window 1720.

[0249] In this example, three second-order curve filters 1722-1726 are selected, for example, corresponding to the expected signal curve, or more precisely, corresponding to the filtered output of such a signal curve. The low pile-up curve filter 1722 provides the leftmost second-order filter output, the moderate pile-up curve filter 1724 provides the middle second-order filter output, and the high pile-up curve filter 1726 provides the rightmost second-order filter output. Since the original raw histogram has moderate pile-up and the selected maximum window 1720, the selected signal 1732 will be the middle second-order output.

[0250] In some embodiments, more than one curve filter can be selected for a given shape. For example, multiple moderately stacked curve filters can be used, each with slightly different properties depending on when the rising edge of the reflected pulse is received within the time bin. This set of curve filters can effectively interpolate the detected raw histogram to identify the reception time of the reflected pulse at a resolution finer than the width of the time bin. The next section provides further details on this interpolation.

[0251] C. Interpolation

[0252] In certain embodiments, interpolator is carried out the interpolation between the histogram interval.Digital interpolation is usually completed by zero filling and finite impulse response (FIR) or infinite impulse response (IIR) filter is applied to data.In other embodiments, use more accurate and economical scheme.In these embodiments, interpolator uses the prior knowledge according to the signal shape identified above.Therefore, interpolator can be used many matched filters, each of which corresponds to the interpolation offset less than 1 histogram interval.As an exemplary embodiment, interpolator realizes this point by using 10 different matched filters of every curve, each of which corresponds to different interpolation offsets.

[0253] Therefore, a block in a digital signal processor (DSP) solution may include an interpolator, which may use an interpolation filter. The interpolation filter may be a first-order filter or a second-order filter. The interpolation filter may provide information about the reception time of a detected pulse with a greater accuracy than the width of the time interval.

[0254] As an example of interpolation, assume that the rising edge of a square reflected pulse arrives exactly in the middle of a time interval (e.g., 500 ps in a one nanosecond time interval), where the detected pulse is 3 ns wide. Since the rising edge arrives halfway through the time interval, the photon flux that will be detected in the first time interval is approximately half of that in the next two time intervals. The fourth time interval will also have approximately half the detected flux because the falling edge will fall halfway through the fourth time interval. Thus, the original histogram might have four values {5, 10, 10, 5} in four consecutive time intervals. '10' indicates that the full light flux was detected in that interval, and '5' indicates that half of the full flux was detected in that interval.

[0255] The ratio of the values in the leading and tail intervals can be used to determine the position of the rising edge. For example, if the leading interval has a value of 4 and the tail interval has a value of 6, the rising edge will be at 600ps (e.g., 100.6ns) entering the leading interval. In fact, the specific value of the original histogram may not have such ideal values, for example, because of the random nature of noise and detection process. Therefore, rather than calculating this ratio, different interpolation filters can be used. For example, if the leading interval is between 100ns and 101ns, there may be an interpolation filter for 100.1ns, 100.2ns, 100.3ns, etc. In this example, the best matched filter can identify the reception time in 100ps. Such interpolation filters can perform fixed-point multiplication of more bits than the bits used for coarse filters when applying interpolation filters as second-order filters.

[0256] Different sets of interpolation filters can be used for different curves. For example, one set of interpolation filters can be used for a high-pile-up curve, and a different set of interpolation filters can be used for a moderate-pile-up curve, and so on. Each interpolation filter in a set can have a slightly different curve that accounts for the slight quantization shift in the pulse curve as the filter slides in time. To interpolate 10x (e.g., 0.1 accuracy when the width of the time interval is 1), 10 interpolation filters can be used. In the simple case of a 2-interval rectangular pulse (e.g., a high-pile-up curve), the interpolation filters can be (10, 0), (9, 1), (8, 2), (7, 3), (6, 4), (5, 5), (4, 6), (3, 7), (2, 8), (1, 9). Each of these interpolation filters can be applied to the signal to identify the best-fit signal, resulting in an interpolated range as low as 1 / 10 of the interval. Those skilled in the art will appreciate the application of such interpolation filters to pulses with more than two time intervals, as well as more complex pulse shapes and lengths. Furthermore, consecutive pulse trains (eg, consecutive pulses of the same pulse pattern) may be offset from each other (eg, by 1 / 10 of a staggered interval) to ensure that the original histogram occupies more than one time interval, as described in more detail in a later section.

[0257] 1. Application of interpolation filter to original histogram

[0258] Figure 18 FIG18 illustrates the application of different interpolation filters to an original histogram according to an embodiment of the present invention. FIG18 illustrates four interpolation filters applied to an original histogram 1810 generated in response to a 3-ns reflected pulse 1817. The original histogram 1810 (labeled H) has a rising edge 1815 that is shifted by a quarter of a time interval relative to a time interval 1812. Each of the interpolation filters may have a different pattern of values in the corresponding time interval to provide interpolation, e.g., a different pattern of non-zero values.

[0259] The zero-shift interpolation filter 1820 includes three taps with the same value. The zero-shift interpolation filter 1820 is convolved with H to provide a zero-shift output 1825. The convolution process can provide J+K-1 time bins with significant values, where J is the number of taps of the interpolation filter and K is the number of time bins of significant values (e.g., above the background threshold) of the original histogram.

[0260] At each time step, a convolution (e.g., overlap) can be determined between the two curves, and the resulting value can be used for the time interval corresponding to the location of the first interpolation tap. In this example, peak 1827 occurs when the leftmost interpolation tap coincides with time interval 1812 because these three time intervals have higher values than time interval 1811.

[0261] -3 / 4 shift interpolation filter 1830 has a higher leading tap than a trailing tap because the time interval 3 1831. The convolution provides a filtered output 1835 having a peak 1837 that is lower than the peak 1827 because a zero shift is closer to an actual shift of -1 / 4 than a -3 / 4 shift.

[0262] The -1 / 4 shift interpolation filter 1840 has a lower leading tap than a trailing tap because the time interval 1 1841. The convolution provides a filtered output 1845 having a peak 1847, which is the highest peak of all the filtered outputs because the -1 / 4 shift is the same as the actual shift of the rising edge 1815.

[0263] The -1 / 2 shift interpolation filter 1850 has equal leading and trailing taps because each corresponding 1 / 2 of the time contains a reflected pulse when it has a rising edge 1851. The convolution provides a filtered output 1855 having a peak 1857 that is approximately equal to peak 1857 because zero shift is the same distance as the actual shift of -1 / 4 compared to zero shift.

[0264] Figure 18 The example of assuming no pile-up filter, but a specific curve may be unknown. In this case, various curve filters can be applied to the original histogram, wherein each curve has different widths. These curve filters can be coarse filters respectively, because the purpose of this first level is to identify which curve best matches the digitized signal. Such different curve filters or their outputs can be normalized, as described herein. Depending on which curve filter best matches, the interpolation filter of the type (e.g., high pile-up) can be applied in the interpolation stage subsequently. The initial coarse filter with the best match can be one of the group of interpolation filters, for example, corresponding to the specific shift (e.g., zero shift for a square coarse filter) in the rising edge.

[0265] Interpolation filter is applied to original histogram and can be more effective than being applied to filter output (for example, interpolation filter is second order filter) when detecting the signal in noise, because filter can be matched to the multiple signal shape in histogram better, but may be expensive on calculation, because the position of pulse may be unknown.In this example, each in interpolation filter can be applied to whole original histogram.But, in one embodiment, the peak position of first interpolation can be used to specify where other interpolation filters should be applied.In other embodiments, can first apply one or more coarse filter curves (for example, as described herein) to realize the many benefits of running interpolation filter on original histogram, it can be used for finding the window around reflected pulse, and then interpolation filter can be applied.

[0266] 2. Application to filtered histogram

[0267] In some embodiments, an interpolation filter may be applied to the filter output rather than directly to the original histogram. Figure 16 , when applying a second stage filter to provide various second stage filter outputs, more than one second stage filter may be applied. Thus, there may be more than one high pile-up second stage filter. Figure 16 The medium, high pile-up second stage filter is nominally similar to the coarse high pile-up filter output 632. However, instead of a single square filter, multiple (e.g., 10) interpolation filters can be used, each with a different curve corresponding to a different position of the rising edge of the reflected pulse, such as similar to Figure 18. The number of interpolated pulses can depend on the required accuracy.

[0268] Thus, if the original high-pile signal has one time bin with a large value, but the next time bin has a smaller but significant value (e.g., due to the rising edge of a strong pulse arriving near the end of the first time bin), the filtered output will not be perfectly square. For example, with a 5-bin square filter, there will be four time bins of equal height, with two smaller time bins on each side, where the leftmost time bin is larger than the rightmost time bin because the original histogram had a larger first bin. If only a 5-bin square filter is used as the second-stage filter, the maximum value will appear the same as if only one time bin had a significant value (i.e., the next time bin is essentially zero, e.g., less than background), which will not be as accurate as using multiple interpolation filters.

[0269] The interpolation filter can use 6 taps, where the first and last bins are different, for example, so that the filtered output will exactly match the best-matched interpolation filter within a specified resolution. For example, using 20 interpolation filters with different combinations of first and last taps can provide an accuracy of 50 ps. Depending on the relative ratio of the two time bins in the original histogram, the maximum value will shift slightly; calculating this ratio is another way to determine the position of the rising edge, i.e., once the coarse filter has been used to identify the portion of the original histogram to be analyzed.

[0270] In some embodiments, a single second order curve filter may be used for each of the curves, e.g. Figure 16 The three curves shown in FIG. The curve filter that provides the maximum value can then be used to select the corresponding interpolation filter to use, for example in a manner similar to how the initial coarse filter can be used to select which interpolation filters to apply to the original histogram, but here the interpolation filter (corresponding to the best matching coarse filter) is a second-order filter. Therefore, not all interpolation filters for each curve need to be used.

[0271] 3. Select filter

[0272] As mentioned in the previous part, as an alternative to applying all interpolator filter curves to the measured histogram in all measurements, only specific interpolation filters are used. Because some interpolation filters are associated with some coarse filters or belong to a group jointly, the best match of a filter can be used to select which interpolation (or additional interpolation) filters to use. In various embodiments, these interpolation filters can be selected when they correspond to, for example, the best match coarse filter, preceding N best match coarse filters (e.g., 2 or more), the best match interpolation filter of a particular group (e.g., a high pile-up interpolation group), best N match interpolation filters, or any filter higher than the matching of a threshold.

[0273] In other embodiments, a set of interpolation filters can be selected based on measurements such as the maximum value in the raw histogram and / or the maximum value in the filter output. As an example, by comparing the ratio of these two numbers, it is possible to determine a threshold that reliably identifies certain levels of pile-up. For example, if the maximum value in the raw histogram is equal to the maximum output of the multi-tap coarse filter, this can indicate very strong pile-up because all the light energy is contained in a single histogram bin. This simple ratio approach is a computationally efficient alternative for selecting which interpolation filter to apply.

[0274] In other embodiments that use the maximum value of the filter output as the selection criterion, additional checks can be implemented. For example, if there are multiple SPADs per pixel and most of them are activated, the system can assume that pile-up has occurred. The degree of pile-up (related to the maximum value in the raw histogram) can be directly related to the number of detected photons. To exploit this relationship, a threshold or multiple thresholds on the maximum value in the raw histogram or on the coarse filter output can determine which pile-up curves to use.

[0275] For example, a maximum value above a first threshold may indicate that only a high pile-up interpolation filter should be used. A maximum value below the first threshold may indicate that a low and moderate (medium) pile-up interpolation filter should be used. A second threshold may indicate that the moderate and high pile-up interpolation filters should be used when the maximum value is above the second threshold, and that the low pile-up interpolation filter should be used when the maximum value is below the second threshold. The values used for these thresholds may be determined based on the current operating set point of the device in addition to static or real-time knowledge of the environment in which the sensor is operating.

[0276] 4. Change

[0277] Interpolation can be performed in additional ways. For example, a distribution function (e.g., Gaussian) can have a width similar to the width of the filtered output (or original histogram, depending on when interpolation is performed). Slowly moving this distribution function (e.g., 100ps step size) will provide an approximation of the center of the original histogram or filtered histogram. This will not require a 10-tap filter for each curve, but may require more than one filter value per nanosecond. Such a distribution can be symmetrical or asymmetrical, for example, with different curves within it. Different widths of the distribution can be used, which can provide benefits over using multi-tap filters with different patterns for a single curve.

[0278] D. Combination of multiple coarse filters and interpolation filters

[0279] As described above, a coarse filter can be applied to the raw histogram to provide a filter output, and an interpolation filter having a best match curve can be applied to the filter output. In some embodiments, multiple coarse filters can be applied, and the interpolation filter corresponding to the best match can be used. Since the interpolation filters correspond to the best match curve, they can be part of a digital signal processing (DSP) solution designed to identify the type of accumulation and thereby more accurately identify the distance and nature of objects in the scene.

[0280] Figure 19 is a diagram illustrating a two-stage filtering scheme using multiple coarse filters according to an embodiment of the present invention. Histogram data 1910 corresponds to an original histogram, which can be determined using multiple pulse trains.

[0281] According to some embodiments, to estimate the shape of the digitized signal, a plurality of matched coarse filters are first applied to the original histogram. Figure 19 In the figure, the coarse filters are labeled as matched filters 1-N. Each of these coarse filters can provide a filter output, labeled as filtered curves 1-N. In some embodiments, a Barker code matched filter is applied. This filter uses Barker codes as filter taps and performs a cross-correlation of the original histogram with these taps. According to an embodiment of the present invention, there may be N different matched filters. Therefore, the histogram data from each measurement can be filtered with N different matched filters. The number of coarse filters can correspond to the number of different curves expected or possible in a given environment (e.g., based on the type of object) and based on the shape of the curve from the light sensing module.

[0282] In the next step, a maximum window finder 1940 can identify the maximum value of the output signal that matches the filter. A window of values around the index can be saved. If multiple coarse matched filters are used, only the normalized matched filter with the maximum value is considered. The coarse matched filter with the maximum value is recorded.

[0283] The maximum value window is passed to the interpolator to perform fine interpolation. The interpolation filter to be used is determined by the coarse matched filter with the maximum value. Therefore, there may be a total of N*M interpolator filters, where N is the number of coarse matched filters applied to each measurement, and M is the number of fine interpolation filters for each coarse filter. In various embodiments, the interpolation filter can be applied to the original histogram or to the filtered curve corresponding to the best match.

[0284] E. Decoded Pulse and Curve Filters

[0285] Matched filters (including different curve filters) can also be used in combination with the decoded pulses described in the previous section. Below is an example of a coarse filter with a tap pattern matched to the pulse pattern. Also, the operation of the maximum window finder is described.

[0286] 1. Coarse filter

[0287] Figure 201. The raw histogram, matched filter, and corresponding filter outputs generated from decoded pulses according to an embodiment of the present invention are shown. The raw histogram 2010 represents a no-pile or low-pile situation, but with some noise. The raw histogram 2010 has a {+1, +1, -1} pattern corresponding to a Barker code of length 3. The values in the various time intervals from 0-16 are between -2 and 2. Negative values can be achieved by assigning negative weights to pulses detected during a specific portion of the detection interval. In the example shown, the raw histogram 2010 only has integer values, i.e., -2, -1, 0, 1, or 2.

[0288] The matched filter 2020 has filter taps that match the expected pulse pattern {+1+1-1}. However, the taps only have values of +1, 0, and -1. The matched filter 2020 has one tap 2022 for each time interval. A cross-correlation is performed between the histogram input data and the filter taps 2022. The matched filter 2020 can constitute a coarse filter.

[0289] Filter output 2030 shows the cross-correlation of the input signal (raw histogram 2010) with filter tap 2022, resulting in a roughly filtered signal. As can be seen in filter output 2030, there is a central positive peak in the cross-correlation function. This combines all the power from all three pulses in the histogram into a single large pulse. In addition, the cross-correlation of the input histogram with the matched filter produces a triangular shape as seen in filter output 2030. The scale on the vertical axis of filter output 2030 illustrates the cross-correlation aspect by essentially providing the sum of the data values in the time interval of the raw histogram 2010 when the matched filter 2020 is in a sliding time step that exactly matches the pattern of the raw histogram 2010.

[0290] In some embodiments, cross-correlation can be performed without floating-point operations when the values of the original histogram 2010 and the matched filter 2020 are integers. A more efficient fixed-point modification can be used. Also, in this example, since the matched filter 2020 only has values of -1, 0, or +1, an efficient implementation of this first-stage filter can be important because the coarse filter can be applied to the entire histogram (e.g., 1,000 time bins).

[0291] This coarse filter allows identification of where the signal is located on a rough scale. This can be useful when emitting a single laser pulse, as these additional pulses may otherwise increase computational requirements. Application of a matched filter can reassemble the pulses into a single signal corresponding to the peak at time interval 15 in the filter output 2030.

[0292] Figure 20This illustrates the benefits of using decoded pulses. The original histogram 2010 is a very weak signal, i.e., the values in the time bins are small and vary due to noise. When the matched filter is precisely matched to the histogram input, the resulting signal becomes much stronger, as evidenced by the clear maximum in the filter output 2030. Furthermore, the sidelobes are negative, a property of Barker codes.

[0293] 2. Maximum Window Finder

[0294] Figure 19 As shown in FIG, the next block in the DSP solution may be a maximum window finder. The purpose of this block is to find the maximum value of the filter output and save the values around this maximum value for interpolation and / or application of different curve filters. The maximum window finder can identify the maximum value in each matched filter and determine the global maximum value from the individual maxima.

[0295] In certain embodiments, in order to compare different filter outputs, matched filter can be through normalization, for example as described herein.For example, normalization can be with individual maximum value divided by the number of taps.This can provide the average power that is captured in matched filter and can reduce even when the filter with less tap is better matched to input shape, the filter with more taps improperly takes precedence over the filter with less tap and is selected possibility.In the embodiment that wherein filter output directly compares each other, the filter power of each in the filter curve can be through normalization so that relatively can not be biased.

[0296] Figure 21 Shown is the maximum windowing device result 2100 according to an embodiment of the present invention.Find maximum value in time interval (index) 15, and around the maximum index, preserve the window 2110 of 11 values.These 11 values can be preserved and used in second-order filtering, and second-order filtering can relate to interpolation and / or different curve filters.For example, low pile-up curve filter, appropriate pile-up curve filter and high pile-up curve filter can be applicable to 11 values in the time interval of window 2110.The best match of these curve filters can be used to select one group of interpolation filters for receiving time with the resolution greater than a time interval, as described herein.In this way, compared with using complete one group of interpolation filters (possibly for different curves) to whole original histogram, these 11 values are carried out fine interpolation and cost is much lower and much faster in computation.The value of interpolator filter tap can be any real value, rather than being limited only to the value of +1, 0 and -1.

[0297] 3. Multiple rough curve filters for different objects

[0298] Using the techniques described herein can make LIDAR systems more efficient at detecting certain objects. For example, street signs are known to have strong reflections. This can cause high pile-up in sensors that include SPADs. Using appropriate matched filters and corresponding interpolation can greatly improve detection and proper distance estimation of such surfaces. Similarly, radiation reflections from tilted surfaces can be smeared and spread out, which can also be better detected and have better estimated distances using such embodiments.

[0299] Figure 22 The application of multiple coarse curve filters with different widths according to an embodiment of the present invention is shown. The outputs of the different coarse curves can be used to identify which second stage filters to use. Figure 22 Specific scenarios and corresponding processing corresponding to reflections of laser pulses from three different types of surfaces according to an embodiment of the present invention are schematically illustrated.As shown, each of the coarse curve filters has a different number of consecutive time intervals with non-zero values.

[0300] exist Figure 22 In Figure 2, consecutive laser pulses in pulse train 2210 are separated by a time interval t1, and they have a specific, well-defined pulse shape. When these pulses are reflected from different surfaces and the reflected signals are measured by the SPAD at the pixel sensor, the resulting detected signals have different curves depending on the object they reflected from, even if they all have the same separation time t1 in the original pulse train.

[0301] Low pile-up digitized signal 2222 represents a weak return signal, which is an indication of a reflection from a distant object or a reflection from a low reflectivity surface. These two scenarios can be further distinguished by time-of-flight calculations. The reflected signal has the same spacing t1 as pulse train 2210 and has a detected pulse width of t2. Weak matched filter 2224 has a width corresponding to the weak return signal. Weak filter output 2226 corresponds to the application of weak matched filter 2224 to low pile-up digitized signal 2222. The resulting shape is similar to that shown in FIG22.

[0302] The high pile-up digitized signal 2232 represents a strong return signal, which is an indication of reflection from a nearby object or reflection from a highly reflective surface. Similarly, these two scenarios can be further distinguished by time-of-flight calculations. The reflected signal has the same spacing t1 as the pulse train 2210, but has a pulse width t3 that is less than t2. This occurs due to the large amount of reflected radiation that causes high pile-up at the SPAD. Therefore, the digitized signal curve from the SPAD has a typical sharp rise and rapid fall, which makes the full width at half maximum (FWHM) smaller. The strong matched filter 2234 has a curve suitable for calculating the cross-correlation with the high pile-up digitized signal 2032. The strong filter output 2236 has a different peak in the middle of the plot in a similar manner to the weak filter output 2226, but has a different shape in the middle.

[0303] Trailing digitized signal 2242 represents the trailing return signal, an indicator of reflection from an inclined surface relative to the optical axis of the laser radiation. The reflected signal shares the same spacing t1 as pulse train 2210 and has a pulse width t4 that is greater than t2. This can occur because the reflected radiation from the inclined surface reaches the SPAD over a longer period of time, as close edges are closer together and far edges are farther from the detector. Consequently, the digitized signal curve from the SPAD is spread out, resulting in a longer duration. Trailing matched filter 2244 has a suitable spreading curve for calculating cross-correlations with trailing digitized signal 2242. The trailing filter output 2246 has a distinct peak in the middle of the plot, which is more spread out than the other two cases shown above, but will exhibit a higher peak and a more accurate position estimate than filters 2224 and 2234. Coarse matched filters 2224, 2234, and 2244 can correspond to the expected signal curve corresponding to emitted pulse train 2210.

[0304] It should be noted that even though three curves are discussed here, the techniques of embodiments of the present invention can also encompass multiple curves associated with various types of reflections and corresponding scenarios. Any interpolation filter described herein can have various sizes, for example, up to 17 bins wide. For the tailing case, all 17 may be non-zero.

[0305] In some embodiments, such different coarse matched filters 2224, 2234, and 2244 can be used to determine the best match to the crossover. The various widths of such filters can be selected based on the width of the transmitted pulse and can therefore be greater or less than the transmitted pulse. The width of the square filter and its best match position can identify the location of the rising edge of the pulse. This stop may only produce 1ns accuracy, but this can still be better than the error caused by using only one coarse curved filter. For higher accuracy, an interpolation filter with a width similar to the best matching coarse filter can be used. In some embodiments, the corresponding interpolation filter can be loaded into memory in real time to be applied to the current raw histogram.

[0306] The reason for using multiple matched filters can be twofold: 1) to account for the nonlinear response of the SPAD to varying signal power, and 2) to better detect pulses that are trailing in the time domain (which also corresponds to pulses that are trailing in the physical domain, such as highly angled or diffuse surfaces). The ability to identify trailing pulses is a useful feature that can be output to later stages of the LIDAR system or to the end user for, for example, object classification. For example, a series of tree-like points can be better classified as a tree if the system is provided with information that the trunk has non-trailing points and the crown has many trailing pulses, indicating diffuse foliage.

[0307] In addition, the use of matched filters tuned for trailing pulses allows detection of extremely angled surfaces (like road surfaces) at greater distances than conventional systems because the trailing filters capture a higher portion of the signal energy. This can be significant because pulses hitting the road surface far ahead of the car always hit at a very high angle, which causes the pulse to trail significantly in time. Without a set of tuned filters for optimally detecting trailing pulses, the range at which a LIDAR system can adequately identify the road can be limited. This type of detection of objects at great distances can provide significant advantages when decisions need to be made, either by autonomous vehicles or simply by warning systems.

[0308] As mentioned above, using multiple first-stage (first-order) coarse filters alone can provide increased accuracy. For example, a pulse that is temporally streaked by 10x but filtered with a nominal filter that is 1x wide will have a filtered peak and a signal-to-noise ratio (SNR) that is 10x lower than the unstreaked pulse. This has a high probability of not being correctly detected and not being passed to the second-stage filter, and may actually pass a random noise spike. If a second first-stage filter is used in parallel (or in series) with a tap tuned to detect 10x streaked pulses, the filtered SNR can be reduced from 10x to sqrt(10)x, i.e., only 3.16x lower in SNR than the unstreaked case.

[0309] F. System Components

[0310] Figure 23 A filtering optical system 2310 is shown according to an embodiment of the present invention. Figure 23 The components can be Figure 13 The components operate in a similar manner and include Figure 13 components, but for clarity can be found in Figure 13 Some components are omitted. For example, the filtering optical system 2310 can be Figure 13 The filtering optical system 2310 may interact with the user interface hardware 2315, which may be part of the same system as the CPOS 1310. Figure 2 The user interface 215 and / or Figure 13 The user interface hardware 2315 may specify the matched filter to be used, among other operating parameters. The user interface 1315 may display the filtering results, which may include a 3-dimensional map of detected objects and distance values for specific objects.

[0311] The system controller 2320 can communicate with the optical sensing module 2330 and the optical transmission module 2340, for example. Figure 13 The system controller 1320 of FIG. 2340 may be implemented in a similar manner. The light transmission module 2340 may include a pulse code 2344 and an array of emitters 2342 capable of transmitting light (eg, one or more laser diodes). The system controller 2320 may include a DSP 2322 for analyzing data received from the light sensing module 2330.

[0312] Light sensing module 2330 includes one or more coarse filters 2332, a histogram 2334, a sensor array 2336, and a maximum window finder 2338. Sensor array 2336 detects light photons and generates digitized intensity values for each time interval, for example, based on a time-to-digital converter. Data values from sensor array 2336 can be stored in histogram 2334, for example, one memory cell per time interval. Histogram 2334 can be generated for a given measurement time interval and then cleared for a new measurement time interval. In some embodiments, multiple histograms can be calculated for different overlapping measurement time intervals, for example, where data values from a particular detection interval may contribute to multiple histograms, each histogram for a different overlapping measurement time interval containing the particular detection interval.

[0313] Coarse filter 2332 can be stored as a sequence of integers or floating-point numbers in a register or other memory location. For example, when multiple coarse filters are used, each coarse filter can be associated with a label indicating the width or curve corresponding to the coarse filter. The processor of the light sensing module can select the coarse filter to use based on the data values in histogram 2334, for example. For example, the maximum value in histogram 2334 can indicate which coarse filter(s) to use, as described herein.

[0314] The maximum window finder 2338 can analyze one or more outputs of one or more coarse filters, as described herein. The maximum window finder 2338 or another processing module can determine which coarse filter is the best match, for example, based on the maximum value of each filter output. In some embodiments, the light sensing module 2330 can output certain values in the best match filter output (e.g., in the window identified by the maximum window finder 2338) and in the window.

[0315] DSP 2322 can use interpolation filters to analyze the filter output within the identified window. In some embodiments, the light sensing module can indicate which coarse filter is the best match, for example, so that certain interpolation filters (e.g., loaded into a cache) can be used to analyze the current filter output. Different interpolation filters can be used for different filter outputs during different measurement intervals, for example, because different objects will reflect corresponding pulses. In other embodiments, interpolation filters 2324 and DSP 2322 can be part of the light sensing module (e.g., on the same chip as sensor array 2336). Thus, light sensing module 2330 can optionally incorporate additional DSP and interpolation filters 2324. In another embodiment, light sensing module 2330 can include complete ranging system controller functionality.

[0316] G. Method using curve filter

[0317] Figure 24 2400 is a flow chart illustrating a method 2400 for performing ranging using a curve filter of an optical ranging system according to an embodiment of the present invention. The optical ranging system can be part of a LIDAR system that also detects objects. Method 2400, as described with respect to a single pulse, can be equally applied to pulse trains and multiple pulse trains over a measurement time interval. As with method 1400, method 2400 and other methods can be used for purposes other than ranging, such as for communication purposes or detecting interfering signals.

[0318] At block 2410, a pulse is emitted from a light source (e.g., a laser or light emitting diode) of the optical ranging system. The pulse can be reflected from an object so that the pulse can be detected at the optical ranging system. For example, the light source can be light sensing module 2340, emitter array 2342, or any specific emitter in emitter array 2342. Examples of types of lasers are provided herein.

[0319] At block 2420, the pulsed photons are detected by a light sensor of a pixel of the optical ranging system. As a result of the detection, data values may be generated at multiple points in time. For example, a light detector (e.g., a SPAD) of the pixel light sensor may provide a digital signal indicating the time at which the photon was received. In other embodiments, the light sensor may be an APD or other light sensor that provides an analog signal that can be converted into a non-binary value corresponding to the data value (e.g., on a scale from 0-255).

[0320] At block 2430, a histogram corresponding to data values is determined for a plurality of time intervals. A counter of the histogram for a particular time interval (e.g., 100-101 ns) may correspond to one or more data values at one or more time points within the particular time interval. For example, a data value may be a positive signal indicating that a photon has been received at a particular light detector of the light sensor. This positive signal may be received at different times during the time interval.

[0321] At block 2440, multiple curve filters are applied to the histogram. Each curve filter can correspond to a different rate of photons detected by the light sensor in a continuous time interval. In various embodiments, different curve filters can correspond to a high pile-up curve, a moderate pile-up curve, and a low / no pile-up curve. In some embodiments, the application of the curve filter can be performed directly on the histogram. In other embodiments, an initial coarse filter can be applied to the histogram to provide an initial filter output, and a curve filter can be applied to the initial filter output. In this example, the curve filter is still applied to the histogram.

[0322] Different curve filters can be determined in various ways. For example, test measurements can be performed to identify different curves in the detected signal. Such measurements can be performed on a variety of subjects under various conditions to identify a representative set of curve filters. In other embodiments, simulations can be performed to determine the type of curve that will occur.

[0323] At block 2450, a first curve filter of the plurality of curve filters is identified as being the best match to the histogram. For example, a cross-correlation function (e.g., an overlap function) between the curve filter and the histogram may be used as described herein to determine the maximum value at a particular sliding time step of the best matching filter.

[0324] In some embodiments, each of the multiple curve filters can slide on the histogram. For example, an embodiment can perform a sliding curve filter on the histogram to calculate a filtered histogram with a counter, and the counter corresponds to the different sliding positions (e.g., different time steps) of the curve filter relative to the histogram. Each of the counters of the filtered histogram can correspond to the overlap of the curve filter and the histogram at a specific sliding position. The maximum value of the filtered histogram can be identified. In this way, multiple maxima can be obtained for multiple filtered histograms. Subsequently, a global maximum value can be determined from the multiple maxima. This global maximum value corresponds to the reception time of the first curve filter and the pulse (e.g., the sliding position of the curve filter compared to the global maximum value).

[0325] At block 2460, the reception time of the pulse is determined using the filtered output of the first curve filter. In various embodiments, the reception time may correspond to the leading edge of the pulse, the middle of the pulse, or the trailing edge of the pulse. The reception time may be measured relative to the start time of the detection interval, for example, from the time the pulse was transmitted.

[0326] At block 2470, the distance to the object is determined using the reception time. The distance can be determined based on the elapsed time from the emission of the pulse to the detection of the pulse. This elapsed time serves as an example of distance. In other embodiments, this elapsed time can be converted into actual distance using the speed of light.

[0327] VI. Interleaved Pulses

[0328] Interpolation filters can be used to provide increased accuracy, as described above. For example, different interpolation filters can then correspond to different time positions of the rising edge (e.g., varying the time position by 100 ps within a 1 ns time interval). However, this can be problematic for strong pulses that cause high pile-up, which may result in only one time interval having any significant value. In this example, interpolation will not help because the relative heights of two or more time intervals cannot be used to interpolate a more precise time for the rising edge.

[0329] As a solution, pulse trains can be staggered so that the detected pulses arrive at different times, for example, different pulse trains with the same pulse pattern have delays relative to each other. If the amount of staggering extends over a time interval (e.g., 1 ns staggering for a 1-ns time interval), then at least two time intervals will have distinct values, except when the rising edge is exactly at the beginning of the time interval. Again, instead of having a distinct value for one time interval, interpolation can be performed. However, such an interpolation curve would then need to take into account the signal shape resulting from the staggering.

[0330] A. Impossibility of interpolation for extremely high stacking curves

[0331] Figure 25A A single rectangular pulse 2510 is shown, which is commonly used in LIDAR systems to illuminate a scene. Depending on the surface (e.g., orientation and smoothness), the reflected pulse can have different shapes. Specifically, when reflected from a flat and smooth surface that is relatively perpendicular to the optical axis of the outgoing pulse, the shape of the reflected signal will closely resemble the outgoing laser pulse train.

[0332] Figure 25B A reflected pulse 2520 is shown with some noise. The reflected pulse 2520 will have a lower intensity than the emitted pulse 2510, but the intensity level can vary. The intensity level (amplitude) of the pulse can be larger for flat and smooth surfaces, especially when the pulse is perpendicular to the surface. In these cases, a high pile-up signal can occur. Strong reflections from this surface can cause pile-up in the detector (e.g., SPAD), as explained above. As a result, the signal at the SPAD may have a high pile-up curve with a sharp rise and a fast fall.

[0333] Figure 25C Figure 2 shows a high pile-up signal 2530 detected at the rising edge of a reflected pulse 2535, according to an embodiment of the present invention. High pile-up signal 2530 does not resemble a true reflected signal, which is more similar to a rectangular pulse. Determining the true signal location becomes increasingly difficult when pile-up is as severe as high pile-up signal 2530, and systems employing detectors with dead time, such as SPADs, are particularly susceptible to pile-up. For example, the resulting histogram in this case might include only a single filled time bin, with all other bins empty, which is unlike the incoming photon signal from reflected pulse 2535.

[0334] Figure 26The resulting histogram 2640 shows a SPAD signal 2630 according to an embodiment of the present invention. Similar to the high pile-up signal 2530, the SPAD signal 2630 rises quickly and falls almost as quickly. The histogram 2640 shows the counters for four time intervals up to the time when the corresponding reflected pulse is received. Time interval 2644 has a very small value, which may be due to background noise alone. The negative values of the time intervals may be due to the application of negative weights to the data values detected in one or more detection intervals, such as described herein for decoded pulses.

[0335] The main characteristic of the histogram 2640 is the time bin 2642 as the only significant value. In this case, the histogram 2640 cannot be interpolated because there is no information about where in the time bin 2642 the rising edge 2633 of the SPAD signal 2630 occurs.

[0336] Even when using multiple pulse trains, two corresponding pulses sent during different detection intervals but at approximately the same time relative to the start signal may provide similar histograms. Therefore, the cumulative histograms of the two pulses may be identical. A delay between the two pulse trains that is less than the time interval width can be introduced, but this delay may still cause similar problems.

[0337] Figure 27A and 27B Two examples of pulses from different pulse trains delayed relative to each other are shown, resulting in a histogram with only one bin having significant values. The horizontal axis corresponds to the time bin; the start and end of the bin are demarcated by hash marks. Counters are shown for five different bins. As shown, the counter bars only occupy a portion of the bin. Therefore, the bars do not correspond to the full width of the bin.

[0338] exist Figure 27A In FIG, solid signal 2702 shows the detected signal for the pulses of the first pulse train. Dashed signal 2704 shows the detected signal for the pulses of the second pulse train. In some embodiments, these detected signals may correspond to signals obtained from a series of SPADs. The specific value at any moment may correspond to the total number of SPADs triggered at that moment. Both signals 2702 and 2704 arrive in time interval 3 and drop to approximately zero before the start of time interval 4 because essentially all SPADs enter their dead time after being activated by the strong signal. Even though dashed signal 2704 occurs after solid signal 2702, only time interval 3 has any significant value accumulated over the two detection intervals of the two pulse trains.

[0339] exist Figure 27BIn the histogram, the pulses of the solid and dashed lines are swapped. Solid signal 2754 occurs after dashed signal 2752. Again, both signals contribute significantly only to time bin 3. Therefore, in both histograms, time bin 3 is the only filled bin, except for some noise. Interpolation will not help to improve resolution in these cases. Figure 27A and 27B It also illustrates that even if two signals are received at different times, the resolution of the detection circuit cannot distinguish between the two signals.

[0340] B. Interleaved pulses for interpolation of high pile-up signals

[0341] As an example, the resolution of the imaging detector electronics is 1 ns. Embodiments can use interpolation to achieve higher accuracy. However, high pile-up signals can cause problems, as described above. However, laser pulse modulation can achieve finer resolution. For example, two distinct consecutive laser pulses can be generated within 0.1 ns. As shown in Figures 27 and 28 , some offset between the two pulses will not necessarily result in finer resolution in all instances.

[0342] In some embodiments, to achieve higher resolution, several identical laser pulses can be staggered in time by a fraction of the resolution of the imaging detector. This staggering can result in the rising edges of different pulses spanning at least two time intervals. Again, interpolation can be performed compared to having a distinct value in one time interval.

[0343] In some embodiments, several laser pulses are staggered within 0.1 ns of each other. When reflections from these different trains of laser pulses reach the SPAD, the SPAD may experience high pile-up after the leading edge of each pulse. However, because the pulses are staggered in time, the leading edge of the pulse group will fall into different histogram bins. The staggered successive laser pulses effectively act as introduced noise, which is known and can be used to increase the temporal resolution of the imager.

[0344] This effectively reduces the effect of quantization noise in the TDC which limits the histogram resolution.By averaging multiple measurements with known offsets, a timing resolution finer than the TDC bin width can be achieved according to embodiments of the present invention.

[0345] Figure 28A and 28B An example is shown of staggering the emitted pulses of different pulse trains so that the detected high pile-up pulses span multiple time intervals according to an embodiment of the present invention. Figure 28A and 28BDetected pulses and corresponding histograms in different detection intervals are shown. Specifically, each figure shows the signal in four detection intervals spanning two time intervals. The histogram shows two time intervals with significant values.

[0346] exist Figure 28A In the histogram 2830, column 2810 corresponds to time interval 3. Column 2820 corresponds to time interval 4. Each row corresponds to a different detection interval 2801-2804. Thus, each of the pulses is emitted and detected before the next pulse of the next detection interval is emitted. Each of the pulses is offset from the pulse of the previous detection interval. In this example, the offset is approximately 1 / 2 of the time interval. 1 / 4, but other offsets can be performed. In addition, the offset does not have to be uniform, e.g. 1 / 2 offset can be 1 / 4 offset is performed before, as may occur in the case where the pulse train in detection interval 2802 occurs before the pulse train in detection interval 2801.

[0347] The positions of the signals in columns 2810 and 2820 are mirrored in histogram 2830. Time bin 3 has a higher value because more pulses are detected in column 2810, which provides a zoomed-in view of time bin 3. However, time bin 4 still has a significant value because one pulse is within column 2820. When histogram 2830 is interpolated, the approximate 3:1 ratio between the values in time bins 3 and 4 would indicate that the first pulse in detection interval 2801 (zero offset in this example) occurred approximately 100 seconds into time bin 3. 1 / 4ns, i.e. when the time interval has a width of 1ns. If the rising edge of the first pulse (or more generally the pulse with zero offset) occurs at 1 / 2ns, then time bins 2 and 4 will be approximately equal, since both will have two pulses. If time bin 3 is the only time bin with a significant value, then the reception time of the rising edge will be at the beginning of time bin 3.

[0348] exist Figure 28B Column 2860 corresponds to time bin 3 in histogram 2880. Column 2870 corresponds to time bin 4 in histogram 2880. Each row corresponds to a different detection interval 2851-2854. Since three pulses are in time bin 4, histogram 2880 has higher values in time bin 4 than in time bin 3. The approximate ratio of the counters for time bins 3 and 4 in histogram 2880 is 1:3 (accounting for some noise), indicating that the pulses in the detection interval occur approximately 1 / 3 of the time bin 3. 3Thus, staggering the laser pulses at a fraction of the imaging detector resolution can be used to obtain a higher resolution of the signal. This higher resolution can, in turn, be used to better identify the distance (distance) from the surface where the laser pulses reflect and strike the SPAD.

[0349] In other embodiments, the number of different offsets at different detection intervals can be 10, where the different offsets differ by 0.1 times the width of the time interval. In this example, the laser pulses are offset by 100 ps for a 1 ns time interval. The laser pulses can be several time intervals wide (e.g., Figure 28A and 28B 5 time intervals wide), but the rising edges are staggered relative to each other.

[0350] C. Interpolation for interleaved pulse trains

[0351] As discussed above, the rising edge of a pulse may be calculated differently when the pulse train is interleaved. To illustrate this, the interpolation for interleaving is contrasted with the interpolation for no interleaving when the two time intervals have distinct values.

[0352] When no interleaving is performed and assuming the FWHM of the detected pulse is 0.2 ns, the rising edge of the detected pulse can be determined to be at 0.9 ns entering the first time interval when the two time intervals have equal values. The fact that the two time intervals have approximately equal values can be determined when the interpolation filter with two taps with equal values is the best match (i.e., relative to other interpolation filters with taps with unequal values). Therefore, interpolation can use knowledge of the FWHM and the time interval values to determine the rising edge of the first pulse. For example, a pulse with a FWHM of 0.6 ns detected entering the first interval 0.7 ns will have equal values in the first and second intervals. There is no need to determine the ratio of the time interval values because the best matched filter will automatically provide the correct time, for example, given knowledge of the FWHM. Each matched filter can have a predetermined time associated with it, such as described below.

[0353] When interleaving is performed using 10 different offsets and assuming the FWHM of the detected pulses is 0.2 ns, the rising edge of the detected pulse can be determined to be 0.45 ns into the first time interval when the two time intervals have equal values. Since there are 10 pulses with a width of 0.2 ns, the total integration time is 2 ns. With an offset of 0.1 ns and starting at 0.45 ns, a total of 5 complete pulses can be attributed to each time interval, resulting in equal values for the two time intervals in the histogram. The times corresponding to a particular interpolation filter can be stored in memory and retrieved when the best matching interpolation filter is identified.

[0354] As a simpler example for interleaving, if the pulses can be considered to be 0.1 ns wide, then a best-matched interpolation filter of (8, 2) will indicate that the rising edge begins at 0.2 ns (or at least within 100 ps of 0.2 ns) in the first time interval. If (10, 0) is the best-matched interpolation filter, then the rising edge of the first pulse occurs in the first 100 ps. If (5, 5) is the best-matched interpolation filter, then the rising edge begins at 500 ps of the first time interval. Therefore, the exact method of interleaving can depend on the number of interleaved pulse patterns (e.g., the number of excitations) and the increment (jitter) in the interleaving.

[0355] D. Methods for Interleaving Pulses

[0356] Figure 29 2 is a flow chart illustrating a method 2900 for performing ranging using staggered pulses in an optical ranging system according to an embodiment of the present invention. The method 2900 can stagger the pulses of a pulse train to provide an accuracy greater than the resolution of the timing circuit of the optical sensing module, even when the received signal is so strong that a high pile-up signal is generated from the optical sensor.

[0357] At block 2910, N pulse trains are transmitted from a light source (e.g., a laser) as part of a range measurement. The pulse trains may have a coded pattern as described herein. Aspects of block 2910 may be similar to Figure 14 Box 1420 and / or Figure 24 The N pulse trains may be reflected from the object, wherein detection of the reflected portion may be used to determine the distance to the object.

[0358] Each of the N pulse trains can include one or more pulses from a light source, and each pulse train can correspond to a different time interval triggered by a start signal. For example, a VCSEL can emit a first pulse train of two pulses, where the emission can be triggered by a start signal, such as start signal 1101. The first pulse train can be part of a first detection time interval, in which the reflected portion of the first pulse train can be detected. Subsequently, as part of a second detection time interval (still part of the same measurement), the same VCSEL can emit a second pulse train, triggered by a start signal (e.g., start signal 1102). The start signal can be a periodic signal.

[0359] At block 2920, the light sensor of the pixel of the optical ranging system detects the photons of N pulse trains, thereby generating data values at multiple time points. Figure 24 Box 2420 and / or Figure 14 Box 2920 is performed in a similar manner to boxes 1430 and 1440.

[0360] At block 2930, a histogram corresponding to the data values is determined in a plurality of time intervals. Aspects of block 2930 may be similar to Figure 24 Box 2430 and / or Figure 14 The histogram counters for a particular time interval may correspond to one or more data values at one or more time points within the particular time interval. For illustration, the time points may correspond to three different times for three different detected signals in column 2810 corresponding to detection intervals 2801-2803. For other methods, pulses from different pulse trains may arrive at the same time point within the time interval, or at different time points due to noise. However, in some embodiments, the different time points may be the result of staggering the emitted pulse trains, for example, as described above.

[0361] As part of providing increased accuracy, the N pulse trains can have varying offsets from one another. For example, the N pulse trains can be offset by different amounts from the start of the corresponding detection time interval. Alternatively, the clock running the detector histogram can be offset by different amounts from the start of the corresponding detection time interval. Thus, the same effect is achieved regardless of whether the transmitter or receiver is time-interleaved, and pulse trains with such offsets can be implemented in either manner.

[0362] At least two of the N pulse trains may be offset by less than the width of a time bin, e.g., when the histogram has bins that are 1 ns wide, the two pulse trains may be offset by less than 1 ns. The two pulse trains may have the same pattern, e.g., such that the rising edges of the corresponding pulses are offset by the same amount as the offset of the pulse train. The assigned weights (e.g., for coding the pulse trains as described herein) may be of the same sign (i.e., positive or negative) and in the same direction (e.g., when a higher dimensional coding scheme is used for more complex quadrature).

[0363] Some pulse trains may not be offset from one or more other pulse trains. Thus, the measurement may include pulses other than the N pulse trains. In other embodiments, the N pulse trains may be all the pulse trains for the ranging measurement. For example, there may be 10 staggered offsets (e.g., differing by 100 ps), but there may be a total of 20 pulse trains used in the measurement, with two pulse trains being sent at each of the offsets (e.g., the same shift from the start signal).

[0364] In some embodiments, consecutive pulse trains of N pulse trains can be offset by the same time offset, T. For example, the first pulse train can be offset by zero relative to the start signal, the second pulse train can be offset by T relative to the start signal, and the third pulse train can be offset by 2T from the start signal, and so on, thereby having consecutive pulse trains offset by T. The total time span from the first pulse train to the last pulse train can be equal to the width of the time interval, for example, N*T can be equal to the width. In various embodiments, the time offset, T, is between 0.5 and 0.01 of the width of the time interval, for example, 0.1 of the width of the time interval. This example is 10 offsets multiplied by 100 picoseconds to achieve a span of 1 ns, thereby having two time intervals with distinct value histograms, except when the initial rising edge is at the very beginning of the first time interval.

[0365] At block 2940, a reception time corresponding to the N pulse trains relative to the start signal is determined. The reception time can be determined using a matched filter, such as an interpolation filter. Thus, determining the reception time can include applying a matched filter to the histogram to obtain a filtered histogram, and determining the reception time using the maximum value of the filtered histogram and a time offset T. The correspondence between the matched filter and a specific time can be determined based on an interleaving pattern in the N pulse trains. The reception time can correspond to the rising edge of the pulse with the smallest shift relative to the start signal.

[0366] At block 2950, the time of reception may be used to determine the distance to the object. For example, block 2950 may be associated with Figure 24 Block 2470 may be performed in a similar manner or by any of the techniques described herein.

[0367] VII. Sensors with different gains

[0368] SPAD saturation and buildup / quenching can affect the performance of LIDAR systems in several ways. These include managing the dynamic range of the SPAD and managing the SPAD power. These issues are particularly important in high-signal conditions resulting from high levels of reflection from the laser pulse or high levels of background radiation (especially from the sun).

[0369] In embodiments using multiple photodetectors (e.g., SPADs) grouped to appear as a single pixel, these problems may be solved by classifying different SPADs based on their dynamic range and thereby providing different signal levels. Thus, different SPADs may need to trigger (start) more or fewer photons and produce a positive signal for inclusion in a histogram. Once classified (e.g., as set by a circuit), embodiments can modify the operating state of a SPAD with a certain dynamic range under certain conditions. For example, SPAD sensitivity can be reduced at high light flux. As another example, the power level of a SPAD with a strong signal level can be reduced, such as disconnected. As another example, only signals from certain SPADs can be used to build a histogram, thereby effectively disconnecting those SPADs. Similar operations can be performed for SPADs with weak signal levels (i.e., low sensitivity), such as increasing the dynamic range or reducing the power at low light flux.

[0370] A. Arrangement of detectors with varying signal levels

[0371] Figure 30A A conventional arrangement 3000 of 16 photodetectors 3002 (e.g., SPADs) forming a single pixel light sensor is shown in accordance with an embodiment of the present invention. In some circumstances, such as when the light flux reaches certain levels, e.g., too low or too high, it may be beneficial to change the operating state of the photodetectors 3002. Various arrangements may be used in various circumstances, such as using different attenuation levels (i.e., different dynamic ranges), different power levels (e.g., disconnected), and / or different weightings for the detected signals (e.g., set to zero to specify which photodetector signals do not contribute to the histogram). Furthermore, different settings may be used for different subsets of the photodetectors. Thus, a particular arrangement of operating states for the sensor's photodetectors may be set, for example, in a very short time, in one or a few cycles, using data values detected from the photodetectors themselves.

[0372] In some embodiments, to enhance the dynamic range of the LIDAR system, the photodetectors of a pixel can be configured to have (or identified as naturally having) different attenuation levels. The classification of the photodetectors can be dynamic, for example, when the operating state of one or more photodetectors is changed by changing the attenuation level. This change can occur based on detected background light signals (e.g., due to bright background light sources such as the sun or street lights) or reflected pulses from highly reflective objects.

[0373] Figure 30BAn arrangement 3010 of 16 photodetectors with varying attenuation levels is shown, according to an embodiment of the present invention. Photodetector 3012 has a high signal level in response to detecting a photon. Photodetector 3014 is attenuated and, therefore, provides a weak signal level in response to detecting a photon. Consequently, when low light flux is present, photodetector 3014 may be untriggered or only slightly triggered. The high and low classifications are relative to each other and therefore do not necessarily correlate to any absolute range.

[0374] In some embodiments, all 16 photodetectors can be enabled. In this way, photodetector 3012 can detect low-level light pulses when the background light is not too high (e.g., below a background threshold). However, when the background light is too high or the reflected pulse is too strong, photodetector 3012 may always be activated, thus not providing usage data. In such cases, their signals can be ignored (e.g., by not including their signals in the histogram). While photodetector 3012 cannot be used in this high-flux scenario, photodetector 3014 can detect low-level light pulses and therefore will not necessarily always activate under high background or strong reflected pulses. Therefore, by using photodetectors with different dynamic ranges, the dynamic range of the entire photosensor can effectively be greater than that of any one photodetector. Dynamically changing the active operation (e.g., which signals from which photodetectors are used in the histogram) can achieve clean signals in both high and low light fluxes.

[0375] Figure 30C An arrangement 3020 of 16 photodetectors with different attenuation levels and different effective operation is shown, according to an embodiment of the present invention. In various embodiments, different levels of effective operation can be specified by different power levels (e.g., turning on or off) or different contribution levels to the histogram (e.g., setting weights to 0 for certain photodetectors). For example, photodetector 3022 can be deactivated by power or contribution level. This can be done when photodetector 3022 has a high dynamic range (high signal levels in response to detected photons) and the system expects to receive a high light flux (e.g., based on one or more previous measurements).

[0376] The light detector 3014 is attenuated and therefore provides a weak signal level in response to detecting a photon. When there is a low light flux, the light detector 3014 may not be triggered or may be triggered less frequently. The high and low classifications are relative to each other and therefore are not necessarily associated with any absolute range.

[0377] In some embodiments where a strong signal is present, the decay mode of arrangement 3020 may be employed. This may occur due to extremely high ambient radiation or an extremely high reflected signal. One situation where this may occur is when the imager is looking directly at the sun. Signals from the photodetectors themselves may be used as an indication to set this configuration. For example, this configuration may be enabled when there was high accumulation across all photodetectors during a previous measurement.

[0378] Thus, if there are some detectors that are 100 times less sensitive than others, the system can essentially ignore the sensitive detectors in a high light flux environment (e.g., a stop sign or a bright light source) because the sensitive detectors (e.g., photodetector 3022) will saturate and pile up, where the insensitive detectors (e.g., photodetector 3024) may not pile up. These embodiments can also be valuable even for curved filters because a higher dynamic range can be achieved for more accurate estimation of signal strength.

[0379] B. Arrangement for different directions

[0380] In some embodiments, the LIDAR system identifies different directions and associates these different directions with specific radiation ranges (light fluxes). Thus, the LIDAR system can dynamically adjust the operating state of the light detectors, where different light detectors may have different settings for operating state, as specified in different arrangements.

[0381] An example of an application of these embodiments is a vehicle moving on a road during dusk or twilight when there is strong radiation from the sun in a particular direction. The LIDAR system can dynamically adjust the operating state of each light detector (e.g., on / off configuration, attenuation level / gain control, or contribution level to the histogram).

[0382] Figure 31 Diagram showing different detector arrangements for a pixel sensor under different lighting conditions at different angles according to an embodiment of the invention. Figure 31 The LIDAR system 3111 is shown on a vehicle 3110 in the center surrounded by a detector arrangement 3120-3170. Different operating states in the arrangement are shown by different labels, such as Figures 30A-30C Detector 3112 is enabled and at a high signal level. Detector 3114 is enabled and at a weak signal level (ie, low sensitivity). Detector 3122 is disabled.

[0383] Each of the detector arrangements 3120-3170 is at a different angle relative to the LIDAR system 3111. For ease of illustration, only six different angles are shown, but more angles may exist. For example, each angle corresponds to 32 arc minutes or approximately 0.5° of a total 360° horizontal (or 4π steradians). In some embodiments, each angle may correspond to an amount of rotation that occurs during one or more measurement intervals. An initial measurement may be performed (e.g., during one or more initial detection intervals of measurement) to determine the light flux, and then the operating state settings for each of the detectors may be specified to achieve a particular arrangement.

[0384] When looking directly at a strong light source 3105 (e.g., the sun) as in the detector arrangement 3120, the LIDAR system, for example, deactivates all detectors after an initial detection interval, with many detectors being continuously activated. Some detectors (e.g., certain SPADs) may use some power each time they are activated, and such detectors will be constantly activated for high background light. The threshold criteria for deactivating all detectors can be specified by a threshold number / percentage of detectors of pixels triggered within the same time interval (e.g., 60%, 70%, 80%, 90%, or 100%). The criteria may require such a large number of triggers in multiple time intervals (e.g., 2, 3, 5, 10, 20, etc.) during the same detection interval. Specifically, when a large number of triggers are due to a strong background light source, the number of time intervals with a large number of triggers will be higher. When due to a highly reflective object (e.g., when the detected pattern matches the emitted pulse pattern), the number of time intervals with a large number of triggers will be lower, and therefore a lower number of time intervals may need to be deactivated.

[0385] In other embodiments, the signal level on detector 3122 can be changed to cause the background light flux to substantially drop below a threshold value, so that the time interval in which the reflected laser pulse is detected has a significantly higher count. In some embodiments, the degree to which the dynamic range needs to be changed (e.g., the bias voltage of the SPAD) can be determined over several detection intervals without eliminating all sensitivity. For example, the sensitivity in the signal level can be incrementally reduced (or in a binary or other search tree manner) to identify a tradeoff between removing noise and maintaining signal. This search can be implemented with different settings for different SPADs of the pixel sensor (each subset of SPADs with different settings), thereby allowing simultaneous searches to be performed in parallel.

[0386] Detector arrangements 3170 and 3130 are offset from the center of the light source 3105 from an angle that is directly pointed at the light source. For example, these angles can differ from the detector arrangement 3120 by 0.1°, 0.2°, 0.3°, 0.4°, or 0.5°. Due to the low total light flux, the weak detector 3114 can be enabled. In some embodiments, the setting at a specific angle can be maintained for multiple rotations and then checked again (e.g., during an initial detection interval measured later). Thus, when the LIDAR system is a rotating LIDAR, once the setting at a specific angle is determined (e.g., as determined by an encoder), the setting can be reused when the optical ranging device returns to the same angle. The encoder can identify the angular position of the optical ranging system, which can be used to identify when the optical ranging system determines the intensity level at the initial angular position, which can be marked for a specific setting of the optical detectors.

[0387] Detector arrangements 3160 and 3140 are at angles significantly different from those pointing directly at light source 3105, and thus all detectors may be enabled, but some may still be at a weak setting, e.g., to provide a larger dynamic range, since the amount of background light (e.g., reflected from other objects) may still affect detectors set at a strong signal level (i.e., high sensitivity). Detector arrangement 3150 may be pointing in the opposite direction of light source 3105, and thus all detectors may be enabled and set to have a strong signal level.

[0388] In some embodiments, when the LIDAR system detects an object entering the field of view while at an angle pointing toward the light source 3105, it can change the pattern of SPAD gain (attenuation / signal level) to optimize for detection of the object. For example, when another vehicle moves in a direction that blocks direct sunlight, the system can increase the SPAD gain along the evolving viewing angle of the object.

[0389] In some embodiments, power savings can be enhanced by adjusting the intensity of the laser source in the LIDAR system. For example, the system can choose not to illuminate in directions where extremely high ambient radiation is detected. Alternatively, the laser source intensity can be reduced when low ambient radiation is detected. Furthermore, in conditions of high background flux, the laser source intensity can be increased in combination with a lower attenuation level of the detector so that reflected pulses can still be detected without the background light causing a noticeable signal in the cumulative histogram.

[0390] C. Other arrangements

[0391] Figure 32An arrangement 3200 of peripheral photodetectors 3222 is shown having different operating states than central region photodetectors 3212, in accordance with an embodiment of the present invention. Photodetectors 3222 can have settings that differ in various ways from photodetectors 3212. For example, photodetectors 3222 can have reduced power levels, be ignored (e.g., when accumulating data values for a histogram), or have weak signal levels.

[0392] For configurations where the light detector 3222 reduces power level, changing the power level of the detector can save power consumption. Figure 32 The configuration shown in FIG can be used to save 75% of the power applied to the SPAD in one pixel. This arrangement can allow the central photodetector 3212 to continue checking whether the light flux is still too high, and then once the light flux decreases, the photodetector 3222 can be enabled. The arrangement with such disabled photodetectors on the periphery and enabled photodetectors in the central area means that the central area will be more illuminated, and therefore if the light flux is low enough in the central area, it will be low enough for the periphery.

[0393] For a setting where the light detectors 3222 have reduced signal levels, the arrangement 3200 will cause some detectors to have high signal levels and some to have low signal levels, as is also the case with the other arrangements described above. This mixed arrangement can increase the dynamic range (e.g., by an order of magnitude). When there is a high light flux, the sensitive detectors will detect too much light and saturate. However, the attenuated detectors will receive enough light to produce a detectable signal. In this case, the system can ignore the high signal detectors and only use the attenuated detectors. When the high light flux is caused by a highly reflective object, this can allow the detection of rectangular pulses instead of high pile-up pulses. Alternatively, when the strong detector is triggered in a manner consistent with the pulse pattern but the weak detector is not, the weak detector can be disconnected or otherwise reduced in power level.

[0394] To determine which settings and arrangements to use, data values detected during an initial detection interval can be used. This detection interval may involve emitting a laser pulse or not (e.g., detecting only the background). When using an emitted pulse train, information about the object can be obtained, such as how reflective it is and the relative orientation of the surface. Later detection intervals can cause certain detectors (e.g., those that are not attenuated) to be turned off or otherwise reduced in power level when threshold criteria are met, as mentioned above.

[0395] D. System components

[0396] Figure 33 A configurable optical system 3310 is shown in accordance with an embodiment of the present invention. Figure 33 The components can be Figure 13 and 23 33. The system controller 3320 may operate in a similar manner to the components of the system controller 3320 and may include such components. The system controller 3320 may function in a similar manner to other ranging system controllers, for example by controlling the light sensing module 3330 and the light transmission module 3340. The light transmission module 3340 may include an intensity modulator 3344 and an array of emitters 3342 capable of transmitting light (e.g., one or more laser diodes). The intensity modulator 3344 may cause a change in the light flux of the emitted pulses, for example, due to the amount of light detected, as described above. The intensity modulator 3344 may also be included in the ranging system controller 3320.

[0397] Light sensing module 3330 includes one or more detector arrangements 3332, a histogram 3334, a sensor array 3336, and a detector controller 3338. Detector controller 3338 can analyze data values from a detection interval (e.g., by looking at the median value of histogram 3334) and determine which settings should be used for the operating state of the detectors at various pixels of sensor array 3336. In some embodiments, detector arrangement 3332 can store certain settings that can be selected when the detected data values have certain properties (e.g., indicating a high flux from background light or a reflective object). Detector arrangement 3332 can be stored for various or all angles when configurable optical system 3310 is rotated.

[0398] E. Methods for configuring detectors

[0399] Figure 34 is a flow chart illustrating a method 3400 of performing ranging using a configurable optical ranging system, according to an embodiment of the present invention. Aspects of the method 3400 can be implemented via techniques described for other methods described herein.

[0400] At block 3410, photons are detected by a light sensor of a pixel of the optical ranging system, thereby generating initial data values at multiple time intervals. The light sensor may include multiple light detectors (e.g., SPADs). These initial data values may be analyzed as described above to determine a set arrangement of operating states for the multiple light detectors. In some embodiments, the initial data values may be obtained by transmitting an initial pulse from a light source (e.g., a laser), wherein the initial data values may be used to determine a reception time after reflection from an object.

[0401] At block 3420, an intensity level of the detected photons is determined based on the initial data value. In various embodiments, the intensity level may correspond to the number of photodetectors activated simultaneously, the number of photodetectors activated continuously over a specified time period, the average number of photodetectors activated over a specified time period, and the like. The intensity level may be analyzed by a controller (e.g., detector controller 3338) on the same chip as the photodetectors or on a different chip. The controller may determine a new or saved arrangement of settings for the photodetectors.

[0402] At block 3430, a first pulse is emitted from a light source of the optical ranging system. The first pulse may be reflected from an object so that a distance to the object may be determined.

[0403] At block 3440, before detecting the photons of the first pulse, the operating state of a set of photodetectors is changed based on the determined intensity level. As described herein, according to various embodiments, the operating state can be a power level, a contribution level (e.g., whether the detected signal is used for ranging measurement), and an attenuation level. In various embodiments, such a change can be due to the determined intensity level being above or below a threshold. The set of photodetectors can be all or part of the plurality of photodetectors.

[0404] The changed operating state can be maintained for a specified amount of time (e.g., until the next rotation when the optical ranging system returns to the same angle). For example, the encoder can specify a specific angular position, and a specific arrangement of operating settings can be specified for that angular position. A first transmit pulse can be sent when the system returns to that angular position. In some embodiments, the specified amount of time is defined relative to a number of pulse trains.

[0405] At block 3450, photons of the first pulse are detected by the photodetectors in active operation according to the changed operating state, thereby generating first data values for the plurality of time intervals. Some of the changed photodetectors may not be powered and / or may not be counting, and therefore not in active operation. Not counting data values from the photodetectors may be caused by determining which signals from which photodetectors were used to generate the first data values for the plurality of time intervals (e.g., not counting strong photodetectors when there is a high light flux, as determined by a signal level above a threshold).

[0406] In some embodiments, when the determined intensity level is above a high threshold, the attenuation level of the set of light detectors is increased. When the determined intensity level is below a low threshold, the attenuation level of the set of light detectors can be decreased. In instances where only the attenuation level is changed, all light detectors can be in active operation.

[0407] At block 3460, a reception time corresponding to a first pulse is determined using the first data values at the plurality of time intervals. For example, a histogram may be generated and a matched filter may be used, as described herein.

[0408] At block 3470, the distance to the object is determined using the reception time. The distance may be determined in various ways, such as described herein.

[0409] In some embodiments, the light sensor may include a first group of light detectors categorized as having weak signal levels and a second group of light detectors categorized as having strong signal levels. Either group may be changed in various circumstances. For example, the power level of one or more of the second group of light detectors may be reduced (e.g., turned off) when the determined intensity level is above a high threshold. As another example, the power level of one or more of the first group of light detectors may be reduced (e.g., turned off) when the determined intensity level is below a low threshold. The categorization may be dynamic (e.g., as set by the system for a given detection interval) or longer-lasting (e.g., set to a number of minutes, hours, days, etc.), or even permanent.

[0410] There may be more than two categories of photodetectors for a pixel at a time. For example, in addition to changing the operating state of one of the first or second groups of photodetectors, the operating state of the third group of photodetectors may be based on a determined intensity level (e.g., an intermediate intensity level indicating an intermediate sensitivity).

[0411] VIII. Example Chipset of Sensor Chips with SPADs

[0412] The data values of the signals detected by the sensor array can be tracked up to small time intervals, such as every 1ns or 500ps. To achieve this speed, avalanche photodiodes (APDs) can be used. Since APDs are analog devices that output intensity directly via an analog current or voltage, APDs can use standard analog-to-digital converters (ADCs), which convert an analog voltage stream that follows the number of photons received at the APD. However, APDs cannot provide a compact design because current technology does not allow for economical placement of multiple APDs on the same chip. In contrast, SPADs can be placed on the same chip with high yield and low cost.

[0413] Embodiments overcome the difficulties in using SPADs by producing a custom chip that includes timing circuits, histogram circuits, and other signal processing circuits as well as the SPAD, thereby allowing rapid processing of signals generated from the SPAD. The timing circuit and histogram circuit can enable capture of the binary signal generated by the SPAD. The timing circuit and histogram circuit can be viewed as local circuits that are part of a larger circuit that includes an integrated circuit, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a CPU. For a CPU, the integrated circuit may include a memory that stores program code. When an ASIC or FPGA is used, the timing circuit and histogram circuit are dedicated circuits during operation of the optical ranging system. For an FPGA, configuration logic can program the gates so that a specific configuration has a timing circuit and histogram that are dedicated to its corresponding function once the gate is configured.

[0414] Furthermore, for a single SPAD, there is an upper limit to the measurable light flux, for example, as controlled by the dead time of the SPAD, and there are limitations on the timing circuitry used to capture the signal from the SPAD. This is in addition to the inherent binary instantaneous dynamic range of the SPAD. Depending on the resolution of the timing circuitry (e.g., the time-to-digital converter, TDC), the SPAD can register only one photon per time interval. Therefore, the dynamic range is limited by the minimum time interval and is not a dead time issue per se. To address this issue, embodiments may use multiple SPADs per "pixel."

[0415] Figure 35 A compact optical system 3510 of an optical ranging system according to an embodiment of the present invention is shown. The elements of FIG. 35 may be used with Figure 13 、 23 The ranging system controller 3520 may operate in a manner similar to the elements of 33 and may include such components. The ranging system controller 3520 may function in a manner similar to other ranging system controllers, for example by controlling the light sensing module 3530 and the light transmission module 3540. The light transmission module 3540 may include a laser source 3542 (e.g., an emitter array of laser diodes) capable of emitting laser pulses.

[0416] The light sensing module 3530 includes a sensor integrated circuit (e.g., an ASIC or FPGA) that includes a filter 3532, a histogram circuit 3534, a sensor array 3536, and a timing circuit 3538. The sensor array 3536 may include a plurality of light sensors, each of which includes a plurality of light detectors (e.g., SPADs). For example, Figure 5 Such arrays in .

[0417] Timing circuit 3538 may provide a signal to histogram circuit 3534 so that histogram circuit 3534 may determine which counter to increment in response to the signal from sensor array 3536. Histogram circuit 3534 may include a histogram for each pixel sensor of sense array 3536. For example, sensor array 3536 may provide a signal with an identifier of its corresponding pixel, or this correspondence may be hard-wired.

[0418] The sensor signal can indicate how much to increment the histogram counter. For example, the signal from each detector (e.g., SPAD) of the light sensor can indicate whether to increment the histogram counter based on the signal. Thus, the plurality of light detectors of the sensor array 3536 can be configured to output a binary signal when triggered by a photon, indicating that one or more photons have been detected. The timing circuit 3538 can be configured to determine the time when a photon is detected based on the binary signal.

[0419] The histogram circuit 3534 can be configured to determine and store counters each corresponding to the number of light sensors triggered during a time interval. Thus, a histogram can be generated that can be used to determine the reception time of one or more pulses from the laser source reflected from the object.

[0420] The windowing circuit 3537 of the sensing integrated circuit 3531 can be configured to apply one or more matched filters 3532 to the histogram to identify the time window within which the reception time falls. The sensing integrated circuit 3531 may also include an interpolation circuit 3539 configured to apply multiple interpolation filters to the histogram or the filtered output of the histogram within the time window. In some embodiments, the best matched interpolation filter can identify the reception time with an accuracy less than the width of the time bin. In other embodiments, the interpolation circuit can be part of the ranging system controller 3520, which resides on a second integrated circuit communicatively coupled to the sensor integrated circuit.

[0421] In some embodiments, a rotary motor (e.g., Figure 2 The motor 260 is connected to the laser source and sensor integrated circuit for rotating the laser source and sensor integrated circuit. The laser source 3542 can itself be an integrated circuit and can include multiple laser devices (e.g., vertical cavity surface emitting lasers (VCSELs)).

[0422] IX. Additional Embodiments

[0423] Although some embodiments disclosed herein focus on applying light ranging within the context of 3D sensing for automotive use cases, the systems disclosed herein may be used in any application without departing from the scope of this disclosure. For example, the system may have a small or even miniature form factor that enables several additional use cases, such as for solid-state light ranging systems. For example, the system may be used for a 3D camera and / or depth sensor within a device, such as a mobile phone, tablet PC, laptop computer, desktop PC, or other peripheral and / or user interface device. For example, one or more embodiments may be employed within a mobile device to support facial recognition and face tracking capabilities, eye tracking capabilities, and / or for 3D scanning of objects. Other use cases include forward-facing depth cameras for augmented and virtual reality applications in mobile devices.

[0424] Other applications include deployment of one or more systems on aerial vehicles, such as airplanes, helicopters, drones, and the like. Such instances can provide 3D sensing and depth imaging to aid navigation (autonomous or otherwise) and / or generate 3D maps for later analysis, such as to support geophysical, architectural, and / or archaeological analysis.

[0425] The system can also be mounted to fixed objects and structures, such as buildings, walls, utility poles, bridges, scaffolding, and the like. In these cases, the system can be used to monitor outdoor areas, such as manufacturing facilities, assembly lines, industrial facilities, construction sites, excavation sites, roads, railways, bridges, and the like. In addition, the system can be installed indoors and used to monitor the movement of individuals and / or objects within a building, such as the movement of inventory within a warehouse or the movement of people, luggage, or cargo within an office building, airport, train station, and the like. As will be appreciated by those skilled in the art having the benefit of this disclosure, many different applications of optical ranging systems are possible, and therefore, the examples provided herein are provided for illustrative purposes only and should not be construed to limit the use of such systems to only the explicitly disclosed examples.

[0426] X. Computer System

[0427] Any of the computer systems or circuits described herein may utilize any suitable number of subsystems. The subsystems may be connected via a system bus 75. For example, the subsystems may include input / output (I / O) devices, system memory, storage devices, and network adapters (e.g., Ethernet, Wi-Fi, etc.) that may be used to connect to other devices of the computer system (e.g., engine control devices). The system memory and / or storage devices may embody computer-readable media.

[0428] A computer system may include multiple identical components or subsystems connected together, for example, by external interfaces, by internal interfaces, or via removable storage devices that can be connected and removed from one component to another. In some embodiments, the computer system, subsystem, or device can communicate over a network.

[0429] Aspects of the embodiments may be implemented using hardware circuits (e.g., application specific integrated circuits or field programmable gate arrays) and / or in a modular or integrated manner using computer software in the form of control logic with a substantially programmable processor. As used herein, a processor may include a single-core processor, a multi-core processor on the same integrated chip, or multiple processing units on a single circuit board or network, as well as dedicated hardware. Based on the disclosure and teachings provided herein, one of ordinary skill in the art will know and appreciate other ways and / or methods of implementing embodiments of the present invention using hardware and combinations of hardware and software.

[0430] Any of the software components or functions described in this application can be implemented as software code to be executed by a processor using, for example, conventional or object-oriented techniques using any suitable computer language, For example, Java, C, C++, C#, object-oriented C language, Swift, or scripting languages such as Perl or Python. The software code can be stored as a series of instructions or commands on a computer-readable medium for storage and / or transmission. Suitable non-transitory computer-readable media may include random access memory (RAM), read-only memory (ROM), magnetic media such as a hard drive or floppy disk, or optical media such as a compact disc (CD) or DVD (digital versatile disc), flash memory, etc. The computer-readable medium can be any combination of such storage or transmission devices.

[0431] Such program can also be encoded and transmitted using carrier signals, and the carrier signal is suitable for transmitting via wired, optical and / or wireless networks that meet multiple protocols, including the Internet. Therefore, computer-readable media can be produced using data signals encoded with these programs. The computer-readable media encoded with program code can be encapsulated with compatible devices or provided separately with other devices (for example, downloaded via the Internet). Any such computer-readable media can reside on or in a single computer product (for example, a hard drive, a CD or an entire computer system), and can be present on or in different computer products within a system or network. A computer system can include a monitor, a printer, or other suitable displays for providing any result mentioned herein to the user.

[0432] Any of the methods described herein can be performed in whole or in part by means of a computer system comprising one or more processors, which can be configured to perform steps. Therefore, an embodiment can relate to a computer system configured to perform any method step described herein, which potentially performs corresponding steps or corresponding step groups with different components. Although presented as numbered steps, the steps of the method herein can be performed simultaneously or at different times or in different orders. In addition, the part of these steps can be used together with the part of other steps of other methods. In addition, all or part of the step can be optional. In addition, any step in the step of any method can be performed using a module, circuit or other device for performing these steps.

[0433] The specific details of the specific embodiments may be combined in any suitable manner without departing from the spirit and scope of the embodiments of the invention. However, other embodiments of the invention may be directed to specific embodiments related to each individual aspect or specific combinations of these individual aspects.

[0434] The above description of example embodiments of the present invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form described, and many modifications and variations are possible in light of the above teaching.

[0435] References to "a," "an," or "the" are intended to mean "one or more" unless specifically indicated to the contrary. Unless specifically indicated to the contrary, the use of "or" is intended to mean an inclusive or, not an exclusive or. Reference to a "first" component does not necessarily require the presence of a second component. Furthermore, reference to a "first" or "second" component does not limit the referenced component to a particular location unless explicitly stated. The term "based on" means "based at least in part on."

[0436] All patents, patent applications, publications, and descriptions mentioned herein are incorporated by reference in their entirety for all purposes. No admission is made that they are prior art.

Claims

1. A method for performing distance measurement using an optical distance measurement system, the method comprising: detecting photons by a light sensor of a pixel of the optical ranging system, thereby generating initial data values at a plurality of time intervals, wherein the light sensor comprises a plurality of light detectors; determining an intensity level of the detected photons based on the initial data value; transmitting a first pulse from a light source of the optical ranging system, the first pulse being reflected from an object; prior to detecting photons of the first pulse, changing an operational state of a group of photodetectors in the photodetectors based on the determined intensity level; detecting photons of the first pulse by a photodetector in effective operation according to the changed operation state, thereby generating first data values in the plurality of time intervals; determining a reception time corresponding to the first pulse using the first data values at the plurality of time intervals; as well as The distance to the object is determined using the reception time. The method of claim 1 , wherein the set of light detectors is the entirety of the plurality of light detectors. The method of claim 1 , wherein the operating state is a power level or an attenuation level of the set of photodetectors.

4. The method of claim 1, wherein the operating states of the set of photodetectors are changed by specifying which signals of which photodetectors are used to generate the first data values during the plurality of time intervals, thereby specifying which photodetectors are in active operation.

5. The method of claim 1 , wherein the determined intensity level is above a high threshold, and wherein changing the operating state of the set of light detectors in the light detectors based on the determined intensity level comprises: An attenuation level of the set of light detectors is increased.

6. The method of claim 1 , wherein the determined intensity level is below a low threshold, and wherein changing the operating state of the set of light detectors in the light detectors based on the determined intensity level comprises: An attenuation level of the set of light detectors is reduced.

7. The method of claim 1 , wherein the light sensor comprises: are classified as a first group of photodetectors having low sensitivity, and A second group of light detectors is classified as having high sensitivity, wherein the operating state of the first group of light detectors or the second group of light detectors is changed based on the determined intensity level and not otherwise.

8. The method of claim 7, wherein changing the operating state of the first set of light detectors based on the determined intensity level comprises: The power level of one or more photodetectors in the second set of photodetectors is reduced when the determined intensity level is above a high threshold.

9. The method of claim 8, wherein the determined intensity level is a result of a bright background light source.

10. The method of claim 8, wherein reducing the power level of the one or more photodetectors in the second set of photodetectors comprises turning off the one or more photodetectors in the second set of photodetectors.

11. The method of claim 7, wherein changing the operating state of the first set of light detectors based on the determined intensity level comprises: The power level of one or more photodetectors of the first set of photodetectors is reduced when the determined intensity level is below a low threshold. 12 . The method of claim 11 , wherein reducing the power level of the one or more photodetectors in the first set of photodetectors comprises turning off the one or more photodetectors in the first set of photodetectors.

13. The method of claim 7, wherein the light sensor comprises a third group of light detectors classified as having intermediate sensitivity, the method further comprising: Prior to detecting photons of the first pulse, in addition to changing the operating state of one of the first or second groups of photodetectors, the operating state of the third group of photodetectors is changed based on the determined intensity level.

14. The method of claim 1, wherein the detected photons providing the initial data value correspond to background light.

15. The method according to claim 1, further comprising: transmitting an initial pulse from the light source of the optical ranging system, the initial pulse being reflected from the object, wherein the detected photons providing an initial data value correspond to the initial pulse, The receiving time is determined for the first pulse and the initial pulse using the first data values and the initial data values at the plurality of time intervals.

16. The method of claim 15, wherein the determined intensity level is above a high threshold because the object is reflective, the method further comprising: The operating state of the set of light detectors is reduced in response to the determined intensity level being above a high threshold. The method of claim 1 , wherein the changed operating state is maintained for a specified amount of time.

18. The method of claim 17, wherein the specified amount of time is defined relative to the number of pulse trains.

19. The method of claim 1 , wherein the optical ranging system rotates and includes an encoder to identify an angular position of the optical ranging system, the method further comprising: Identifying that the intensity level is determined when the optical ranging system is in an initial angular position, wherein the first pulse is transmitted when the optical ranging system returns to the initial angular position after one or more rotations.

20. The method of claim 1, further comprising: An intensity of one or more emitted pulses from the light source is varied based on the determined intensity level.

21. A computer product comprising a computer readable medium storing a plurality of instructions for controlling a computer system to perform the operation of the method according to any one of claims 1 to 20.

22. An optical ranging system comprising: The computer product according to claim 21; as well as One or more processors for executing instructions stored on the computer-readable medium.

23. An optical distance measurement system comprising means for performing the method according to any one of claims 1 to 20.

24. An optical ranging system comprising one or more processors configured to perform the method according to any one of claims 1-20.

Images