Time series measurements of lidar accuracy

Abstract

An optical measurement system can include a light source and a corresponding light sensor configured to emit and detect photons reflected from objects in the surrounding environment for optical measurements. An initial peak can be identified as arising from reflections off the housing of the optical measurement system. This peak can be removed or used to calibrate the measurement calculations of the system. Peaks arising from reflections off surrounding objects can be processed using on-chip filters to identify potential peaks, and unfiltered data can be passed to an off-chip processor for distance calculations and other measurements. Spatial filtering techniques can be used to combine values from histograms of spatially adjacent pixels in a pixel array. This combination can be used to improve the confidence of distance measurements.

Description

[0001] Cross-Reference to Related Applications

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 913,604, filed October 10, 2019, entitled “PROCESSING TIME-SERIES MEASUREMENTS FOR LIDAR ACCURACY,” which is incorporated by reference herein. BACKGROUND

[0003] Light Detection and Ranging (LIDAR) systems are used for object detection and ranging, for example, for vehicles such as cars, trucks, boats, etc. LIDAR systems are also used for mobile applications (e.g., for facial recognition), home entertainment (e.g., gesture capture for video game input), and augmented reality. A LIDAR system measures distance to an object by illuminating a scene with pulses from a laser and then measuring the time for a photon to travel to the object and back after reflection, as measured by a receiver of the LIDAR system. The detected signal is analyzed to detect the presence of a reflected signal pulse among background light. Distance to the object can be determined based on the time of flight from transmission of a pulse to receipt of a corresponding reflected pulse.

[0004] In particular, at the economic cost of LIDAR systems, it can be difficult to provide robust distance accuracy down to a few centimeters under all conditions. Promising new detector technologies such as single-photon avalanche diodes (SPADs) are attractive, but have significant deficiencies when used to measure time of flight and other signal characteristics due to their limited dynamic range, particularly over a wide range of environmental conditions and target distances. Additionally, because of their sensitivity to even small numbers of photons, SPADs can be extremely susceptible to environmental levels of background noise light. SUMMARY

[0005] In some embodiments, an optical measurement system can include a housing of the optical measurement system and a light source configured to transmit one or more pulse trains in one or more time intervals as part of an optical measurement, where each of the one or more first time intervals can include one of the one or more pulse trains. The system can also include a light sensor configured to detect photons from the one or more pulse trains reflected off of the housing of the optical measurement system and to detect photons from the one or more pulse trains reflected off of an object in an environment surrounding the optical measurement system. The system can additionally include a plurality of registers configured to accumulate counts of the photons from the light sensor received during the one or more time intervals. Each of the one or more time intervals can be subdivided into a plurality of time bins. Each of the plurality of registers can be configured to accumulate counts of the photons received during a corresponding one of the plurality of time bins in each of the one or more time intervals to represent a histogram of the counts of the photons received during the one or more time intervals. The system can also include circuitry configured to identify an initial peak in the histogram of the counts of the photons. The initial peak can represent photons reflected off of the housing of the optical measurement system.

[0006] In any embodiment, any and all of the following features can be included in any combination, but are not limited thereto. The circuitry can be configured to identify the initial peak by identifying a predetermined number of registers of the plurality of registers that first appear in the plurality of registers. The circuitry can be configured to identify the initial peak by identifying one or more registers of the plurality of registers that store a highest number of counts of the photons. The circuitry can be configured to identify the initial peak by identifying a register of the plurality of registers having a time bin corresponding to a distance between the light source and the housing of the optical measurement system. The circuitry can also be configured to identify a subset of the plurality of registers representing the initial peak. The subset of the plurality of registers can be identified by selecting a predetermined number of registers near a register storing a maximum value of the initial peak. The subset of the plurality of registers can be identified by selecting registers near the register storing the maximum value of the initial peak that store values within a predetermined percentage of the maximum value. The circuitry can also be configured to estimate the distance between the light source and the housing of the optical measurement system based on a location of the initial peak in the plurality of registers. The circuitry can also be configured to use the estimated distance between the light source and the housing to calibrate distance measurements. The system can also include a processor configured to receive additional peaks stored in the histogram in the plurality of registers to calculate distances to objects in the surrounding environment corresponding to the additional peaks, where the initial peak can be excluded from the additional peaks received by the processor. The processor can be implemented in an integrated circuit separate and distinct from an integrated circuit in which the plurality of registers are implemented.

[0007] In some embodiments, a method of detecting a peak reflected from an enclosure in an optical measurement system can include transmitting one or more pulse trains in one or more time intervals as part of an optical measurement. Each of the one or more first time intervals can include one of the one or more pulse trains. The method can also include detecting photons from the one or more pulse trains reflected off of an enclosure of the optical measurement system and detecting photons from the one or more pulse trains reflected off of an object in an environment surrounding the optical measurement system. The method can additionally include accumulating counts of the photons received during the one or more time intervals into a plurality of registers. Each of the one or more time intervals can be subdivided into a plurality of time bins, and each of the plurality of registers accumulates counts of the photons received during a corresponding one of the plurality of time bins in each of the one or more time intervals to represent a histogram of the counts of the photons received during the one or more time intervals. The method can also include identifying an initial peak in the histogram of the counts of the photons. The initial peak can represent photons reflected off of the enclosure of the optical measurement system.

[0008] In any embodiment, any and all of the following features can be included, in any combination, but are not limited to. The method can also include identifying a second initial peak as part of a second optical measurement; and comparing the second initial peak to the initial peak. The method can also include characterizing a change in transparency of a window in an enclosure of the optical measurement system based on comparing the second initial peak to the initial peak. The method can additionally include identifying a plurality of initial peaks detected by a plurality of different light sensors in the optical measurement system; and determining a level of transparency of a corresponding section of the window in the enclosure of the optical measurement system in front of each of the plurality of light sensors based on the plurality of initial peaks. The method can also include comparing a maximum of the initial peak to a threshold value; and determining whether an obstruction is located outside of the optical measurement system based on comparing the maximum of the initial peak to the threshold value. The method can additionally include identifying a plurality of initial peaks in a plurality of measurements; and storing a baseline initial peak based on a combination of the plurality of initial peaks in the plurality of measurements for comparison to further optical measurements. The method can also include subtracting the baseline initial peak from the plurality of registers. The second peak can at least partially overlap the initial peak, and subtracting the baseline initial peak can cause the second peak to be detectable by the peak detection circuit. The second peak can correspond to an object in the environment surrounding the optical measurement system that is within two feet of the optical measurement system.

[0009] In some embodiments, an optical measurement system can include a light source configured to transmit one or more pulse trains in one or more time intervals as part of an optical measurement. Each of the one or more time intervals can include one of the one or more pulse trains. The system can also include a light sensor configured to detect photons from the one or more pulse trains reflected from an object in an environment surrounding the optical measurement system. The system can additionally include a plurality of registers configured to accumulate counts of the photons from the light sensor received during the one or more time intervals to represent an unfiltered histogram of the counts of the photons received during the one or more time intervals. The system can also include a filtering circuit configured to provide a filtered histogram of the counts of the photons from the plurality of registers. The system can also include a peak detection circuit configured to detect locations of peaks in the filtered histogram and use the locations of the peaks in the filtered histogram to identify locations in the plurality of registers that store unfiltered representations of the peaks.

[0010] In any embodiment, any and all of the following features can be included in any combination, but are not limited thereto. The system can also include a processor configured to receive the unfiltered representations of the peaks and use the unfiltered representations of the peaks to calculate distances to the object in the environment surrounding the optical measurement system. The filtering circuit can be configured to provide the filtered histogram by applying a matched filter corresponding to the one or more pulse trains. A pulse train of the one or more pulse trains can include a plurality of square pulses. The filtering circuit can be configured to low-pass filter the unfiltered histogram. The system can also include a second plurality of registers that store the filtered histogram. The filtered histogram can be generated by the plurality of registers in a single pass. The peaks can be detected by the plurality of registers during the single pass such that the filtered histogram is not entirely stored. The peak detection circuit can be configured to detect the locations of the peaks by detecting increasing values after decreasing values in the plurality of registers. The processor can be implemented on an integrated circuit (IC) that is separate and distinct from an IC on which the plurality of registers are implemented. The light source and the light sensor can form a pixel in a plurality of pixels in the optical measurement system.

[0011] In some embodiments, a method of analyzing filtered and unfiltered data in an optical measurement system can include transmitting one or more pulse trains as part of an optical measurement for one or more first time intervals. Each of the one or more first time intervals can include one of the one or more pulse trains. The method can also include detecting photons from the one or more pulse trains reflected off of an object in an environment surrounding the optical measurement system; populating a plurality of registers with the photons to represent an unfiltered histogram of photon counts received during the one or more first time intervals; filtering the unfiltered histogram in the plurality of registers to provide a filtered histogram of photons from the plurality of registers; detecting a location of a peak in the filtered histogram; using the location of the peak in the filtered histogram to identify a location in the plurality of registers that stores an unfiltered representation of the peak; and sending the unfiltered representation of the peak to a processor to use the unfiltered representation of the peak to calculate a distance to the object in the environment surrounding the optical measurement system.

[0012] In any embodiment, any and all of the following features can be included, in any combination, but are not limited to. Sending the unfiltered representation of the peak can include sending information representing an identified histogram time interval in the plurality of registers that stores the unfiltered representation of the peak. Filtering the unfiltered histogram in the plurality of registers can include applying a convolution of the unfiltered histogram with at least one square filter having a plurality of identical values. The plurality of identical values can include a series of binary "1" values and / or a series of "-1" values. Filtering the unfiltered histogram in the plurality of registers can include convolving the unfiltered histogram with at least one sequence of non-zero values followed by a plurality of zero values. The at least one sequence can include a single binary "1" value or a single binary "-1" followed by a plurality of "0" values. The method can also include sending a filtered representation of the peak to the processor in addition to sending the unfiltered representation of the peak. The location of the peak in the filtered histogram can be detected as a single peak in the filtered histogram; and the unfiltered representation of the peak can include at least two peaks in the unfiltered histogram. One of the at least two peaks in the unfiltered histogram can represent a peak caused by a reflection of the one or more pulse trains off of a housing or window of the optical measurement system. The photons can be detected using a plurality of photodetectors in a photosensor.

[0013] In some embodiments, an optical measurement system can include a plurality of light sources configured to emit one or more pulse trains in one or more time intervals as part of an optical measurement. The system can also include a plurality of light sensors configured to detect reflected photons from the one or more pulse trains emitted from a corresponding light source of the plurality of light sources. The plurality of light sensors can include a first light sensor and one or more other light sensors spatially adjacent to the first light sensor. The system can additionally include a plurality of memory blocks configured to accumulate, by a corresponding light sensor of the plurality of light sensors, photon counts of photons received during the one or more time intervals to represent a plurality of histograms of photon counts. The plurality of histograms can include a first histogram corresponding to the first light sensor and one or more histograms corresponding to the one or more other light sensors. The system can also include circuitry configured to combine information from the first histogram with information from the one or more other histograms to generate a distance measurement for the first light sensor.

[0014] In any embodiment, any and all of the following features can be included in any combination, but are not limited thereto. The one or more light sensors can be physically adjacent to the first light sensor in the array of light sensors. The array of light sensors can include a solid state array of light sensors. The one or more light sensors can include eight light sensors orthogonally adjacent or diagonally adjacent to the first light sensor. The one or more light sensors need not be physically adjacent to the first light sensor in the array of light sensors, but the one or more light sensors can be positioned to receive photons from a physical area that is adjacent to a physical area from which the first light sensor receives photons. The plurality of light sensors can be arranged in an array of light sensors that rotates around a central axis of the optical measurement system. The information from the first histogram can include a first distance measurement calculated based on the first histogram; the information from the one or more histograms can include one or more other distance measurements calculated based on the one or more other histograms; and the distance measurement can include a combination of the first distance measurement and the one or more other distance measurements. The first distance measurement can be below a detection limit of the optical measurement system before the first distance measurement is combined with the plurality of other distance measurements. The distance measurement can be above the detection limit of the optical measurement system after the first distance measurement is combined with the plurality of other distance measurements. The detection limit can represent a minimum number of photons received by a corresponding light sensor. The circuitry to combine the information from the first histogram with the information from the one or more histograms can include a processor implemented on an integrated circuit that is different from an integrated circuit on which the plurality of memory blocks are implemented. The circuitry and the plurality of memory blocks can be implemented on the same integrated circuit.

[0015] In some embodiments, a method of using spatially proximate pixel information in an optical measurement system can include transmitting one or more pulse trains over one or more first time intervals as part of an optical measurement; and detecting reflected photons from the one or more pulse trains using a plurality of photosensors. The plurality of photosensors can include a first photosensor and one or more photosensors spatially proximate to the first photosensor. The method can also include accumulating, by the plurality of photosensors, counts of photons received during the one or more time intervals to represent a plurality of histograms of photon counts. The plurality of histograms can include a first histogram corresponding to the first photosensor and one or more histograms corresponding to the one or more photosensors. The method can additionally include combining information from the first histogram with information from the one or more histograms to generate a distance measurement for the first photosensor.

[0016] In any embodiment, any and all of the following features can be included, in any combination, but are not limited to. The reflected photons received by the first photosensor and received by the one or more photosensors can be reflected from the same object in the surrounding environment. The information from the first histogram can include photon counts in the first histogram; the information from the one or more histograms can include photon counts in the one or more histograms; and the distance measurement can be calculated based on an aggregation of the photon counts in the first histogram and the photon counts in the one or more histograms. The information from the first histogram can include a first one or more peaks in the first histogram; the information from the one or more histograms can include a second one or more peaks in the one or more histograms; and the distance measurement can be calculated based on a combination of the first one or more peaks and the second one or more peaks. The distance measurement can be calculated based on a sum of the first one or more peaks and the second one or more peaks. The distance measurement can be calculated based on a Gaussian combination of the first one or more peaks and the second one or more peaks. The distance measurement can be calculated based on a convolution of the first one or more peaks and the second one or more peaks. The distance measurement can be calculated based on a weighted combination of the first one or more peaks and the second one or more peaks. BRIEF DESCRIPTION OF DRAWINGS

[0017] Further understanding of the nature and advantages of various embodiments can be realized by reference to the remaining portions of the specification and the drawings, wherein like reference numerals are used throughout several of the figures to refer to similar components. In some instances, a sub-label is associated with a reference numeral to denote one of multiple similar components. When reference is made to a reference numeral without specification to an existing sub-label, it is intended to refer to all such multiple similar components.

[0018] Figure 1A and 1B An automotive light ranging device, also referred to herein as a LIDAR system, is shown in accordance with some embodiments.

[0019] Figure 2 A block diagram showing an exemplary LIDAR device for implementing various embodiments is shown.

[0020] Figure 3 Operation of a typical LIDAR system that can be improved by embodiments is described.

[0021] Figure 4 An illustrative example of a light transmission and detection process for a light ranging system according to some embodiments is shown.

[0022] Figure 5 Various stages of a sensor array and associated electronics according to embodiments of the invention are shown.

[0023] Figure 6 A histogram according to embodiments of the invention is shown.

[0024] Figure 7 Accumulation of a histogram over multiple pulse trains for selected pixels according to embodiments of the invention is shown.

[0025] Figure 8 Circuitry for receiving photons and generating a set of signals stored in a memory representing a histogram according to some embodiments is shown.

[0026] Figure 9 Timing associated with different shots in a measurement according to some embodiments is shown.

[0027] Figure 10 A representation of a histogram memory including a peak caused by a reflection off a housing of an optical measurement system is illustrated.

[0028] Figure 11 A flowchart of a method for detecting a peak caused by an earlier reflection off a system housing using an optical measurement system is illustrated.

[0029] Figure 12 Contents of a histogram memory that can be used to calibrate distance measurements for an optical measurement system according to some embodiments are illustrated.

[0030] Figure 13A A portion of a histogram memory receiving an initial peak corresponding to a reflection off a system housing according to some embodiments is illustrated.

[0031] Figure 13B Variation in magnitude of a peak reflected off a housing according to some embodiments is illustrated.

[0032] Figure 14 A portion of a light sensor array that can be partially obscured by a contaminated portion of a window according to some embodiments is illustrated.

[0033] Figure 15A Illustrates initial peak detected by optical measurement system relative to blockage threshold according to some embodiments.

[0034] Figure 15B Illustrates initial peak at later in life of optical measurement system when blockage is present according to some embodiments.

[0035] Figure 16A Illustrates histogram memory with initial peak caused by enclosure reflection and second peak caused by reflection from object in ambient environment according to some embodiments.

[0036] Figure 16B Illustrates proximity peak detection problem without compensation for reflections out of enclosure according to some embodiments.

[0037] Figure 17A Illustrates Figure 16B Histogram memory.

[0038] Figure 17B Illustrates Figure 16B Histogram memory after baseline peak has been subtracted from register in histogram memory.

[0039] Figure 18 Illustrates circuitry for removing effects of reflections out of system enclosure during proximity measurement according to some embodiments.

[0040] Figure 19 Illustrates contents of histogram memory after single pulse is transmitted and received by optical measurement system according to some embodiments.

[0041] Figure 20 Illustrates filtered version of histogram data using filter of Figure 19 according to some embodiments.

[0042] Figure 21A Illustrates example of two adjacent peaks that are relatively close in time.

[0043] Figure 21B Illustrates how low pass filter can affect unfiltered histogram data of Figure 21A according to some embodiments.

[0044] Figures 22A-22B Illustrates how filtered data can be used to identify peaks in unfiltered data according to some embodiments.

[0045] Figure 23 Illustrates schematic of circuitry for using filtered data to pass unfiltered data to processor for distance calculation according to some embodiments.

[0046] Figure 24 A portion of a histogram memory after receiving reflected photons from a multi-pulse code is illustrated in accordance with some embodiments.

[0047] Figure 25 A filtered version of peaks received from a multi-pulse code is illustrated in accordance with some embodiments.

[0048] Figure 26 A filter that uses only a single binary indicator is illustrated in accordance with some embodiments.

[0049] Figure 27 A flowchart of a method for analyzing filtered and unfiltered data in an optical measurement system is illustrated in accordance with some embodiments.

[0050] Figure 28 An example of an object that can provide reflected photons to adjacent photosensors is illustrated in accordance with some embodiments.

[0051] Figure 29 An example of how spatially adjacent photosensor perspectives can be used to calculate a distance measurement for one of the corresponding photosensors is illustrated in accordance with some embodiments.

[0052] Figure 30 A way in which histograms can be combined for adjacent photosensors is illustrated in accordance with some embodiments.

[0053] Figure 31 An example of a rectangular photosensor layout is illustrated in accordance with some embodiments.

[0054] Figure 32 A configuration for rotating an optical measurement system is illustrated in accordance with some embodiments.

[0055] Figure 33 Circuitry for combining information from spatially adjacent histograms is illustrated in accordance with some embodiments.

[0056] Figure 34 Alternative circuitry for combining information from histograms is illustrated in accordance with some embodiments.

[0057] Figure 35 Another circuit for combining information from histograms is illustrated in accordance with some embodiments.

[0058] Figure 36 A flowchart of a method for using spatially adjacent pixel information in an optical measurement system is illustrated.

[0059] Terminology

[0060] The term "ranging" particularly when used in the context of methods and apparatus for measuring an environment or assisting vehicle operation can refer to determining a distance or distance vector from one location or position to another location or position. "Optical ranging" can refer to a class of ranging methods that utilize electromagnetic waves to perform the ranging method or function. Thus, an "optical ranging apparatus" can refer to an apparatus for performing an optical ranging method or function. "Lidar" or "LIDAR" can refer to a class of optical ranging methods that measure distance to a target by illuminating the target with a pulsed laser and thereafter measuring the reflected pulse with a sensor. Thus, a "lidar apparatus" or "lidar system" can refer to a class of optical ranging apparatus for performing a lidar method or function. An "optical ranging system" can refer to a system that includes at least one optical ranging apparatus (e.g., a lidar apparatus). The system can also include one or more other apparatus or components in various arrangements.

[0061] A "burst" can refer to one or more pulses transmitted together. The emission and detection of a burst can be referred to as a "shot." A shot can occur in a "detection time interval" (or "shot interval").

[0062] A "measurement" can include N bursts emitted and detected in N shots, each shot lasting a detection time interval. The entire measurement can be in a measurement time interval (or just "measurement interval"), which can be equal to the N shot intervals of the measurement or longer, e.g., when a pause occurs between shot intervals.

[0063] A "light sensor" or "light sensitive element" can convert light into an electrical signal. A light sensor can include a plurality of "light detectors," e.g., single photon avalanche diodes (SPADs). A light sensor can correspond to a particular resolution pixel in a ranging measurement.

[0064] A "histogram" can refer to any data structure representing a series of values over time, such as values discretized over time intervals. A histogram can have a value assigned to each time interval. For example, a histogram can store a counter of the number of light detectors that fired during a particular time interval in each of one or more shot intervals. As another example, a histogram can correspond to the digitization of an analog signal at different times. A histogram can include signal (e.g., pulses) and noise. Thus, a histogram can be viewed as a combination of signal and noise as a time series of photons or photon flux. A raw / digitized histogram (or cumulative photon time series) can contain signal and noise digitized in memory without filtering. A "filtered histogram" can refer to the output after a raw histogram passes through a filter.

[0065] The transmitted signal / pulse can refer to the "nominal," "ideal," or "template" pulse or pulse train that is not distorted. The reflected signal / pulse can refer to the reflected laser pulse from the object and can be distorted. The digitized signal / pulse (or raw signal) can refer to the digitized result from the detection of one or more pulse trains from a detection interval as stored in memory, and thus can be equivalent to a portion of a histogram. The detected signal / pulse can refer to the location in memory where the signal was detected. The detected pulse train can refer to the actual pulse train found by the matched filter. The expected signal profile can refer to the shape of the digitized signal caused by a particular transmitted signal with a particular distortion in the reflected signal. DETAILED DESCRIPTION

[0066] The present disclosure relates generally to the field of object detection and ranging, and more specifically to the use of time-of-flight optical receiver systems for applications such as real-time three-dimensional mapping and object detection, tracking, and / or classification. Various improvements can be realized with various embodiments of the present invention.

[0067] The sections below introduce an illustrative automotive LIDAR system, followed by a description of example techniques for detecting signals by a light ranging system, and then different embodiments are described in more detail.

[0068] I. Illustrative Automotive LIDAR System

[0069] Figure 1A And 1B An automotive light ranging device, also referred to herein as a LIDAR system, according to some embodiments is shown. The automotive application of the LIDAR system is chosen here for illustrative purposes only, and the sensors described herein can be used in other types of vehicles, such as boats, airplanes, trains, etc., as well as a variety of other applications where 3D depth images are useful, such as medical imaging, mobile phones, augmented reality, geodesy, geomatics, archaeology, topography, geology, geomorphology, seismology, forestry, atmospheric physics, laser guidance, airborne laser swath mapping (ALSM), and laser altimetry. According to some embodiments, a LIDAR system, such as scanning LIDAR system 101 and / or solid-state LIDAR system 103 can be mounted on the roof of a vehicle 105, such as shown in Figure 1A and 1B .

[0070] Figure 1AThe scanning LIDAR system 101 shown in FIG. 1 can employ a scanning architecture in which the orientation of the LIDAR light source 107 and / or detector circuit 109 can be scanned about one or more fields of view 110 within an external field or scene outside the vehicle 105. In the case of a scanning architecture, the emitted light 111 can be scanned, as shown, over the surrounding environment. For example, the output beam of one or more light sources (such as infrared or near-infrared pulsed IR lasers, not shown in the figure) positioned in the LIDAR system 101 can be scanned (e.g., rotated) to illuminate the scene in the vicinity of the vehicle. In some embodiments, the scanning, represented by the rotating arrow 115, can be implemented by mechanical means, for example, by mounting the light emitter to a rotating column or platform. In some embodiments, the scanning can be implemented by other mechanical means, such as by using a galvanometer. Chip-based steering techniques can also be employed, for example, by using a microchip employing one or more MEMS-based reflectors, such as a digital micromirror (DMD) device, a digital light processing (DLP) device, and the like. In some embodiments, the scanning can be achieved by non-mechanical means, for example, by using electronic signals to steer one or more optical phased arrays.

[0071] For a fixed architecture, as Figure 1B In the solid-state LIDAR system 103 shown in FIG. 1, one or more solid-state LIDAR subsystems (e.g., 103a and 103b) can be mounted to the vehicle 105. Each solid-state LIDAR unit can face a different direction (with possibly partially overlapping and / or non-overlapping fields of view between units) in order to capture a larger field of view than each unit could capture on its own.

[0072] In a scanning or fixed architecture, objects within a scene can reflect portions of the light pulses emitted from the LIDAR light source. The one or more reflected portions then travel back to the LIDAR system and can be detected by the detector circuit. For example, the reflected portion 117 can be detected by the detector circuit 109. The detector circuit can be housed in the same housing as the emitter. Aspects of scanning systems and fixed systems are not mutually exclusive and can thus be used in combination. For example, Figure 1B The individual LIDAR subsystems 103a and 103b in FIG. 1 can employ steerable emitters, such as optical phased arrays, or the entire composite unit can be rotated by mechanical means, thereby scanning the entire scene in front of the LIDAR system, for example, from field of view 119 to field of view 121.

[0073] Figure 2 A more detailed block diagram of a rotating LIDAR system 200 is illustrated in accordance with some embodiments. More specifically, Figure 2 A rotating LIDAR system that can employ a rotating actuator on a rotating circuit board, which can receive power and data from a stationary circuit board (as well as transmit power and data), is optionally illustrated.

[0074] The LIDAR system 200 can interact with one or more instances of a user interface 215. Different instances of the user interface 215 can vary and can include, for example, a computer system with a monitor, keyboard, mouse, CPU, and memory; a touch screen in a car; a hand-held device with a touch screen; or any other appropriate user interface. The user interface 215 can be local to the object on which the LIDAR system 200 is installed, but can also be a remotely operated system. For example, commands and data to / from the LIDAR system 200 can be routed through a cellular network (LTE, etc.), a personal area network (Bluetooth, Zigbee, etc.), a local area network (WiFi, IR, etc.), or a wide area network such as the Internet.

[0075] The user interface 215, whether hardware and software, can present LIDAR data to a user from the device, but can also allow the user to control the LIDAR system 200 with one or more commands. Example commands can include commands to activate or deactivate the LIDAR system, specify light detector exposure levels, bias, sampling durations, and other operational parameters (e.g., transmit pulse patterns and signal processing), specify light emitter parameters such as brightness. In addition, commands can allow the user to select a method for displaying results. The user interface can display LIDAR system results, which can include, for example, single frame snapshot images, constantly updated video images, and / or displays of other light measurements for some or all pixels. In some embodiments, the user interface 215 can track the distance of an object from the vehicle (proximity) and potentially provide a warning to the driver or provide such tracking information for analysis of driver performance.

[0076] In some embodiments, the LIDAR system can communicate with a vehicle control unit 217 and can modify one or more parameters associated with control of the vehicle based on received LIDAR data. For example, in a fully autonomous vehicle, the LIDAR system can provide real-time 3D images of the environment around the car to assist in navigation. In other cases, the LIDAR system can be used as part of an advanced driver assistance system (ADAS) or part of a safety system, which can provide 3D image data to any number of different systems, such as adaptive cruise control, automatic parking, driver drowsiness monitoring, blind spot monitoring, collision avoidance systems, etc. When the vehicle control unit 217 is communicably coupled to the light ranging device 210, a warning can be provided to the driver or tracking of proximity of an object can be tracked.

[0077] Figure 2The LIDAR system 200 shown in FIG. 1 includes a light ranging device 210. The light ranging device 210 includes a ranging system controller 250, a light transmission (Tx) module 240, and a light sensing (Rx) module 230. Ranging data can be generated by the light ranging device by transmitting one or more light pulses 249 from the light transmission module 240 to an object in a field of view surrounding the light ranging device. Reflected portions 239 of the transmitted light are then detected by the light sensing module 230 after some delay time. Based on the delay time, a distance to the reflecting surface can be determined. Other ranging methods, such as continuous wave, Doppler, and the like, can also be employed.

[0078] The Tx module 240 includes a transmitter array 242, which can be a one- or two-dimensional array of transmitters, and a Tx optical system 244, which when combined together can form a micro-optical transmitter channel array. The transmitter array 242 or individual transmitters are examples of laser sources. The Tx module 240 also includes a processor 245 and a memory 246. In some embodiments, a pulse coding technique, such as a Barker code or the like, can be used. In such cases, the memory 246 can store a pulse code that indicates when light should be transmitted. In one embodiment, the pulse code is stored as a sequence of integers stored in the memory.

[0079] The Rx module 230 can include a sensor array 236, which can be, for example, a one- or two-dimensional array of light sensors. Each light sensor or photosensitive element (also referred to as a sensor) can include a series of light detectors, such as APDs or the like, or the sensor can be a single photon detector (e.g., a SPAD). Similar to the Tx module 240, the Rx module 230 includes an Rx optical system 237. The Rx optical system 237 and the sensor array 236, when combined together, can form a micro-optical receiver channel array. Each micro-optical receiver channel measures light corresponding to an image pixel in a distinct field of view of the surrounding volume. For example, due to the geometric configuration of the light sensing module 230 and the light transmission module 240, each sensor (e.g., a series of SPADs) of the sensor array 236 can correspond to a particular transmitter of the transmitter array 242.

[0080] In one embodiment, the sensor array 236 of the Rx module 230 is fabricated as part of a monolithic device on a single substrate (using, e.g., CMOS technology) that includes an array of photon detectors and an ASIC 231 for signal processing of raw histograms from individual photon detectors (or groups of detectors) in the array. As an example of signal processing, for each photon detector or group of photon detectors, the memory 234 (e.g., SRAM) of the ASIC 231 can accumulate counts of photons detected over successive time intervals, and these time intervals, taken together, can be used to recreate a time series of the reflected light pulse (i.e., photon counts vs. time). This time series of aggregated photon counts is referred to herein as an intensity histogram (or just histogram). The ASIC 231 can implement matched filter and peak detection processing and time identification of the return signal. In addition, the ASIC 231 can implement certain signal processing techniques (e.g., by the processor 238), such as multi-curve matched filtering, to help recover the photon time series that are less affected by pulse shape distortions that occur due to SPAD saturation and quenching. In some embodiments, all or part of such filtering can be performed by the processor 258, which can be implemented in an FPGA.

[0081] In some embodiments, the Rx optical system 237 can also be part of the same monolithic structure as the ASIC, with a separate substrate layer for each receiver channel layer. For example, the aperture layer, collimating lens layer, filter layer, and light detector layer can be stacked and bonded at wafer level before dicing. The aperture layer can be formed by disposing an opaque substrate on top of a transparent substrate or by coating the transparent substrate with an opaque film. In yet other embodiments, one or more components of the Rx module 230 can be external to the monolithic structure. For example, the aperture layer can be implemented as a separate sheet of metal with pinholes.

[0082] In some embodiments, the photon time series output from the ASIC are sent to the ranging system controller 250 for further processing, e.g., the data can be encoded by one or more encoders of the ranging system controller 250 and then sent as data packets to the user interface 215. The ranging system controller 250 can be implemented in a number of ways, including, for example, by using a programmable logic device such as an FPGA as an ASIC or as part of an ASIC, using a processor 258 with memory 254, and some combination of the above. The ranging system controller 250 can operate in cooperation with the fixed base controller or independently of the base controller (via preprogrammed instructions) to control the light sensing module 230 by sending commands including starting and stopping light detection and adjusting light detector parameters. Similarly, the ranging system controller 250 can control the light transmission module 240 by sending commands or forwarding commands from the base controller including starting and stopping light emission control and can adjust other light emitter parameters (e.g., pulse code). In some embodiments, the ranging system controller 250 has one or more wired interfaces or connectors for exchanging data with the light sensing module 230 and the light transmission module 240. In other embodiments, the ranging system controller 250 communicates with the light sensing module 230 and the light transmission module 240 via wireless interconnections such as optical communication links.

[0083] A motor 260 can be an optional component needed when system components such as the Tx module 240 and or Rx module 230 need to be rotated. The system controller 250 controls the motor 260 and can start rotation, stop rotation, and change the rotational speed.

[0084] II. Detection of Reflected Pulses

[0085] The light sensors can be arranged in a number of ways for detecting the reflected pulses. For example, the light sensors can be arranged in an array and each light sensor can include an array of light detectors (e.g., SPADs). Different patterns of transmitted pulses (pulse trains) during a detection interval are also described below.

[0086] A. Time-of-Flight Measurement and Detector

[0087] Figure 3 The operation of a typical LIDAR system that can be improved by some embodiments is illustrated. A laser generates a short duration light pulse 310. The horizontal axis represents time and the vertical axis represents power. An example laser pulse duration, characterized by a full width at half maximum (FWHM), is a few nanoseconds with a peak power of about a few watts for a single emitter. Embodiments using side emitter lasers or fiber lasers can have much higher peak powers, while embodiments with small diameter VCSELs can have peak powers of tens of milliwatts to hundreds of milliwatts.

[0088] The start time 315 of the transmission of the pulse need not coincide with the leading edge of the pulse. As shown, the leading edge of the optical pulse 310 can be after the start time 315. It can be desirable for the leading edge to be different in cases where pulses of different patterns are transmitted at different times, e.g., for encoded pulses.

[0089] The optical receiver system can start detecting received light at the same time as the laser is started, i.e., at the start time. In other embodiments, the optical receiver system can start at a later time, which is a known time after the start time of the pulse. The optical receiver system initially detects background light 330 and after some time detects the laser pulse reflection 320. The optical receiver system can compare the detected light intensity to a threshold to identify the laser pulse reflection 320. The threshold can distinguish between the background light 330 and light corresponding to the laser pulse reflection 320.

[0090] The time of flight 340 is the time difference between the sending of the pulse and the receiving of the pulse. The time difference can be measured by subtracting the transmission time of the pulse (e.g., as measured relative to the start time) from the time of receipt of the laser pulse reflection 320 (e.g., also measured relative to the start time). The distance to the target can be determined as half the product of the time of flight and the speed of light. Pulses from the laser device are reflected at different times from objects in the scene, and the pixel array detects the reflected pulses of radiation.

[0091] B. Object detection using array lasers and light sensor arrays

[0092] Figure 4 An illustrative example of a light transmission and detection process for a light ranging system is shown in accordance with some embodiments. Figure 4 A light ranging system (e.g., solid state or and / or scanning) that collects three-dimensional distance data for a volume or scene around the system is shown. Figure 4 Is an idealized drawing highlighting the relationship between the emitters and the sensors, and thus other components are not shown.

[0093] The light ranging system 400 includes a light emitter array 402 and a light sensor array 404. The light emitter array 402 includes an array of light emitters, e.g., a VCSEL array and the like, such as emitter 403 and emitter 409. The light sensor array 404 includes an array of light sensors, e.g., sensors 413 and 415. The light sensors can be pixelated light sensors that employ a set of discrete light detectors, such as single photon avalanche diodes (SPADs) and the like, for each pixel. However, various embodiments can deploy any type of photonic sensor.

[0094] Each emitter may be slightly offset from its neighbor and may be configured to transmit light pulses into a different field of view than its neighboring emitter, thereby illuminating only the corresponding field of view associated with that emitter. For example, emitter 403 emits an illumination beam 405 (formed by one or more light pulses) into a circular field of view 407 (magnified for clarity). Similarly, emitter 409 emits an illumination beam 406 (also referred to as the emitter channel) into a circular field of view 410. Although Figure 4 Not shown to avoid complexity, but each emitter emits a corresponding illumination beam into its corresponding field of view, thus causing a 2D array of illumination fields of view (in this example, 21 distinct fields of view).

[0095] Each field of view illuminated by the transmitter can be considered as a pixel or spot in the corresponding 3D image that generates self-range data. Each transmitter channel can be distinct from each transmitter and non-overlapping with other transmitter channels; that is, there is a one-to-one mapping between the set of transmitters and the set of non-overlapping fields or viewpoints. Therefore, in Figure 4 In this example, the system can sample 21 dissimilar points in 3D space. Denser point sampling can be achieved by having a denser array of transmitters or by scanning the angular position of the transmitter beam over time so that a single transmitter can sample several points in space. As described above, scanning can be achieved by rotating the entire transmitter / sensor assembly.

[0096] Each sensor can be slightly offset from its neighbor, and similar to the transmitter described above, each sensor can see a different field of view of the scene in front of it. Furthermore, the field of view of each sensor is substantially consistent with the field of view of the corresponding transmitter channel, for example, overlapping with it and being the same size.

[0097] exist Figure 4 In this system, the distance between the corresponding transmitter-sensor channels is magnified relative to the distance to the object in the field of view. In reality, the distance to the object in the field of view is much larger than the distance between the corresponding transmitter-sensor channels, and therefore the path of light from the transmitter to the object is roughly parallel to the path of reflected light returning from the object to the sensor (i.e., it is almost "reflected back"). Therefore, there exists a range of distances in front of the system 400 where the fields of view of the individual sensors and transmitters overlap.

[0098] Because the field of view of a transmitter overlaps with the field of view of its corresponding sensor, each sensor channel can ideally detect a reflected illumination beam originating from its corresponding transmitter channel that ideally has no cross-talk, i.e., no reflected light from other illumination beams. Thus, each light sensor can correspond to a corresponding light source. For example, transmitter 403 transmits an illumination beam 405 into a circular field of view 407, and some of the illumination beams reflect from an object 408. Ideally, reflected beam 411 is detected only by sensor 413. Thus, transmitter 403 and sensor 413 share the same field of view, e.g., field of view 407, and form a transmitter-sensor pair. Likewise, transmitter 409 and sensor 415 form a transmitter-sensor pair, sharing field of view 410. Although transmitter-sensor pairs are shown in FIG. 4 as being in the same relative position in their respective arrays, any transmitter can be paired with any sensor, depending on the design of the optics used in the system. Figure 4

[0099] During a ranging measurement, reflected light from different fields of view distributed around a volume surrounding the LIDAR system is collected by various sensors and processed to yield distance information for any objects in each respective field of view. As described above, a time-of-flight technique can be used, in which a light transmitter transmits a precisely timed pulse, and the reflection of the pulse is detected by a corresponding sensor after some elapsed time. The elapsed time between transmission and detection, along with a known speed of light, is then used to calculate the distance to the reflecting surface. In some embodiments, additional information can be obtained by the sensor to determine other properties of the reflecting surface in addition to distance. For example, the Doppler shift of the pulse can be measured by the sensor and used to calculate the relative velocity between the sensor and the reflecting surface. The pulse intensity can be used to estimate the target reflectivity, and the pulse shape can be used to determine whether the target is a hard or diffuse material.

[0100] IIn some embodiments, the LIDAR system can be composed of a relatively large 2D array of transmitter and sensor channels and operate as a solid-state LIDAR, i.e., it can obtain frames of range data without needing to scan the orientation of the transmitters and / or sensors. In other embodiments, the transmitters and sensors can be scanned, e.g., rotated about an axis, to ensure that the fields of view of the collection of transmitters and sensors sample a full 360 degree region (or some useful portion of a 360 degree region) of the surrounding volume. For example, range data collected from a scanning system over some predefined period of time can then be post-processed into one or more data frames, which can then be further processed into one or more depth images or 3D point clouds. The depth images and / or 3D point clouds can be further processed into map tiles for 3D mapping and navigation applications.

[0101] C. Multiple light detectors in each light sensor

[0102] Figure 5 ​Various stages of a sensor array and associated electronics according to embodiments of the application are shown. Array 510 shows light sensors 515, each corresponding to a different pixel. Array 510 can be a staggered array. In this particular example, array 510 is 18x4 light sensors. Array 510 can be used to achieve high resolution (e.g., 72x1024) as the implementation is amenable to scanning.

[0103] Array 520 shows a magnified view of a portion of array 510. As can be seen, each light sensor 515 is made up of a plurality of light detectors 525. The signals from the light detectors of a pixel collectively contribute to the measurement of the pixel.

[0104] In some embodiments, each pixel has a large number of single-photon avalanche diode (SPAD) cells, which increases the dynamic range of the pixel itself. Each SPAD can have an analog front-end circuit for biasing, quenching, and recharging. The SPAD is typically biased with a bias voltage above the breakdown voltage. A suitable circuit senses the leading edge of the avalanche current, generates a standard output pulse synchronized with the avalanche build-up, quenches the avalanche by lowering the bias down below the breakdown voltage, and restores the photodiode to an operating level.

[0105] The SPADs can be positioned so as to maximize their fill factor in the local area, or a microlens array can be used, which allows for high optical fill factor at the pixel level. Thus, the imager pixel can include an array of SPADs to increase the efficiency of the pixel detector. A diffuser can be used to diffuse the rays passing through the aperture and collimated by the microlenses. A pot diffuser is used to diffuse the collimated rays in a way that some radiation is received by all the SPADs belonging to the same pixel.

[0106] Figure 5 A particular light detector 530 (e.g., SPAD) that detects a photon 532 is also shown. In response to the detection, light detector 530 generates an avalanche current 534 of charge carriers (electrons or holes). A threshold circuit 540 regulates the avalanche current 534 by comparing it to a threshold. When a photon is detected and the light detector 530 is functioning properly, the avalanche current 534 rises above the comparator threshold, and threshold circuit 540 generates a time-accurate binary signal 545 that indicates the exact time of the SPAD current avalanche, which in turn is an accurate measurement of the photon arrival. The correlation of the current avalanche to the photon arrival occurs with nanosecond resolution, in turn providing high timing resolution. The rising edge of binary signal 545 can be latched by a pixel counter 550.

[0107] The binary signal 545, avalanche current 534, and pixel counter 550 are examples of data values that can be provided by a photosensor including one or more SPADs. The data values can be determined from respective signals from each of a plurality of photodetectors. Each of the respective signals can be compared to a threshold to determine whether the corresponding photodetector was triggered. The avalanche current 534 is an example of an analog signal, and thus the respective signals can be analog signals.

[0108] The pixel counter 550 can count the number of photodetectors for a given pixel that have been triggered by one or more photons during a particular time interval (e.g., a 1, 2, 3, etc. nanosecond time window) controlled by the periodic signal 560 using the binary signal 545. The pixel counter 550 can store a counter for each of a plurality of time intervals for a given measurement. The value of the counter for each time interval can start at zero and be incremented based on the binary signal 545 indicating that a photon was detected. The counter can be incremented when any photodetector of the pixel provides such a signal.

[0109] The periodic signal 560 can be generated by a phase-locked loop (PLL) or a delay-locked loop (DLL) or any other method of generating a clock signal. The coordination of the periodic signal 560 and the pixel counter 550 can act as a time-to-digital converter (TDC), which is a device for recognizing events and providing a digital representation of the time at which they occurred. For example, the TDC can output the time of arrival of each detected photon or optical pulse. The measured time can be the time elapsed between two events (e.g., a start time and a detected photon or optical pulse) rather than an absolute time. The periodic signal 560 can be a relatively fast clock that switches between a set of memories including the pixel counter 550. Each register in the memories can correspond to one histogram bin, and the clock can switch between them at the sampling interval. Thus, when the respective signal is greater than the threshold, a binary value indicating a trigger can be sent to a histogram circuit. The histogram circuit can aggregate the binary values across a plurality of photodetectors to determine the number of photodetectors that were triggered during a particular time interval.

[0110] The time intervals can be measured relative to a start signal, such as in Figure 3The start time 315 of the time interval. Thus, the counter of the time interval just after the start signal can have a low value corresponding to a background signal, such as background light 330. The last time interval can correspond to the end of the detection time interval (also referred to as the shot) of a given burst, which is further described in the next section. The number of periods of the periodic signal 560 since the start time can serve as a timestamp when the rising edge of the avalanche current 534 indicates a detected photon. The timestamp corresponds to the time interval for a particular counter in the pixel counter 550. Such operation is different from a simple analog-to-digital converter (ADC) following a photodiode, such as for an avalanche photodiode (APD). Each of the counters of the time intervals can correspond to a histogram, which is described in more detail below. Thus, while an APD is a linear amplifier for the input optical signal with a finite gain, a SPAD is a flip-flop device that provides a binary output of yes / no to a triggered event that occurs in a time window.

[0111] D. Burst

[0112] Ranging can also be achieved by using a burst, defined as containing one or more pulses. Within a burst, the number of pulses, the width of the pulses, and the duration between pulses (collectively referred to as the pulse pattern) can be selected based on several factors, some of which include:

[0113] 1 - Maximum laser duty cycle - The duty cycle is the fraction of time that the laser is on. For pulsed lasers, this can be determined by the FWHM as explained above and the number of pulses emitted during a given period.

[0114] 2 - Eye safety limits - This is determined by the maximum amount of radiation that the device can emit without harming the eyes of a bystander who happens to be looking in the direction of the LIDAR system.

[0115] 3 - Power consumption - This is the power consumed by the emitter in order to illuminate the scene.

[0116] For example, the interval between pulses in a burst can be on the order of single digits or tens of nanoseconds.

[0117] Multiple bursts can be emitted during the time span of one measurement. Each burst can correspond to a different time interval, such as not emitting a subsequent burst before the time limit for detecting a reflected pulse of a previous burst expires.

[0118] For a given emitter or laser device, the time between the emission of pulse trains determines the maximum detectable range. For example, if pulse train A is emitted at time t0 = 0 ns and pulse train B is emitted at time t1 = 1000 ns, then reflected pulse trains detected after t1 must not be assigned to pulse train A, as they are more likely reflections from pulse train B. Therefore, the time between pulse trains and the speed of light define the maximum limit of the system's range, as given in the following equation:

[0119] R max = c×(t1-t0) / 2

[0120] The time between excitation (emission and detection of the pulse train) can be about 1 μs to allow the entire pulse train enough time to travel to a distant object approximately 150 meters away and then return.

[0121] III. Histogram signal from the photodetector

[0122] One operating mode of a LIDAR system is Time-Correlated Single-Photon Counting (TCSPC), which is based on counting individual photons in a periodic signal. This technique is well-suited for low levels of periodic radiation, making it appropriate for LIDAR systems. This time-correlated counting can be affected by... Figure 5 The periodic signal 560 is controlled and can be used within a time interval, such as for... Figure 5 The discussion.

[0123] The frequency of a periodic signal specifies a time resolution within which data values ​​of the signal are measured. For example, a measurement can be obtained for each photodetector in each cycle of the periodic signal. In some embodiments, the measurement can be the number of photodetectors triggered during the cycle. The time period of the periodic signal corresponds to a time interval, where each cycle is a different time interval.

[0124] Figure 6 A histogram 600 is shown according to an embodiment of the invention. The horizontal axis corresponds to the time interval as measured relative to a start time 615. As described above, the start time 615 may correspond to the start time of the pulse train. Any offset between the rising edge of the first pulse of the pulse train and the start time of any one or both of the pulse train and the detection time interval may be considered, where the reception time will be used for time-of-flight measurement. The vertical axis corresponds to the number of SPADs triggered. In some embodiments, the vertical axis may correspond to the output of an ADC following an APD. For example, the APD may exhibit conventional saturation effects, such as a constant maximum signal rather than the dead-time-based effect of the SPAD. Some effects may occur for both SPADs and APDs; for example, pulse tailing on a highly tilted surface may occur for both SPADs and APDs.

[0125] The counters for each of the time intervals correspond to different bins in the histogram 600. The counters for early time intervals are relatively low and correspond to background noise 630. At some point, a reflected pulse 620 is detected. The corresponding counter is much larger and can be above a threshold that distinguishes between background and detected pulses. The reflected pulse 620 (after digitization) is shown as corresponding to four time intervals, which can result from a laser pulse with a similar width, e.g., a 4 ns pulse when the time intervals are each 1 ns. But as described in more detail below, the number of time intervals can vary, e.g., based on properties of the particular object in the angle of incidence of the laser pulses.

[0126] The time position of the time intervals corresponding to the reflected pulse 620 can be used to determine a reception time, e.g., relative to the start time 615. As described in more detail below, a matched filter can be used to identify the pulse pattern, which in turn effectively increases the signal-to-noise ratio, and more accurately determines the reception time. In some embodiments, the accuracy of determining the reception time can be less than the time resolution of a single time interval. For example, for a time interval of 1 ns, the resolution would correspond to about 15 cm. However, an accuracy of only a few centimeters can be needed.

[0127] Thus, a detected photon can cause a particular time interval of the histogram to be incremented based on its arrival time relative to a start signal, e.g., indicated by the start time 615. The start signal can be periodic, such that multiple pulse trains are sent during a measurement. Each start signal can be synchronized to a laser pulse train, with multiple start signals such that multiple pulse trains are transmitted over multiple detection intervals. Thus, a time interval (e.g., from 200 to 201 ns after a start signal) will occur for each detection interval. The histogram can accumulate counts, with the count for a particular time interval corresponding to the sum of measured data values that all occurred in that particular time interval across multiple excitations. When detected photons are histogrammed based on such techniques, they result in a signal-to-noise ratio for a return signal that is the square root of the number of excitations made larger than a single pulse train.

[0128] Figure 7 Cumulative accumulation of a histogram over multiple pulse trains for a selected pixel is shown in accordance with an embodiment of the application. Figure 7 Three detected pulse trains 710, 720, and 730 are shown. Each detected pulse train corresponds to a transmitted pulse train of the same pattern with two pulses separated by the same amount of time. Thus, each detected pulse train has the same pulse pattern, as shown by the two time intervals with distinct values. The counters for the other time intervals are not shown for ease of illustration, but the other time intervals can have relatively low non-zero values.

[0129] In the first detected pulse train 710, the counters for time intervals 712 and 714 are the same. This can result from the same number of photons being detected by the light detector during the two time intervals. Or in other embodiments, approximately the same number of photons are detected during the two time intervals. In other embodiments, more than one consecutive time interval can have a consecutive non-zero value; but for ease of illustration, individual non-zero time intervals have been shown.

[0130] Time intervals 712 and 714 occur 458 ns and 478 ns after start time 715, respectively. The shown counters for the other detected pulse trains occur at the same time intervals relative to their respective start times. In this example, start time 715 is identified as occurring at time 0, but the actual time is arbitrary. A first detection interval for the first detected pulse train can be 1 μβ. Thus, the number of time intervals measured from start time 715 can be 1,000. Then, this first detection interval ends, and a new pulse train can be transmitted and detected. The start and end of different time intervals can be controlled by a clock signal, which can be part of a circuit acting as a time-to-digital converter (TDC), such as described in Figure 5 .

[0131] For the second detected pulse train 720, start time 725 is at 1 μβ, such as when the second pulse train can be emitted. Such separate detection intervals can occur so that any pulses transmitted at the start of the first detection interval will have been detected, and thus will not cause confusion with pulses detected in the second time interval. For example, if there is no additional time between excitations, the circuit can confuse the back-reflected light barrier sign at 200 m with an object that reflects much less at 50 m (assuming an excitation period of about 1 us). The two detection time intervals for pulse trains 710 and 720 can be the same length and have the same relationship to the respective start times. Time intervals 722 and 724 occur at the same relative times as time intervals 712 and 714, 458 ns and 478 ns. Thus, when the accumulation step occurs, the corresponding counters can be added. For example, the counter values at time intervals 712 and 722 can be added together.

[0132] For the third detected pulse train 730, start time 735 is at 2 μβ, such as where the third pulse train can be emitted. Time intervals 732 and 734 also occur at 458 ns and 478 ns relative to their respective start time 735. Even if the emitted pulses have the same power, the counters for different time intervals can have different values, such as due to the random nature of the scattering process by which the light pulses leave the object.

[0133] Histogram 740 shows the accumulation from the counters of three detected pulse trains at time intervals 742 and 744, which also correspond to 458 ns and 478 ns. Histogram 740 can have a fewer number of time intervals measured during the respective detection interval, e.g., due to discarding time intervals at the beginning or end, or having values less than a threshold. In some implementations, about 10 to 30 time intervals can have significant values, depending on the pattern of pulse trains.

[0134] For example, the number of pulse trains emitted to create a single histogram during a measurement can be about 1 to 40 (e.g., 24), but can also be much higher, e.g., 50, 100, or 500. Once a measurement is complete, the counters used for the histogram can be reset, and a set of pulse trains can be emitted to perform a new measurement. In various embodiments and depending on the number of detection intervals in the respective duration, a measurement can be performed every 25, 50, 100, or 500 ps. In some embodiments, the measurement intervals can overlap, e.g., so a given histogram corresponds to a particular sliding window of pulse trains. Any weighting applied to detected photons can be the same for each histogram, or such weighting can be independently controlled.

[0135] IV. Histogram Data Path

[0136] Figure 8 Circuitry for receiving photons and generating a set of signals stored in a memory representing a histogram is shown, according to some embodiments. As with respect to Figure 5 As described above, the array of photosensors can be used to receive reflected pulses and background photons from ambient light in the optical measurement system. A single photosensor 802 can include a plurality of photodetectors. Each photodetector can be implemented by a SPAD or other light-sensitive sensor, and the photodetectors can be arranged in a grid pattern for the photosensor 802, as illustrated in Figure 8 A filter can be used in each of the photosensors to block light received by the photosensor outside the range centered around the light source of the LIDAR system. However, even with this filter, some ambient light at or near the wavelength emitted by the light source can pass through the filter. This can result in photons from the ambient light as well as photons emitted from the LIDAR light source being received by the photosensor.

[0137] Each photodetector in the photosensor 802 can include an analog front-end circuit for generating an output signal indicating when a photon is received by the photodetector. For example, referring back to Figure 5 , the avalanche current 534 from the SPAD can trigger the threshold circuit 540 to generate an output binary signal 545. Turning back to Figure 8Each photodetector in the photosensor 802 can generate its own signal corresponding to received photons. Thus, the photosensor 802 can generate a set of signals 816 corresponding to the number of photodetectors in the photosensor 802. The photosensor 802 can also be referred to as a "pixel" or "pixel sensor" as it can correspond to a single pixel of information when displayed or analyzed in later stages of the optical measurement system. When a signal is generated (e.g., transitioning from a logical "0" to a logical "1") in response to a received photon, this can be referred to as a "positive" signal.

[0138] An arithmetic logic unit (ALU) 804 can be used to implement the functionality of the pixel counter 550 of Figure 5 . In particular, the ALU 804 can receive the set of signals 816 from the individual photodetectors of the photosensor 802 and aggregate the number of these signals each indicating a detected photon. The ALU 804 can include a combination of digital electronic circuits that perform arithmetic and / or other bitwise operations on the set of signals 816. For example, the ALU 804 can receive each of the set of signals 816 as a binary signal (i.e., "0" or "1") as an input or operand to the ALU 804. By aggregating or adding together the inputs, the ALU 804 can count the number of positive signals in the set of signals 816 indicating that a photon has been received within a particular time interval. For example, by adding each of the signals indicating a "1" signal level, the output of the ALU 804 can indicate the number of signals in the set of signals 816 associated with photodetectors that also received a photon during the time interval.

[0139] The ALU 804 is specifically designed to receive at least the number of inputs corresponding to the number of photodetectors in the photosensor 802. In the example of a 32-bit ALU 804, the ALU 804 can be configured to receive 32 parallel inputs of a single bit width. Internally, the ALU 804 can be implemented with digital logic gates to form ripple carry adders, look-ahead carry adders, carry-save adders, and / or any other type of adder that can aggregate a relatively large number of inputs with low propagation time. The output of the ALU 804 can be referred to as a "total signal count" and can be represented as an n-bit binary number output from the ALU 804 or from a stage of the ALU 804. Figure 8

[0140] ​As described above, the output of the ALU 804 can characterize the total number of photons received by the photosensor 802 during a particular time bin. Each time the ALU 804 completes an accumulation operation, the total signal count can be added to the corresponding memory location in the memory 806 representing the histogram 818. In some embodiments, the memory 806 can be implemented using SRAM. Thus, over the course of multiple excitations, each of which includes a burst, the total signal count from the ALU 804 can be accumulated with the existing value in the corresponding memory location in the memory 806. A single measurement can include multiple excitations that fill the memory 806 to generate the histogram 818 of values in the time bins that can be used to detect reflected signals, background noise, peaks, and / or other signals of interest.

[0141] The ALU 804 can also perform a second accumulation operation that adds the total signal count to the existing value in the memory location of the memory 806. Recall Figure 7 that with each excitation, the new total signal count can be added to the existing value in the corresponding time bin of the memory 806. In this way, the histogram 818 can be built up in the memory 806 over several excitations. When the total signal count is generated by the ALU 814, the current value 820 for the corresponding memory location of the memory 806 for that time bin can be retrieved. The current value 820 can be provided as an operand to the ALU 804, which can be combined with the total signal count from the set of signals 816. In some embodiments, the ALU 804 can be composed of a first stage that calculates the total signal count from the photosensor 802 and a second stage that combines the total signal count with the current value 820 from the memory location in the memory 806 for the time bin. In some embodiments, the accumulation of the set of signals 816 and the accumulation of the total signal count and the current value 820 can be performed as a single operation. Thus, even though these two operations can be functionally described as separate "accumulations," they can actually be performed together using a combination of parallel and sequential circuitry in the ALU 804.

[0142] As described above with respect to Figure 5 the ALU 804 can receive a periodic signal 560 that triggers the accumulation operation. The periodic signal 560 can be generated using any of the techniques described above. The periodic signal 560 can define the length of each time bin. In some embodiments, the periodic signal 560 and the corresponding time bins can be relative to the time of the excitation as Figure 3Each cycle of the periodic signal 560 can cause an accumulation operation to be performed in the ALU 804 and can cause a memory address of the memory 860 to be incremented to the next time interval. For example, a rising edge of the periodic signal 560 can cause the ALU 804 to produce a result that accumulates the total signal count and the current value 820 together. A corresponding periodic signal 808 can also be sent to the memory interface circuit, which increments the address to the memory location of the current time interval, so that each cycle also moves to the next time interval in the memory 806.

[0143] The clock circuit 810 can be used to generate the periodic signal 560 based on inputs that define an excitation and measurement for an optical measurement system. For example, an excitation input 814 can correspond to Figure 3 The excitation input 814 can reset the address of the memory 806 to a start memory location that corresponds to a first time interval of the histogram 818. The excitation input 814 can also cause the clock circuit 810 to start generating the periodic signal 560 for the ALU 804 and / or the periodic signal 808 that increments the address of the memory 806. In addition, the clock circuit 810 can receive a measurement input 812 that defines the start / end of a measurement. The measurement can include a plurality of excitations that incrementally build the histogram 818. The measurement signal 812 can be used to reset the values in the memory 806 so that the histogram can start over for each new measurement.

[0144] The memory 806 can include a plurality of registers that accumulate photon counts from a photodetector. By accumulating photon counts in respective registers that correspond to time intervals, the registers in the memory 806 can store photon counts based on the time of arrival of the photons. For example, photons that arrive in a first time interval can be stored in a first register in the memory 806, photons that arrive in a second time interval can be stored in a second register in the memory 806, and so on. Each "excitation" can include a pass through each of the registers in the memory 806 that correspond to time intervals for the photosensor. The excitation signal 814 can be referred to as an "enable" signal for the plurality of registers in the memory 806, because the excitation signal 814 enables the registers in the memory 806 to store results from the ALU 804 during the current excitation.

[0145] Periodic signal 560 can be generated so that it is configured to capture the set of signals 816 as they are provided asynchronously from light sensor 802. For example, threshold circuit 540 can be configured to hold the output signal high for a predetermined time interval. Periodic signal 560 can be timed so that it has a period that is less than or equal to the hold time of threshold circuit 540. Alternatively, the period of periodic signal 560 can be a percentage of the hold time of threshold circuit 540, such as 90%, 80%, 75%, 70%, 50%, 110%, 120%, 125%, 150%, 200%, and so on. Some embodiments can use a rising edge detection circuit as described in Figure 5

[0146] Figure 9 Timing associated with different firings in a measurement is shown according to some embodiments. The vertical axis represents the total number of light detector signals measured for a single light sensor. For example, the vertical axis can represent total photon counts. The horizontal axis represents time. When a new optical measurement is started, as indicated by measurement signal 812, a first firing can start with a firing signal 814-1 as a start signal. Periodic signal 506 illustrates how each clocking of ALU 804 corresponds to a single time interval and a corresponding memory location in memory 806.

[0147] With each subsequent firing, a histogram can be built in memory 806 as described in Figure 9 Figure 9 Whenever a firing input 814 is received, the addressing of memory 806 can be reset so that a new total signal count can be added to an existing signal count. In the example of

[0148] ​​In some embodiments, the timing of the measurement signal 812, the trigger signal 814, and the periodic signal 506 of the timing ALU 804 can all be coordinated and generated relative to one another. Thus, the timing of the ALU can be triggered by and depend on the start signal of each trigger, in addition, the period of the periodic signal 506 can define the length of each time bin associated with each memory location in the histogram.

[0149] Figure 8 The data path illustrated in FIG. 8 is primarily configured to build a histogram 818 over a number of triggers. However, in some embodiments, the histogram can only be populated during a trigger. Any photons received between triggers will not be binned in time by the ALU 804 and will not be stored in the memory 806. In addition, the contents of the memory 806 can be reset after each measurement is completed and before a subsequent measurement is started. Thus, photons received before and / or after the current measurement can not be saved or readily available. Furthermore, the total count of all photons received in the histogram is not readily available in the histogram data path, and when the histogram is disabled, the total count of all photons received during a measurement period is not available if there is a non-zero interval between triggers. In order to record received photons in a continuous manner that depends on the trigger / measurement timing, a second parallel data path can be used in addition to the histogram data path illustrated in FIG. 8. Figure 8

[0150] V. Identifying Earlier Reflections from the System Enclosure

[0151] The above examples have illustrated a single pulse or multiple pulses as part of a pulse train transmitted from a light source, reflected off an object in the surrounding environment, and then detected by a light sensor. After a number of repeated triggers in a measurement, these reflections are recorded in a histogram memory and form "peaks" in the histogram. The location of these peaks can then be used to determine the distance between the optical measurement system and the object in the surrounding environment. However, these examples are not meant to be limiting. In addition to the expected peaks caused by a pulse train reflected off an object of interest in the surrounding environment, the histogram memory can also contain other peaks caused by more immediate, unintended reflections of the pulse train, as well as peaks that are not necessarily produced by a pulse train at all. For example, additional peaks can correspond to external light sources, sunlight, reflections, or other environmental light phenomena in the surrounding environment. In these cases, instead of detecting a single peak, the optical measurement system can detect multiple peaks that can thus be present in the histogram memory. In addition to extraneous peaks caused by external light sources, the initial peak can be produced by a reflection of the pulse train off the internal enclosure of the optical measurement system.

[0152] Figure 10 ​A representation of the histogram memory containing peaks 1002 caused by reflections off the housing of the optical measurement system is illustrated. When the light source generates one or more pulses for the emitted pulse train, the pulses can pass through a window or other transmissive material in the housing of the optical measurement system. However, some of the light emitted by the light source can reflect off the window of the housing back to the photosensor rather than passing completely through the window. In addition, some of the light emitted by the light source can reflect off the interior surface of the housing itself. For example, some stray light can reflect off the portion of the housing near the transmissive window or from other internal materials of the optical measurement system. Just as reflection pulses from the object of interest in the surrounding environment are detected and stored, portions of the light reflected off the housing of the optical measurement system can be received by the photosensor and stored in the histogram memory.

[0153] In Figure 10 the peaks 1002 caused by earlier reflections off the housing of the optical measurement system from the emitted pulse train can be stored in an initial set of registers in the histogram memory. Because the reflections occur shortly after the pulse train is emitted from the light source of the optical measurement system, the photon counts from these earlier reflections can be stored in the first registers in the histogram memory for measurement. For example, Figure 10 A representation of the histogram memory containing peaks 1002 caused by reflections off the housing of the optical measurement system is illustrated. When the light source generates one or more pulses for the emitted pulse train, the pulses can pass through a window or other transmissive material in the housing of the optical measurement system. However, some of the light emitted by the light source can reflect off the window of the housing back to the photosensor rather than passing completely through the window. In addition, some of the light emitted by the light source can reflect off the interior surface of the housing itself. For example, some stray light can reflect off the portion of the housing near the transmissive window or from other internal materials of the optical measurement system. Just as reflection pulses from the object of interest in the surrounding environment are detected and stored, portions of the light reflected off the housing of the optical measurement system can be received by the photosensor and stored in the histogram memory.

[0154] As the photon counts are received for the remainder of the optical measurement, additional peaks caused by reflections of photons from the object of interest in the surrounding environment can also be detected. For example, after reflecting off an external object within the range of the optical measurement system, Figure 10The peaks 1004 in the histogram can be received later in the measurement. Typically, these objects of interest in the surrounding environment will be far enough away from the optical measurement system that the resulting peaks 1004 can be easily distinguished from the peaks 1002 that reflect off the system housing. Some embodiments can ignore or remove the peaks 1002 from the histogram memory and / or from any calculations performed on the histogram memory to calculate a distance to the external object corresponding to the peaks 1004. For example, some embodiments can mask an initial set of histogram registers that can contain large peaks 1002 from housing reflections. Some embodiments can insert a delay between when the pulse train is emitted by the light source and when the light sensor begins accumulating photon counts into the histogram memory to avoid recording peaks 1002 from housing reflections. Because the peaks 1002 occur so early in the measurement compared to when the peaks 1004 are received, the peaks 1002 can generally be ignored when using the peaks 1004 to perform distance calculations.

[0155] However, some embodiments can receive and store the peaks 1002 caused by housing reflections in the histogram memory and then proceed to use the peaks to improve the close range performance of the optical measurement system. These improvements can include calibrating the optical measurement system, characterizing the opacity of the window of the optical measurement system, detecting a blockage of the optical measurement system, improving the close range performance of the optical measurement system, and so on. Each of these various improvements that can be achieved by first identifying the peaks 1002 caused by housing reflections are described in more detail below.

[0156] Figure 11 A flowchart is illustrated for a method for detecting peaks caused by earlier reflections off the system housing using an optical measurement system. This method can be used to transmit / receive a pulse and identify a particular location in the histogram memory that corresponds to a peak from a housing reflection. The following sections in this disclosure describe different ways in which the identified peaks can be used to improve the performance of the optical measurement system. The following method describes the process for a single light sensor in an optical measurement system. However, as described above, the optical measurement system can include many light sensors and this method can be performed for each light sensor in the optical measurement system.

[0157] At step 1102, the method can include transmitting one or more pulse trains from a light source for one or more first time intervals as part of an optical measurement. Each of the one or more first time intervals can represent a "shot" that is repeated multiple times in the measurement. Each of the first time intervals can include one or more pulse trains encoded and transmitted by the light source such that the pulse trains can be considered to reflect off objects in the surrounding environment. Each of the time intervals can be subdivided into a plurality of time bins such that each of the time bins represents a bin in a histogram of photon counts received during the optical measurement. The above is described with respect to Figure 9An example of how a single measurement can include multiple excitations subdivided into time intervals of aggregated photon counts is described.

[0158] At step 1104, the method can also include detecting photons from the one or more pulse trains using a photosensor. As described in detail above, the photosensor can include a plurality of photodetectors, such as a plurality of SPADs. The photosensor can receive reflected light as well as environmental background noise received from the surrounding environment. The reflected light received by the photosensor can include light reflected off of objects of interest in the surrounding environment. For example, these objects of interest can be at least 30 cm away from the photosensor and can represent surrounding vehicles, buildings, pedestrians, and / or any other objects that can be encountered near the optical measurement system during use. These objects of interest can be distinguished from objects that are part of the optical measurement system itself, such as the housing. The reflected light received by the photosensor can also include light reflected off of the housing or other parts of the optical measurement system. These reflections can include primary reflections that are directly off of the window or housing of the optical measurement system, as well as secondary reflections that are reflected off of the interior of the optical measurement system. The photosensor can also be coupled to threshold detection circuitry and / or to arithmetic logic circuitry that accumulates photon counts. This combination can be referred to as a “pixel” above. Figure 5 An example of how photon counts can be received from a photosensor and counted using threshold circuitry and a pixel counter (e.g., arithmetic logic circuitry) is illustrated.

[0159] At step 1106, the method can also include accumulating photon counts from the photosensor into a plurality of registers to represent a histogram of photon counts received during a current measurement. These photon counts can be accumulated in a plurality of registers in a memory block corresponding to the photosensor. The plurality of registers can be implemented using registers in SRAM of a histogram data path, as described above in Figures 8-9 Each of the time intervals can be subdivided into a plurality of first time intervals for the histogram. A corresponding time interval in each of the one or more time intervals can be accumulated in a single register of the plurality of registers. Each of the time intervals can represent an excitation, and the one or more time intervals together can represent a measurement for the optical measurement system. Each of the time intervals can be delimited by a start signal or excitation signal for a first register in the histogram that the memory resets back to. Received photons that are reflected off of the housing of the optical measurement system can be stored in the histogram memory in the same way as other received photons that are reflected off of surrounding objects outside of the optical measurement system.

[0160] At step 1108, the method can further include identifying a peak in the histogram representing photons reflected off the system housing. Depending on the particular embodiment, detecting this peak can be performed using several different techniques. In some embodiments, a pass through the histogram memory can be made to identify the peak occurring first in time. The system can sequentially access registers in a plurality of registers, beginning at the start of the optical measurement to identify the peak occurring first in time. The center of the peak can be identified by identifying values in the histogram that have smaller values in the time bins on either side. In Figure 10 In the example of FIG. 9, the registers representing the fourth and fifth time bins can be identified as the peak because the time bin values in the registers on either side of the fourth and fifth registers have significantly smaller values stored therein. In some embodiments, the initial peak can be identified by identifying registers in a plurality of registers during the highest number of photon counts. These embodiments can assume that the initial peak can have a higher intensity than other peaks. Some embodiments can also identify the initial peak by identifying registers in a plurality of registers that correspond to a distance between the light source and the housing of the optical measurement system. For example, when this distance is known, the distance per time bin can be used to identify registers that can receive peaks reflected from the housing. Some embodiments can identify the initial peak by identifying a predetermined number of registers in a plurality of registers that appear first. For example, based on an initial calibration and a known distance between the light sensor and the housing, some embodiments can identify the first 15 time bins as storing the initial peak.

[0161] Identifying a peak in the histogram can involve locating a window of time bins occurring near the center of the peak. This process can involve expanding outward from the peak location to determine a range of registers in the histogram memory that contain the entire peak. As Figure 10 As explained in FIG. 8, photons caused by reflections off the system housing can be received in multiple time bins rather than isolated to a single time bin. Accordingly, some embodiments can identify surrounding time bins that should also be designated as representing the reflected peak from the system housing. For example, some embodiments can identify a predetermined number of time bins around a maximum value that can be designated as the peak. For example, a predetermined number of time bins, such as 3 time bins, 5 time bins, 9 time bins, 15 time bins, 17 time bins, etc. can identify the maximum value and center around the maximum value to represent the entire peak. Some embodiments can identify surrounding time bins that have values within a percentage of the maximum value in the peak register. This can result in a variable number of time bins that can be used to represent the peak depending on the width of the peak. For example, time bins around the maximum value can be included in the peak when their values are within 25% of the maximum value.

[0162] Instead of detecting the variable peak position for each measurement, some embodiments can assume a known position of the optical measurement enclosure and designate a particular window of registers that can include values from an earlier reflection off the system enclosure. In Figure 10 In the example of FIG. 6, the system can designate registers corresponding to the second through sixth time intervals as registers that store peaks of reflections off the system enclosure. Physical measurements can be made to determine the distance between the light source and the enclosure and the distance between the enclosure and the light sensor. This distance can then be used in conjunction with the speed of light to determine where reflections off the system enclosure are likely to appear in the histogram memory.

[0163] It should be appreciated that according to various embodiments, Figure 11 The particular steps illustrated in FIG. 6 provide a particular method for identifying peaks representing reflections off the system enclosure. Other orders of steps can also be executed according to alternative embodiments. For example, alternative embodiments of the application can execute the steps outlined above in a different order. Moreover, Figure 11 The individual steps illustrated in FIG. 6 can include a plurality of sub-steps that can be executed in various orders suitable for individual steps. Additionally, additional steps can be added or removed depending on the particular applications. One of ordinary skill in the art would recognize many variations, modifications, and alternatives.

[0164] VI. Using enclosure reflections to improve measurements

[0165] After identifying the location of pulses representing reflections off the system enclosure, the location and / or peaks of the reflections can be used to improve or characterize the optical measurement system in a number of different ways. This can include calibrating additional distance measurements made by the optical measurement system for known distances, characterizing the transparency of a window on the optical measurement system, detecting blockages or other obstacles that obstruct the field of view of the optical measurement system, and / or other similar improvements.

[0166] A. Using enclosure reflections for calibration

[0167] Figure 12The contents of a histogram memory that can be used to calibrate distance measurements for an optical measurement system according to some embodiments are illustrated. As described above, the histogram can include a peak 1002 corresponding to reflections off of the housing of the optical measurement system. The histogram can also include a peak 1004 caused by reflections of photons from an object of interest in the surrounding environment. Typically, the center location of the peak 1004 can be identified and used to calculate the distance between the optical measurement system and the object from which the peak 1004 reflected. This distance calculation can use the speed of light and the known time interval for each of the time intervals represented by the histogram register. For example, for a system operating at 500 MHz, the resulting time interval can represent approximately 2 ns of photon counts received. The distance of the peak 1004 can be calculated using the number of time intervals (e.g., 32) multiplied by the time of each time interval (e.g., 2 ns) multiplied by the speed of light.

[0168] However, if any of the above assumed values are not accurate or vary between different optical measurement systems, the distance calculation can not be accurate. For example, the actual time represented by each time interval can not be as precise as predicted based on the assumed clock period. Accordingly, some systems can be adapted to perform a self-calibration process that can be used to accurately determine different constants used by the distance calculation.

[0169] Some embodiments can use the location of the peak 1002 representing the housing reflection to calibrate values for other distance calculations of the optical measurement system. The distance between the light source and the housing of the optical measurement system can be precisely known based on the manufacturing process of the optical measurement system. Similarly, the distance between the housing and the light sensor of the system can also be precisely known. These distances can be used in conjunction with the peak 1002 to determine the precise values of constants used in the distance calculation, such as the time represented by each time interval and / or any offset in the system. These constants can be part of a transformation function that receives the time of a peak and provides an accurate distance measurement. Such a transformation can be a constant (e.g., uniform delay) linear transformation (e.g., later time intervals offset by an amount that is proportional to the number of time intervals more / less than earlier time intervals) or a non-linear transformation.

[0170] In Figure 12In the example of FIG. 10, the position of the peak 1002 can be determined using any of the techniques described above. The position of the peak 1002 can be used to determine the number of time intervals 1202 between the start of the measurement and the center of the peak from the enclosure. The known distance between the light source, the enclosure, and the light sensor described above can be divided by the speed of light to determine the total time between when the light is emitted from the light source and when the peak 1002 appears in the histogram memory. This value can then be divided by the number of time intervals 1202 in order to calculate the precise amount of time represented by each time interval. It should be noted that this procedure assumes that the histogram memory begins accumulating photon counts in the register when the light is emitted from the light source. If this is not the case, then any delay between the emission of the light from the light source and the start of the accumulation period by the histogram memory can be subtracted from the time estimate.

[0171] The precise value of the constant, such as the time represented by each time interval, can then be used to calculate the distance of other peaks in the measurement. In the example of FIG. 10, the distance between the optical measurement system and the object represented by the peak 1004 can be calculated as described above, but instead of using the assumed time value from the clock period, the system can actually utilize the calibration value calculated using the peak 1002 to calculate the distance to the object. Figure 12

[0172] Using the calibration value from the peak reflected from the enclosure of the optical measurement system can thus be used to generate more accurate distance measurements. These techniques can overcome problems caused by changes over time and sensor drift. These techniques can also be used to detect movement of the system enclosure relative to the light source and / or the light sensor. For example, if the position of the peak 1002 changes over the lifetime of the optical measurement system, this can indicate movement of the enclosure relative to the rest of the system.

[0173] B. Characterizing Window Transparency

[0174] In addition to calibrating values to be used for distance calculations, identifying the position of the reflection out of the system enclosure can also be used to characterize various aspects of the optical measurement system during operation. In general, the peak reflected out of the enclosure of the system can be larger than the peak received from an object of interest in the surrounding environment. The size of this initial peak can be due to the proximity of the enclosure to the light sensor. Because the light sensor is so close to the reflection, the intensity of the reflected light can be relatively large. However, assuming that the size of the reflection peak is within the saturation limit of the histogram memory, certain measurements can be made based on the magnitude of the peak reflected out of the enclosure to characterize portions of the enclosure itself.

[0175] Figure 13A ​A portion of the histogram memory that receives an initial peak 1302 corresponding to reflections off of the system housing is illustrated in accordance with some embodiments. The peak 1302 can have an initial magnitude 1308. This magnitude can be indicative of the transparency of a window on the housing of the optical measurement system. As described above, the housing can include a window through which light can be transmitted from a light source and reflected light can be received by a light sensor through the window. Assuming the transparency of the rest of the housing is substantially non-transparent, the magnitude 1308 of the peak 1302 can be used to characterize the transparency of the window itself. For example, a baseline transparency can be characterized by recording the average magnitude 1308 of the housing reflections during an initial portion of the life cycle of the optical measurement system. This value can be stored over time and used to detect changes in the transparency of the window.

[0176] Many practical applications of optical measurement systems can include use in open environments, where the optical measurement system can be exposed to contamination, rain, snow, and / or other environmental elements that can damage, obscure, and / or otherwise affect the transparency of a window on the housing of the optical measurement system. For example, when an optical measurement system is installed in an automotive application, the system can be positioned on the roof of the vehicle. The system can also be positioned on an area near the perimeter of the vehicle, such as on the bumper or side of the vehicle. In these locations, the optical measurement system can be subjected to weather elements, rocks and debris, exhaust fumes, or fog, and / or other influences that can affect the transparency of the window. For example, a rock on the road can hit the optical measurement system, scratching or otherwise damaging the window. A humid environment can cause moisture to temporarily accumulate on the window. Rain or snow can accumulate on the outside of the window. Mud, contamination, or dirt on the road can build up on the outside of the window. Many other adverse conditions in the automotive environment can also affect the window of the optical measurement system.

[0177] Each of these different environmental influences can cause the transparency of the window to change. For example, when mud or scratches build up on the window, less light can be transmitted through the window. As a result, more light can be reflected off of the window back to the light sensor. Thus, given that the window is part of the housing of the optical measurement system, the techniques described above with respect to monitoring the magnitude of the peak reflected off of the housing of the optical measurement system over time can also be used to track differences in the transparency of the window over time. It can be assumed that the transparency of the rest of the housing remains constant (e.g., the housing is completely non-transparent). Thus, any changes in the magnitude of the peak reflected off of the housing can be attributed to changes in the transparency of the window itself.

[0178] Figure 13B A change in the magnitude of the peak 1304 reflected off of the housing is illustrated in accordance with some embodiments. The peak 1304 can be recorded over time, and the magnitude of the peak 1304 can be monitored to detect changes in the transparency of the window. For example, the baseline transparency of the window can be characterized by recording the average magnitude of the peak 1304 over a period of time. This value can be stored and used to detect changes in the transparency of the window. For example, if the magnitude of the peak 1304 increases, this can indicate that the transparency of the window has decreased. For example, if the magnitude of the peak 1304 decreases, this can indicate that the transparency of the window has increased. Figure 13Athe same optical measurement system as peak 1302, but peak 1304 can be recorded later in the life cycle of the optical measurement system. For example, peak 1304 can be recorded after the optical measurement system has been used for months in a car application. When the magnitude 1310 of peak 1304 is compared to the magnitude 1308 of peak 1302, Figure 13B illustrates how the magnitude of peak 1304 increases over time. The increase can be attributed to changes in the transmissive properties of the window on the housing of the optical measurement system.

[0179] As described above, the magnitude 1308 of the initial peak 1302 can be recorded as a baseline measurement when the window of the housing is clean. This initial value can be stored over time and used as a baseline for comparison with future peak magnitudes to track the transparency of the window over time. For example, some embodiments can track the difference 1306 between the initial magnitude 1308 and the current magnitude 1310 of peaks 1302, 1304 over time. As the difference 1306 increases, various outputs can be provided by the optical measurement system. For example, some embodiments can provide an output that triggers a warning or message to a user or to the car system indicating that the window is dirty (“please clean the window on your LIDAR system”). Some embodiments can automatically trigger a system on the vehicle to clean the optical measurement system, such as a system that sprays a cleaning agent on the optical measurement system.

[0180] Figure 14 illustrates a portion of the array of light sensors 1402 that can be partially obscured by contamination of the window, according to some embodiments. As described above, these methods for detecting reflected pulses from the housing of the optical measurement system can be performed for each light sensor in the array of light sensors 1402. Thus, each individual light sensor can generate a baseline magnitude of reflected peaks, as well as a current magnitude of reflected peaks. Each individual light sensor can then report the percentage or degree to which the corresponding portion of the window is obscured. The transparency of each light sensor can be characterized relative to one of the light sensors or relative to a baseline level of transparency. In this example, a portion 1404 of the window can be covered in contamination, such as mud. The individual light sensors that transmit / receive light through the portion 1404 of the window can report an increase in the magnitude of the reflected pulses, while the light sensors outside of the portion 1404 of the window can continue to report a steady magnitude of the reflected pulses. This can be used to generate a map or characterization of the window. For example, Figure 14 an image of the window can be generated and provided to a user interface, illustrating the current state of the window. A message can be generated characterizing the window as dirty, partially obscured, or completely blocked.

[0181] In some embodiments, the optical measurement system can turn off light sensors behind a portion 1404 of a dirty or clogged window, or for a rotating system, when the light sensors are behind the portion 1404. Because of the intensity of the reflected pulses, the histogram data path of these light sensors can saturate, such that the magnitude of the pulses reflected off the window has a magnitude higher than the bit limit of the histogram data path. Reflections from this portion 1404 of the window can also be scattered by the obstruction and interfere with the photons received by other nearby light sensors. Because the measurements can be skewed by saturation, scattering, or other effects, the system can take corrective measurements, such as causing these light sensors to no longer accumulate photon counts.

[0182] C. Detecting Window Obstruction

[0183] In addition to detecting a dirty or partially obstructed window, the first detected peak can also be used to determine whether the optical measurement system is obstructed by a foreign object. As described above, many operating environments, such as automotive environments, can provide situations in which the optical measurement system can be subjected to adverse conditions. Some of these environments can cause the optical measurement system to become completely obstructed from the surrounding environment. For example, a plastic bag can blow onto the optical measurement system. A person can place their hand in front of the optical measurement system. The optical measurement system can be intentionally disabled with a piece of tape or other object placed in front of the optical measurement system. In any of these environments, it can be useful to detect when such obstructions occur and where such obstructions can be located.

[0184] Figure 15A An initial peak 1502 detected by the optical measurement system relative to an obstruction threshold 1506 is illustrated in accordance with some embodiments. The initial peak 1502 can be the result of a reflection off the housing of the optical measurement system as described above. Based on this baseline measurement of the magnitude of the peak 1502, an obstruction threshold 1506 can be established that is higher than the magnitude of the peak 1502. The obstruction threshold 1506 can represent a magnitude of the peak 1502 that can indicate an obstruction in front of the optical measurement system. The obstruction threshold 1506 can be set a predetermined distance and / or percentage above the magnitude of the initial peak 1502, such that only when an object is in front of the optical measurement system does it cross the predetermined distance and / or percentage.

[0185] Figure 15BThe initial peak 1504 at the end of the life cycle of the optical measurement system when an obstruction is present is illustrated in accordance with some embodiments. When an object is placed in the front of the optical measurement system, a large portion of the pulse can be reflected off of the blocking object right away. In the case where the blocking object is very close to the optical measurement system, the resulting peak caused by the reflection off of the blocking object can be merged with other reflections off of the housing of the optical measurement system as described above. In this example, the optical measurement system is close enough together to receive the reflection from the housing and the reflection from the blocking object, resulting in a peak 1504 that is substantially higher than the peak 1502 caused by the reflection off of the housing alone.

[0186] When the blocking object is close enough to the optical measurement system such that it prevents a substantial portion of the light emitted by the optical measurement system from reflecting off of other objects in the surrounding environment, the system can determine that an obstruction has occurred. This determination can be made when the magnitude of the initial peak 1504 meets or exceeds an obstruction threshold 1506. As described above, the reflection off of the housing and the blocking object can combine to increase the magnitude of the peak 1504 above the threshold 1506. When this occurs, any of the corrective actions described above can be triggered. For example, some embodiments can cause a warning to be generated for the user and / or the car system. Some embodiments can shut off any of the light sensors in the array that detected the obstruction. Some embodiments can activate other emergency or safety systems that indicate that the optical measurement system is no longer providing an accurate depiction of the objects in the surrounding environment, etc. Figure 15B

[0187] VII. Compensation for Housing Reflections

[0188] The use of the peak generated by the reflection off of the housing of the optical measurement system described above generally improves the performance of the optical measurement system by calibrating the system and compensating for external influences. Some embodiments can improve the performance of the optical measurement system by removing the peak caused by the housing reflection from the histogram memory signal analyzed by the peak detection circuit or by the auxiliary processor instead of or in addition to. These embodiments can remove the initial peak from the housing reflection to improve the close range accuracy of the optical measurement system.

[0189] A. Close Range Detection

[0190] Figure 16A ​A histogram memory with an initial peak 1602 caused by enclosure reflections and a second peak 1604 caused by reflections from objects in the surrounding environment according to some embodiments is illustrated. As described above, when the second peak 1604 is sufficiently removed from the initial peak 1602, distance calculations for the second peak 1604 need not be affected by the presence of the initial peak 1602. For example, when an object is several feet away from the optical measurement system, the peak caused by photons reflected off of the object can occur far enough away from the initial peak 1602 in the histogram memory that the shape of the second peak 1604 is not affected. In general, the mid-range and long-range accuracy of the optical measurement system need not be affected by the presence of the initial peak 1602 from enclosure reflections.

[0191] However, as the object moves closer to the optical measurement system, the residual effects of the initial peak 1602 can begin to affect the shape of the second peak 1604. For example, as photons begin to be received from reflections off of the object, photons scattered off of the internal enclosure of the optical measurement system can still be received by the photosensor. Because the additional photon counts from the reflected photons can continue to be stored in the register, representing the time interval of the second peak 1604, the shape of the second peak 1604 can be skewed. When the shape changes, the peak detection circuit can identify an inaccurate location of the peak. For example, if the beginning portion of the second peak 1604 is increased due to photons reflected off of the enclosure, the center of the second peak 1604 can shift to the left when detected by the peak detection circuit.

[0192] Figure 16B A near-range peak detection problem that is not compensated for reflections off of the enclosure according to some embodiments is illustrated. In this example, an object in the surrounding environment has moved close enough to the optical measurement system that the second peak 1604 caused by reflections off of the object has begun to merge with the initial peak 1602 caused by reflections off of the enclosure. As described above, this has the effect of changing the magnitude and / or shape of the initial peak 1602 and altering or even hiding the presence of the second peak 1604. Figure 16B As illustrated in the middle, this has the effect of changing the magnitude and / or shape of the initial peak 1602 and altering or even hiding the presence of the second peak 1604. Because the magnitude of the second peak 1604 will generally be substantially less than the magnitude of the initial peak 1602, the shape and / or location of the second peak 1604 can be subsumed by the initial peak 1602 into a point that can no longer be correctly recognized as a peak by the optical measurement system.

[0193] The overall effect of this combination of the initial peak 1602 and the second peak 1604 will greatly reduce the near-range accuracy of the optical measurement system. Reflections off of objects several feet away from the optical measurement system can be affected by these enclosure reflections to a point that is in close proximity to the optical measurement system that can create a "dead zone."

[0194] B. Removing Shell Reflection

[0195] To compensate for the effects of reflections off the shell, the optical measurement system can use a baseline measurement for the initial peak as described above. In particular, the optical measurement system can record a baseline shape and / or position of the initial peak as observed over time during a measurement. In addition to using changes in the initial peak to characterize the window of the optical measurement system, the shape and / or position of the baseline measurement of the initial peak can be used to compensate for reflections off the shell for the proximate measurement.

[0196] Figure 17A Illustration Figure 16B In addition, Figure 17A The values in the registers of the baseline peak 1702, which represent the average of previous measurements, are also superimposed. Essentially, the baseline peak 1702 represents the magnitude of photon counts that can be attributed to reflections off the shell and not from a nearby object of interest in the surrounding environment. Because the value of each of these registers in the baseline peak 1702 is known, its value can be removed from the currently measured histogram registers.

[0197] Figure 17B Illustration after the baseline peak 1702 has been subtracted from the registers in the histogram memory Figure 16B The histogram memory of the current measurement. For example, the register values from the baseline peak 1702 can be subtracted from each of the registers in the histogram memory of the current measurement. The resulting contents of the histogram memory show a second peak 1604 caused by a proximate object in the surrounding environment. It should be noted that the shape of the second peak 1604 can be preserved by removing the effects of the initial peak 1602. This allows the proximate performance and accuracy of the optical measurement system to be greatly improved. Instead of creating a "dead zone" around the optical measurement system, proximate objects can be accurately detected even when they are in close proximity to the optical measurement system.

[0198] Figure 18 Illustration of circuitry for removing the effects of reflections off the shell of the system during a proximate measurement according to some embodiments. Figure 18 Similar to Figure 8 which includes an arithmetic logic circuit 804, a histogram memory 806, and various periodic signals that control the timing of the arithmetic logic circuit 804 and the histogram memory 806. This basic circuitry can be used to generate a histogram in the histogram memory 806 in a plurality of firings delimited by a firing or start signal 814 and a measurement signal 812 as described above.

[0199] Figure 18The histogram data path illustrated in the middle also includes a baseline peak 1804 that stores the value of the histogram register for the initial peak caused by reflections off the enclosure as described above. This baseline peak 1804 can be an average peak recorded in a number of previous measurements. The value of each register location can be averaged over a sliding window of previous measurements to eliminate any outliers, blockages, or other influences that can only temporarily affect the accuracy of the baseline peak 1804. Other mathematical operations can be used to combine a number of initial peaks recorded in previous measurements to generate a baseline initial peak, including maintaining a running average, identifying the minimum or maximum peak, using a step average finder, and other similar techniques.

[0200] The baseline peak 1804 can be provided to a subtraction circuit 1802 so that the baseline peak 1804 can be subtracted from the corresponding registers in the histogram memory 806. For example, a memory interface can provide the first nine register values from the histogram memory 806 to the subtraction circuit 1802 and subtract the corresponding values from the baseline peak 1804 from those register values. The register values can then be returned to the histogram memory 806 so that the histogram memory 806 has the effects of reflections off the enclosure removed from the histogram. The subtraction circuit 1802 can be implemented using an arithmetic logic unit (ALU) or digital logic gates that implement a subtraction function.

[0201] In some embodiments, the baseline peak 1804 can additionally or alternatively be provided to a processor 1806. Instead of using the subtraction circuit 1802 to compensate for reflections off the enclosure, the baseline peak 1804 can be passed to the processor 1806 along with the raw and / or filtered data in the histogram memory 806, which can then remove the baseline peak 1804 from the current measurement. The processor 1806 can be on the same integrated circuit as the rest of the optical measurement system, including the histogram memory 806 and the arithmetic logic circuit 804. Alternatively, the processor 1806 can be located on a separate integrated circuit from the integrated circuit that implements the optical measurement system. The processor 1806 can receive the filtered and / or raw data from the histogram memory 1806 and can then use an instruction set to perform the operations of removing the effects of reflections off the system enclosure as described above.

[0202] VIII. Detecting peaks using matched filters

[0203] As described in detail above, an optical measurement system can generate bursts of pulses emitted from a light source and directed into a surrounding environment. Light from these bursts can reflect off of objects in the surrounding environment and be received by corresponding light sensors in the optical measurement system. When a photon count is received by a light sensor, it can be accumulated into a register of a histogram memory (e.g., SRAM) to form a histogram representation of the photons received during an optical measurement. The accumulated photon counts in the histogram memory can represent raw measurement data, which can include reflected photons from the light source as well as photons from background noise in the surrounding environment. Depending on the noise level of the background environment, this raw data in the histogram memory can have a relatively low signal-to-noise ratio (SNR). This level of noise in the histogram can increase the difficulty associated with performing on-chip peak detection. To improve the confidence in the on-chip peak detection circuit, the optical measurement system can apply one or more filters to the raw data in the histogram memory, after which a peak detection algorithm can be performed using the filtered data.

[0204] While filtering the data does improve the SNR of the data in the histogram memory, it can also obscure characteristics of the histogram of reflected photons that can be applicable to calculating distance measurements, performing statistical analysis of the histogram data, detecting edge cases, detecting close-range objects, distinguishing between proximate reflection peaks, curve fitting, etc. The embodiments described below take advantage of the benefits of filtering the data in the histogram memory as well as preserving additional information provided by the raw data received from the light sensors. The filtered data can be used to perform on-chip peak detection to then identify time windows in the unfiltered histogram memory that can include detected peaks. After the time windows are identified using the filtered data, the integrated circuit in which the histogram memory is implemented can transfer time intervals containing the unfiltered histogram data during those time windows to an external processor for distance computation and other analysis of the raw histogram data.

[0205] The following sections first describe how the filtering process can use a matched filter for a single pulse scenario to detect time windows in unfiltered data to be transferred off-chip. Then, more complex examples are presented using bursts of pulses containing multi-pulse codes and more complex matched filters. Some filters can be designed to compress the data of these longer pulse codes without sacrificing the shape information of the raw data.

[0206] A. Remote Peak Detection

[0207] Figure 19The contents of the histogram memory after a single pulse is transmitted and received by the optical measurement system are illustrated in accordance with some embodiments. As described above, the initial time bin in the histogram memory can include photon counts caused by the initial pulse reflecting off the housing and / or window of the optical measurement system. When the light source of the optical measurement system emits photons into the surrounding environment, a portion of these photons can reflect back onto the corresponding photosensor without leaving the housing of the optical measurement system. This early reflection of photons can cause a relatively large peak 1902 stored in the initial time bin of the histogram memory. Some embodiments can process the contents of the histogram memory such that the initial peak 1902 can be distinguished from subsequent peaks reflected off objects in the surrounding environment. For example, some embodiments can ignore peaks above a predetermined intensity threshold that can only be exceeded by the initial peak 1902. Some embodiments can ignore peaks occurring in the initial time bin (e.g., during the first 20 time bins). Some embodiments can average multiple signals to estimate a background level that includes the noise of the initial peak 1902, and this average signal can be subtracted from the measurement signal to effectively remove the initial peak 1902. The initial peak 1902 can thus be identified and distinguished from subsequent peaks representing reflections off objects of interest in the surrounding environment.

[0208] After the initial peak 1902, the histogram memory can later include one or more peaks caused by photon reflections off objects in the surrounding environment. For example, peak 1904 can occur after the initial peak 1902 and can represent photons reflected off an object in the surrounding environment. Note that both peak 1902 and peak 1904 can be produced by photons emitted as part of the same pulse train; however, peak 1902 can be reflected off the housing / window of the optical measurement system, while peak 1904 can be reflected off an object in the surrounding environment. The shape of peaks 1902, 1904 can correspond generally to the shape of the emitted pulse from the optical measurement system. For example, some embodiments can emit a pulse having a square shape. As described in the foregoing, Figure 19 As illustrated in the foregoing, the photon counts received by the histogram memory can have a generally square shape corresponding to the shape of the emitted pulse. Background noise, reflection patterns, and other environmental influences can cause the shape of the received peaks 1902, 1904 to vary slightly from the ideal square shape of the emitted pulse.

[0209] As described above, Figure 19The photon counts in the histogram described herein can contain significant background noise and other effects that can make identifying peaks in the histogram memory difficult. Some embodiments may include on-chip peak detection circuitry that identifies peaks in the histogram memory. Because the total length and amount of data stored in the histogram memory can be relatively large, this peak detection circuitry can identify time windows in the histogram memory that contain the identified peaks. The integrated circuit in which the histogram memory is implemented can then transmit the histogram data from these time windows to an off-chip processor for distance calculation. Transmitting windows of peak data, rather than the entire contents of the histogram memory, greatly reduces the bandwidth of data that needs to be transferred between the integrated circuit and the processor. However, these on-chip peak detection algorithms / circuits are best performed when the raw data in the histogram memory is first filtered to increase the SNR.

[0210] Some embodiments may use a matched filter to increase the SNR of the raw data in the histogram memory. The shape of the matched filter may correspond to the shape of the initial pulse emitted by the optical measurement system. For example, matched filter 1906 may be applied to a system with, for example, a matched filter such as... Figure 19 The raw data is stored in a generally square-shaped histogram memory as described herein. Filter 1906 may include a series of constant values, such as "1" or other values, which may be convolved with the raw, unfiltered histogram data to produce a filtered version of the histogram data. Other filtering shapes may be used, such as patterns with one or more zeros between sets of one or more non-zero values ​​(e.g., 1 and -1). Such patterns may occur when different excitations are assigned different weights, as described in U.S. Patent Publication 2018 / 0259645, which is incorporated herein by reference in its entirety.

[0211] Figure 20 Description of usage according to some embodiments Figure 19 The filter 1906 is a filtered version of the histogram data. Generally, the filter 1906 can act as a low-pass filter to reduce the effect of any background noise in the signal, thereby increasing the SNR. The low-pass filter also has the effect of increasing the width of any reflection peaks in the histogram data. In some embodiments, the width of the matched filter 1906 can be approximately the same as the width of the pulse transmitted from the light source of the optical measurement system. For example, for a pulse approximately four time intervals wide (e.g., 8 ns), the corresponding matched filter 1906 can also be approximately four time intervals wide. This can have the effect of approximately doubling the width of the peaks in the filtered histogram data (e.g., eight intervals wide).

[0212] exist Figure 20 In the diagram, the first peak 2002 can be represented by reflections from the housing / window of the optical measurement system. Figure 19The filtered version of the initial peak 1902. Similarly, the second peak 2004 can represent the reflection caused by the object of interest from the surrounding environment. Figure 19 The filtered version of the second peak 1902 in the image. It should be noted that the low-pass filter operation of matched filter 1906 has altered the shape of peaks 2002 and 2004 by increasing the width of peaks 2002 and 2004 and smoothing the overall shape of peaks 2002 and 2004. This can be used to more clearly distinguish peaks 2002 and 2004 from any noise incidentally stored in the histogram memory.

[0213] While matched filter 1906 makes it easier to identify peaks 2002 and 2004 in the filtered histogram data, this filtering operation also distorts the shape of the histogram data. While this results in a higher SNR and a higher confidence level for peak detection, it also removes important information that could be represented in the unfiltered histogram data. This information is useful to processors performing distance calculations, statistical analyses, and other calculations that can be best performed using the details available in the unfiltered data. While some embodiments may send peaks 2002 and / or 2004 from the filtered histogram data to the processor for distance calculation, some embodiments may use the positions of peaks 2002 and 2004 identified in the filtered histogram data to identify the corresponding peak positions in the unfiltered histogram data, and then send the unfiltered peaks from the unfiltered histogram data to the processor. This process will be described in more detail below.

[0214] B. Proximity Peak Detection

[0215] In addition to losing signal information through the filtering process described above, some embodiments may also have difficulty distinguishing adjacent peaks without using unfiltered histogram data.

[0216] Figure 21A This illustrates an example of two adjacent peaks that converge in time. The first peak 2102 may correspond to the initial pulse reflection from the housing / window of the optical measurement system as described above. The second peak 2104 may correspond to an object near the optical measurement system. Figure 21AThis represents the unfiltered histogram data in the histogram memory after photon counts from these two different reflections. It should be noted that in the unfiltered histogram data, the two peaks 2102 and 2104 are distinguishable from each other. Although they begin to merge, the first peak 2102 can be distinguished from the second peak 2104 using a peak detection algorithm. For example, a peak detection algorithm can scan the entire histogram memory register and identify time interval patterns of increasing and then decreasing threshold amounts. Although peaks 2102 and 2104 are temporally converging and at least partially overlapping, the second peak 2104 can still be distinguished using this type of peak detection algorithm.

[0217] Figure 21B The low-pass filter according to some embodiments can be used for... Figure 21A The impact of unfiltered histogram data. A low-pass filter can be a matched filter, such as... Figure 19 The matched filter 1906 increases the total width of the first pulse 2102 and smooths any sudden changes in the histogram data. In doing so, the first peak 2102 can be mixed with the time interval of the second peak 2104. Figure 21B This explains how the two peaks 2102 and 2104 can be encapsulated into a single peak 2106 so that the two peaks 2102 and 2104 are no longer distinguishable in the filtered histogram data. When the filtered histogram data is analyzed and used to calculate distance measurements, near-range objects in the surrounding environment represented by peak 2104 can be lost in the filtered histogram data. Therefore, the near-range accuracy of the optical measurement system can be affected when filtered histogram data is used without unfiltered histogram data.

[0218] Although Figures 21A-21B The examples above illustrate the problems associated with near-field object detection, but the same principle can be applied to objects that are close to each other at any range. For instance, peaks 2102 and 2104 could represent objects in the surrounding environment that are both quite far from the optical measurement system (e.g., 15 feet, 20 feet, etc.). However, if these objects are extremely close to each other (e.g., 1 foot, 2 feet, etc.), the resulting peaks in the unfiltered histogram data can be relatively clustered together. When the histogram data is filtered, these two peaks can merge into a single peak, which can cause the optical measurement system to fail to distinguish between the two objects in the surrounding environment.

[0219] IX. Use filtered data to provide a window in the unfiltered data.

[0220] To address these and other technical problems, some embodiments can filter the histogram data, but can pass unfiltered data to the processor for distance calculations, statistical analysis, curve interpolation, peak fitting, etc. The filtered data can be used to identify the location of peaks in the histogram; however, instead of passing only the time interval window containing the filtered peak data, these embodiments can instead or additionally use the location(s) identified in the filtered histogram data to identify corresponding location(s) in the unfiltered histogram data. The time interval window can then be sent from the unfiltered histogram data to the processor, and the processor can use the unfiltered histogram data to analyze the peak for distance calculations.

[0221] A. Locating the window in unfiltered data

[0222] Figures 22A-22B It is explained how the filtered data can be used to identify peaks in the unfiltered data according to some embodiments. Figure 22A Curve representation in Figure 20 filtered histogram data of Figure 22B Curve representation in Figure 19 unfiltered histogram data of Figure 22B unfiltered histogram data in Figure 22A filtered histogram data of

[0223] The peak detection circuit / algorithm can first identify a first peak 2002. However, as described above, the peak detection circuit can be configured to exclude the first peak 2002 detected in the unfiltered histogram data under the assumption that the first peak 2002 can correspond to an initial reflection of a light pulse out of the housing / window of the optical measurement system. The peak detection algorithm can continue to scan the unfiltered histogram data until a second peak 2004 is detected. In general, the peak detection circuit / algorithm can identify the location of the maximum value of the peak 2004. For example, the peak detection circuit / algorithm can identify a time interval having a local maximum value, with the peak 2004 centered around the time interval.

[0224] After detecting the center of the peak 2004, the peak detection circuit / algorithm can identify a surrounding time interval that can be considered part of the peak 2004. For example, some embodiments can identify a predetermined number of time intervals (e.g., 17 time intervals) centered around the peak location. Some embodiments can identify time intervals that are within a threshold amount of the maximum value of the peak (e.g., within 50% of the maximum value). Some embodiments can select the number of time intervals centered around the maximum value based on the width of the light pulse emitted by the optical measurement system and the width of the filter. For example, if the emitted light pulse is four time intervals wide and the corresponding matched filter is also four time intervals wide, then the resulting peak in the filtered histogram data can be expected to be at least eight time intervals wide. Thus, the peak detection circuit / algorithm can identify at least eight time intervals (e.g., 10 time intervals, 12 time intervals, etc.) centered around the maximum value of the peak as representing the peak in the unfiltered histogram data.

[0225] Some embodiments can transfer one or more time interval windows off-chip to the processor, including the filtered peak data identified as described above. For example, the initial peak, as well as three additional peaks corresponding to objects in the surrounding environment, can be identified by the algorithm described above. Time interval windows centered around these peaks can be identified, and the filtered data in those time intervals can be sent to the processor for processing. Using unfiltered data can be acceptable in situations where the filtered data can sufficiently capture distance information about the surrounding environment.

[0226] Alternatively or additionally, some embodiments can use the locations of the peaks identified in the filtered data to identify time interval windows in the unfiltered data, and the time interval windows in the unfiltered data can be transferred to the processor for processing. In addition to or instead of the filtered data windows described above, these windows of unfiltered data can be transferred.

[0227] To identify the window of unfiltered data in the unfiltered histogram, the optical measurement system can use the location of the maximum of the peak identified in the filtered histogram data. For example, the location of the maximum of the peak 2004 in the filtered histogram data can be used as the location of the maximum of the peak 1904 in the unfiltered histogram data. After identifying this location in the unfiltered histogram data, the window of time bins of surrounding maxima can be identified using the techniques described above with respect to the corresponding location in the unfiltered data (e.g., a predetermined number of time bins, time bins within a percentage range of the maximum, etc.). As described above, the number of time bins identified in the window of time bins of the filtered histogram data can be based on the width of the emitted light pulse and the width of the applied filter. This can result in a window of time bins that is at least twice the width of the emitted light pulse. However, when identifying the window of time bins in the unfiltered histogram data, a smaller number of time bins can be used. Instead of considering the way in which the low pass filter increases the width of the peak 2004, the width of the unfiltered peak 1904 can be smaller. For example, some embodiments can use a number of time bins that is slightly greater than the width of the emitted light pulse (e.g., 2 time bins, 4 time bins, 6 time bins, 8 time bins, etc. greater than the width of the emitted light pulse).

[0228] This method of using the filtered data to identify the window of time in the unfiltered data preserves the benefits of using the unfiltered data while also preserving the benefits of using the filtered data. The higher SNR and smoother peak curve can allow the on-chip peak detection circuit to accurately identify the peak in the histogram. This allows the chip on which the histogram memory is implemented to only transfer a small subset of the information stored in the histogram memory to the processor for processing. However, by transferring the unfiltered data in these windows of time to the processor, the processor can maintain all of the benefits of using the full, unfiltered data for processing.

[0229] Briefly returning to Figures 21A-21B , Figure 21B The peak 2106 in the unfiltered histogram can be easily identified by the on-chip peak detection circuit, and Figure 21A The two peaks 2102, 2104 in the filtered histogram can be transferred to the processor in a window of time using this method described above. The processor can then use various techniques to analyze the data in the window of unfiltered time to recognize that the window of time contains two peaks 2102, 2104 instead of just a single peak. The processor can also use the unfiltered data to more accurately identify the centers of these peaks 2102, 2104 when calculating the distance calculation.

[0230] B. Circuitry for transferring unfiltered data off-chip

[0231] Figure 23A diagram illustrating circuitry for using filtered data to transfer unfiltered data to a processor for distance calculations according to some embodiments. Figure 23 Similar to that described above Figure 8 where it includes an arithmetic logic circuit 804 that receives signals 816 from light detectors in a photosensor and aggregates those signals into photon counts that are aggregated in a histogram memory 806 over multiple shots in an optical measurement. The values stored in the histogram memory 806 can represent the unfiltered histogram data described above.

[0232] Figure 23 Also included is a filter 2302 that can be applied to the photon counts in the histogram memory 806. The filter 2302 can include a matched filter with a shape that corresponds to the shape of the emitted light pulses from the optical measurement system. The filter 2302 can also operate as a low pass filter to smooth the overall shape of the histogram data. In some embodiments, the filter 2302 can be applied incrementally to time bins in the histogram memory 806. For example, the filter 2302 can comprise a sequence of values (e.g., 1 1 1 1) that can be convolved with the time bins in the histogram memory 806. In some embodiments, the filtered values can be transferred to a peak detection circuit 2308 as they are generated by the filter 2302. The peak detection circuit 2308 can then evaluate the filtered values as they are received to detect peaks when a peak occurs in the histogram memory 806. As described above, the peak detection circuit 2308 can operate by identifying peaks in the filtered data that are characterized by an increase in value followed by a decrease in value (and vice versa). The peak resolution circuit 2308 can record the location of the local maxima identified as the filter 2302 passes over the unfiltered histogram memory 806. This approach allows the filtered histogram data to be calculated during a single pass without the need for the entire filtered histogram data to be stored in a separate memory. In the case where the filtered histogram data is transferred to a processor 2314 for processing, the peak detection circuit 2308 can store the filtered values around the detected peaks as a time bin window as described above.

[0233] In some embodiments, the output of filter 2302 can be stored in a separate buffer 2306. As described above, in embodiments where peak detection circuit 2308 operates on the direct output of filter 2302, separate buffer 2306 can not be necessary. However, embodiments where the output of filter 2302 is stored in separate buffer 2306 can then store both the unfiltered histogram data in histogram memory 806 and the filtered histogram data in buffer 2306. Peak detection circuit 2308 can then make passes through buffer 2306 to detect peaks in the filtered histogram data. Storing the filtered histogram data in buffer 2306 allows peak detection circuit 2308 to make multiple passes through the buffer and utilize iterative techniques for identifying peaks in the filtered histogram. For example, each pass through buffer 2306 can detect a maximum value peak that is different than the maximum value detected during a previous iteration of the peak detection algorithm.

[0234] In some embodiments, Figure 23 The histogram data path depicted in FIG. 23 can operate as a multi-stage pipeline. For example, after a current measurement is complete, first filter 2302 can fill buffer 2306 with values from histogram memory 806. During a subsequent measurement, peak detection circuit 2308 can operate on values stored in buffer 2306 from a previous measurement. At the same time, histogram memory 806 can be reset and can begin receiving photon counts from the current measurement. In other embodiments, peak detection circuit 2308 can identify peaks and send values from histogram memory 806 before receiving new values from the next measurement.

[0235] After a maximum value in the filtered data is identified by peak detection circuit 2308, one or more time interval windows 2310 can be identified in the unfiltered data in histogram memory 806. These windows 2310 can include a number of time intervals around the maximum value of the peak identified by peak detection circuit 2308. Windows 2310 can be filled with data from histogram memory 806 as unfiltered values 2312. These unfiltered values 2312 can then be passed to processor 2314 for processing, such as distance calculations, curve fitting, etc. In some embodiments, an interval identifier can be passed to processor 2314 along with or instead of the photon counts in the interval itself.

[0236] Processor 2314 may be implemented on an integrated circuit that is separate from and distinct from the integrated circuit on which histogram memory 806 is implemented. In some embodiments, a first integrated circuit, such as an application-specific integrated circuit (ASIC), may be fabricated including arithmetic logic circuitry 804 and histogram memory 806. The ASIC may also include peak detection circuitry 2308 and optional buffer 2306 (when it is part of the design). Thus, when referring to operations performed "on-chip," these operations may be performed on the first integrated circuit.

[0237] Some embodiments may also include a second integrated circuit including processor 2314. Processor 2314 may include a microcontroller, microprocessor, field-programmable gate array (FPGA) implementing a processor core, ASIC implementing a processor core, and / or the like. Unfiltered value 2312 can be transmitted between the first integrated circuit and the second integrated circuit including processor 2314 via printed leads on a circuit board.

[0238] exist Figure 23 In this example, the unfiltered value 2312 from the histogram memory 806 is only shown as being transmitted to the processor 2314. However, other embodiments may also transmit filtered data to the processor 2314. In some embodiments, filtered data need not be transmitted to the processor 2314 because the processor 2314 can actually re-execute the filter 2302 on the unfiltered data to achieve a similar result. In these embodiments, reducing the bandwidth of information transmitted between the first integrated circuit and the second integrated circuit may be more important than ensuring that the number of processing operations performed by the processor 2314 is minimized.

[0239] X. Filtering of multipulse codes

[0240] The above embodiments use a single pulse code as filtered data as a simplified example of how it can be used to identify unfiltered data and transmit it to a processor. However, the techniques described above are equally applicable to more complex multi-pulse codes. Multi-pulse coding can consist of a burst of pulses or multiple pulses transmitted as part of a single excitation during measurement. These pulse codes can be replicated during each excitation phase of the measurement. Using multi-pulse codes provides a more distinct pattern for detection purposes, and this minimizes false positives and other spurious peaks that can be caused by ambient noise rather than photons reflected from the object of interest in the surrounding environment. The following examples illustrate how these techniques can be used with multi-pulse codes to transmit unfiltered data to a processor.

[0241] A. Low-pass summing filter

[0242] Figure 24A portion of a histogram memory after receiving reflected photons from a multi-pulse code is illustrated in accordance with some embodiments. The multi-pulse code can be significantly larger than a single pulse code, and thus the filtering operation can be even more important to correctly identify the peaks in the data set that characterize the reflection pattern from an object in the surrounding environment. Figure 24 A multi-pulse code is illustrated having the following format:

[0243] 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 -1 -1 -1 -1 0 0 0 0 1 1 1 1 0 0 0 0

[0244] This particular pulse code can be composed of multiple firings from a light source of an optical measurement system during each firing in a measurement. For example, a first firing can include pulses having Figure 24 positive "1" peaks in. When those peaks are received by the light sensor, they can be given a positive weight and added to the histogram memory time interval. A second firing can include pulses having Figure 24 negative "-1" peaks illustrated in. When this peak is received by the light sensor, it can be given a negative weight and added to the histogram memory time interval. The combination of these two firing types can combine to generate positive and negative peaks in the histogram memory as illustrated in Figure 24 . Because of the length of the overall pulse code, the time interval window 2402 in the histogram memory that includes the entire pulse code can be significantly larger than the time interval window described above. For example, the multi-pulse code can occupy approximately 200 time intervals in the histogram memory.

[0245] When the multi-pulse code is received as reflected photons in the histogram memory, this can be represented as positive / negative peaks in the photon counts. In the example of Figure 24 , the multi-pulse code can cause peaks 2404, 2406, 2408, 2409 in the histogram memory. The histogram memory can then be processed with a matched filter 2416 that is based on the shape of the multi-pulse code transmitted from the light source of the optical measurement system.

[0246] Figure 25 A filtered version of the peaks received from the multi-pulse code is illustrated in accordance with some embodiments. When the matched filter 2416 is convolved with the peaks 2404, 2406, 2408, 2409 in the histogram memory, the resulting filtered version of the histogram data can include a large peak 2504 instead of multiple individual peaks. A time interval window 2502 around the filtered peak in the filtered data can be specified, and this peak 2504 can be identified by the peak detection circuit as described above.

[0247] Using the techniques described above, the locations of the peaks 2504 in the filtered data can be used to identify a time interval window 2402 in the unfiltered data Figure 24 The unfiltered peaks 2404, 2406, 2408, 2409 can then be passed to the processor for processing in the time interval window 2402. As described above with respect to other embodiments, in addition to the unfiltered data, Figure 23 the filtered data in the filter 2406 can also be passed to the processor.

[0248] B. Compressed Filter

[0249] Figure 26 An example of a filter using only a single binary indicator is illustrated in accordance with some embodiments. Figure 24 The matched filter 2416 in the filter 2406 uses the same binary indicator for each value in the filter. For example, the sequence "1 1 1 1" is used in the filter 2416 to correspond to the first peak in the multi-pulse encoding. However, using this uniform sequence of values in the filter 2416 has a low-pass effect on the filtered data. As described above, this changes the shape of the filtered data such that many of the characteristics of the unfiltered data are lost in the filtered data.

[0250] In the example of the filter 2406, Figure 26 The reflected peaks 2604, 2606, 2608, 2609 in the unfiltered histogram data can not have the same precise square shape as the multi-pulse encoding transmitted from the light source of the optical measurement system. For example, the light detector can exhibit a "pile-up" effect, where the photons received at the beginning of a peak reflection can create an avalanche effect, after which it can take some time for the light detector to reset. This causes the peaks to have Figure 26 the shape illustrated in the example 2600, such that the peaks 2604, 2606, 2608, 2609 have a more sharp or skewed shape rather than the ideal square shape illustrated in the other examples above.

[0251] To preserve this shape in the filtered data, some embodiments can use a single binary indicator that can still sum the peaks 2604, 2606, 2608, 2609 into a single peak without implementing a low-pass filter operation. The filter 2616 can use a single value or indicator at the beginning of each pulse instead of a uniform set of values for the entire duration of each pulse. In the example of the filter 2616, Figure 26 the filter 2616 can have the following values:

[0252] 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 -1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0

[0253] This filter can generate filtered data similar toFigure 25 The peaks of the filtered data, however, will look more like the shape of the peaks 2604, 2606, 2608, 2609 in the unfiltered data. Thus, the filtered data can more accurately preserve the shape of the peaks in the unfiltered data. Figure 26 Some embodiments can then transmit the filtered data that preserves the shape of the peaks to a processor for processing.

[0254] In some embodiments, Figure 24 and Figure 26 The two filters described in FIGS. 26 and 27 can operate in series at the same time. For example, unfiltered data can be processed using filter 2416 and filter 2616 to generate two filtered versions of the data. Since the first portion of each filter can use the same arithmetic operation, the circuit can share between the two filters due to the two filters running in parallel. For example, the first stage of the summing step for each of these filters can be shared. Some embodiments can then use the low-pass filtered version of the histogram data to identify peaks, and then transmit the compressed version of the histogram data that does not lose its shape due to the low-pass filter to a processor.

[0255] It should be noted that the example filters 2416, 2616 described above with respect to the multi-pulse code use a single bit as the designator in the filter taps. However, this is by way of example in this disclosure. The actual values of each filter can include multi-bit values, such as 10-bit values, rather than a single bit. As described above, these values can be convolved and computed as a sliding window to detect resulting peaks in the filtered data.

[0256] XI. Methods for providing filtered and unfiltered data

[0257] Figure 27 A flowchart of a method for analyzing filtered and unfiltered data in an optical measurement system is described in accordance with some embodiments. The following method describes a process for a single light sensor in an optical measurement system. However, as described above, the optical measurement system can include many light sensors or “pixels,” and this method can be performed for each light sensor in the optical measurement system.

[0258] At step 2702, the method can include transmitting one or more pulse trains from a light source for one or more first time intervals as part of an optical measurement. Each of the one or more first time intervals can represent an “excitation” that is repeated multiple times in the measurement. Each of the first time intervals can include one or more pulse trains encoded and transmitted by the light source such that the pulse trains can be considered to reflect off of an object in the surrounding environment. Each of the time intervals can be subdivided into a plurality of time bins such that each of the time bins represents a bin in a histogram of photon counts received during the optical measurement. The above is described with respect to FIGS. 25 and 26.Figure 9 An example of how a single measurement can include multiple excitations subdivided into time intervals of aggregated photon counts is described.

[0259] At step 2704, the method can also include detecting photons from the one or more pulse trains using a photosensor. As described in detail above, the photosensor can include multiple photodetectors, such as multiple SPADs. The photosensor can receive reflected light as well as environmental background noise received from the surrounding environment. The reflected light received by the photosensor can include light reflected off of objects of interest in the surrounding environment. For example, these objects of interest can be at least 30 cm away from the photosensor and can represent surrounding vehicles, buildings, pedestrians, and / or any other objects that can be encountered near the optical measurement system during use. These objects of interest can be distinguished from objects that are part of the optical measurement system itself, such as the housing. In some embodiments, the reflected light received by the photosensor can also include light reflected off of the housing or other parts of the optical measurement system. These reflections can include primary reflections that are directly off of the window or housing of the optical measurement system, as well as secondary reflections that are reflected off of objects near the interior of the optical measurement system. The photosensor can also be coupled to threshold detection circuitry and / or to arithmetic logic circuitry that accumulates photon counts. This combination can be referred to as a “pixel” above. Figure 5 An example of how photon counts can be received from a photosensor and counted using threshold circuitry and a pixel counter (e.g., arithmetic logic circuitry) is illustrated.

[0260] At step 2706, the method can also include accumulating photon counts from the photosensor into a plurality of registers to represent a histogram of photon counts received during a current measurement. These photon counts can be accumulated in a plurality of registers in a block of memory corresponding to the photosensor. The plurality of registers can be implemented using registers in SRAM of a histogram data path, as described above in Figures 8-9 Each of the time intervals can be subdivided into a plurality of first time intervals for the histogram. A corresponding time interval in each of the one or more time intervals can be accumulated in a single register of the plurality of registers. Each of the time intervals can represent an excitation, and the one or more time intervals together can represent a measurement for the optical measurement system. Each of the time intervals can be delimited by a start signal or excitation signal for a first register in the histogram that the memory resets back to. Received photons that are reflected off of the housing of the optical measurement system can be stored in the histogram memory in the same way as other received photons that are reflected off of surrounding objects outside of the optical measurement system.

[0261] At step 2708, the method can further include filtering the histograms in the plurality of registers to provide a filtered histogram of photon counts from the plurality of registers. The filter can include a matched filter based on the shape of one or more pulse trains transmitted from the optical measurement system. The filter can be convolved with the unfiltered histograms in the plurality of registers and can be stored in a separate buffer. The filter can comprise a square pulse of repeating non-zero values. The filter can also include a single non-zero value followed by a plurality of approximately zero values. The filter can operate as described above with respect to Figure 19 , Figure 24 and / or Figure 26 .

[0262] At step 2710, the method can also include detecting the location of a peak in the filtered histogram. Depending on the particular embodiment, detecting this peak can be performed using a number of different techniques. In some embodiments, a pass can be made through the filtered histogram to identify peaks. The system can sequentially access values in the filtered histogram, starting at the beginning of the measurement to identify peaks. A peak can be identified by identifying an increasing value in the filtered histogram followed by a decreasing value. Similarly, the center of a peak can be identified by identifying values in the histogram that have smaller values in the time intervals on either side.

[0263] At step 2712, the method can also include identifying locations in the plurality of registers that store and unfiltered representation of the peak. These locations can be identified using the location of the peak in the filtered histogram. Thus, the filtered histogram data can be used to identify the location of the peak in the unfiltered histogram data. Identifying the peak in the unfiltered histogram can involve locating a window of time intervals that occur near the center of the peak. This process can involve expanding outward from the peak location to determine a range of registers in the histogram memory that contain the entire peak. For example, some embodiments can identify a predetermined number of time intervals near the location of the maximum value in the unfiltered histogram, which can be designated as the peak. For example, a predetermined number of time intervals, such as 3 time intervals, 5 time intervals, 9 time intervals, 15 time intervals, 17 time intervals, etc. can identify the maximum value and center around the maximum value to represent the entire peak. Some embodiments can identify surrounding time intervals that have values within a percentage of the maximum value in the peak registers. This can result in a variable number of time intervals that can be used to represent the peak depending on the width of the peak. For example, time intervals around the maximum value can be included in the peak when their values are within 25% of the maximum value. This process of identifying a representation of the peak in the unfiltered histogram is described above with respect to Figures 22A-22B .

[0264] At step 2714, the method can also include sending the unfiltered representation of the peak to a processor to calculate a distance to the object. The distance to the object can represent a distance between the optical measurement system and the object in the surrounding environment. This calculation can be performed using the unfiltered representation of the peak sent to the processor. The processor can be physically separate and distinct from the integrated circuit on which the histogram memory is implemented. Thus, the histogram memory can represent a first integrated circuit, and the processor can represent a second integrated circuit in communication with one another over a printed circuit board. The above is described with respect to Figure 23 An example of this circuit is described.

[0265] It should be appreciated, Figure 27 The specific steps illustrated in FIG. 27 provide particular methods of using filtered histogram data to identify the location of a peak in unfiltered histogram data and transferring unfiltered data to a separate processor according to various embodiments. Other sequences of steps can also be performed according to alternative embodiments. For example, alternative embodiments of the present application can perform the steps outlined above in a different order. Moreover, Figure 27 The individual steps illustrated in FIG. 27 can include multiple sub-steps that can be performed in various sequences that are suitable for the individual steps. Additionally, additional steps can be added or removed depending on the particular application. One of ordinary skill in the art would recognize many variations, modifications, and alternatives.

[0266] XII. Combining spatially proximate pixels

[0267] In some environmental conditions, an optical measurement system can have difficulty providing a reliable response above a predetermined confidence level when measuring a distance to some objects. These environmental conditions can include weather phenomena such as rain, thin fog, mist, etc. These conditions can also include objects that are simply too far away from the optical measurement system to generate a reliable measurement based on the number of photons accurately reflected by the object in the environment and received by the light sensors. In either of these scenarios, the optical measurement system can use a detection threshold to prevent false positives from being detected by the system.

[0268] To improve the confidence level in some measurement conditions, some embodiments can use a form of spatial filtering. These embodiments can assume that there is a spatial correlation between photons received by spatially proximate light sensors. For example, light sensors that are physically proximate can receive photons that are reflected off of the same object in the surrounding environment. Even though the response of these individual light sensors falls below the detection threshold, techniques are described below for combining the responses of these proximate light sensors to improve the confidence of the distance calculation such that the combined response is above the detection threshold.

[0269] Figure 28 An example of an object that can provide reflected photons to proximate light sensors according to some embodiments is described. In this example, an object 2800 in the surrounding environment can include a reflective surface 2802 that is oriented to reflect light 2804 from a light source 2806. The light source 2806 can be a light source of the optical measurement system, or it can be a light source of a different system.Figure 28 The other example objects can include other street signs, buildings, vehicles, pedestrians, and so on. Depending on the distance of the object 2800 from the optical measurement system, the object 2800 can be large enough such that photons from multiple light sources are reflected back into the optical measurement system onto corresponding photosensors. Thus, more than one adjacent photosensor can receive photons reflected from the object 2800, and thus multiple photosensors can independently use the photon counts reflected from the object 2800 to calculate a distance to the object 2800. For example, each of these photosensors can be associated with a non-dependent histogram data path and histogram memory as described above with respect to Figure 8

[0270] Figure 28 It is illustrated how multiple photosensors can "see" different portions of the same object 2800. In this example, nine photosensors can have a view of the object 2800. These views are represented by the circles illustrated on the stop sign object 2800. The circles can be referred to herein as "photosensor views" to distinguish what is seen by the photosensor from the photosensor itself. Note that in addition to the nine photosensor views explicitly illustrated in FIG. 28, additional photosensors can also have a view of the object 2800. Figure 28

[0271] Based on environmental conditions or distance to the object 2800, some of the photosensor views can fail to generate a result that meets or exceeds a detection threshold. The detection threshold can be implemented in different ways depending on the embodiment. For example, the detection threshold can be based on receiving at least a threshold number of photon counts reflected from the object 2800. The detection threshold can be based on a calculated distance to the object 2800. The detection threshold can be based on a confidence level of distinguishing a peak in the corresponding histogram from noise in the histogram. For example, a particularly noisy environment with low SNR can fail to generate a result that meets or exceeds the detection threshold. The detection threshold can also be implemented as a peak detection threshold for detecting a peak in the histogram.

[0272] In Figure 28 ​​In the example, light sensor perspectives 2802, 2810, and 2816 may generate results exceeding the detection threshold. These light sensor perspectives 2802, 2810, and 2816 are shaded with a darker color to distinguish them from other light sensor perspectives 2804, 2806, 2808, 2812, 2814, and 2818 of the object 2800 that generate results not exceeding the detection threshold. It should be noted that light sensors with adjacent light sensor perspectives may generate different results. This difference may be due to slightly different environmental conditions for each light sensor and / or slightly different reflective properties at different regions of the object 2800. This situation is more likely to occur at the detection limits of the optical measurement system. For example, when the object 2800 begins to be occluded or moves out of the range of the optical measurement system, the light sensors in the array may begin to "exit" the detection of the object 2800 because their results begin to fall below the detection limit.

[0273] Some embodiments may ignore any light sensor that drops below the detection limit. However, other embodiments may utilize the spatial correlation of neighboring pixels. It is generally assumed that neighboring light sensors are spatially correlated in terms of viewpoint. Figure 28 In these examples, each of the light sensor's viewpoints is spatially correlated because it is reflected from the same object 2800. Because of this spatial correlation, adjacent light sensors can be considered together when evaluating optical measurements. Instead of relying solely on isolated individual light sensors, these embodiments can effectively consider combinations of light sensor responses to characterize and / or generate distance measurements for each light sensor.

[0274] exist Figure 28 In this example, the central optical sensor viewpoint 2810 may be spatially adjacent to eight additional optical sensor viewpoints 2802, 2804, 2806, 2808, 2812, 2814, 2816, and 2818. These additional optical sensor viewpoints can be considered spatially adjacent when they are diagonally adjacent, such as optical sensor viewpoints 2802, 2806, 2814, and 2818, and orthogonally adjacent, such as optical sensor viewpoints 2804, 2808, 2812, and 2816. The characteristics and / or calculations of the distance measurements of the central optical sensor viewpoint 2810 can utilize information derived from the histograms of each of these other spatially adjacent optical sensors.

[0275] A. Combined distance measurements

[0276] Figure 29 Examples illustrating how spatially proximate light sensor viewing angles 2808, 2812, according to some embodiments, can be used to calculate distance measurements for one of the corresponding light sensors 2810. Figure 29 It can be Figure 28simplified view, focusing on how a light sensor can use spatially adjacent perspectives from two other light sensors for clarity. However, as described below, these techniques can be applied to any number of adjacent light sensor perspectives.

[0277] In this example, the light sensor array can include a light sensor 2906 comprising one or more individual light detectors as described in Figure 29 The light sensor 2906 can be positioned physically adjacent to two orthogonally adjacent light sensors 2904, 2908 in the light sensor array 2902 of the optical measurement system. Because the light sensors 2904, 2906, 2908 are physically adjacent in the light sensor array 2902, the corresponding light sensor perspectives 2808, 2810, 2012 can also be considered spatially adjacent in the surrounding environment. However, this is not always the case. Some embodiments described below can use a rotating light sensor array, where the spatial adjacency of light sensor perspectives does not necessarily depend on the physical adjacency of the corresponding light sensors.

[0278] To compute or characterize the distance measurement of the light sensor 2906, some embodiments can rely solely on the photon counts from the light sensor perspective 2810 reflecting off the object 2800. In cases where the resulting histogram and / or distance computation meets or exceeds a detection threshold, relying on the response of the light sensor 2906 itself can be sufficient. However, in some cases, the resulting histogram and / or resulting distance computation of the light sensor 2906 can fall below the detection threshold based on various factors described above (e.g., environmental conditions, distance to the object 2800, ambient noise, etc.). In these situations, the optical measurement system can utilize the responses from the neighboring light sensors 2904, 2908 to improve the confidence of the measurement obtained by the light sensor 2906, such that the resulting peak or computation meets or exceeds the detection threshold.

[0279] Some embodiments can be configured to combine information from the histogram associated with light sensor 2906 with information from the histograms of neighboring light sensors like light sensors 2904, 2908. In some embodiments, this process can share data from neighboring light sensors with each other. For example, the photon counts from the histograms of light sensors 2904, 2908 can be added to the histogram of light sensor 2906. Since the photons all reflect off the same object 2800, this can have the effect of enhancing the reflected signal in the histogram of light sensor 2906. Note that if the light sensor view angles 2808, 2810, 2012 are not spatially adjacent as assumed, then accumulating photons from light sensors 2904, 2908 can not have a negative effect on the histogram of light sensor 2906 since there would not be any reflections at that distance. In effect, this can approximate a spatially matched filter across the histograms of neighboring light sensors. If the two peaks coincide in time, then the light sensor responses can be assumed to be spatially correlated and the two signals can be combined to generate a larger resulting peak in the histogram.

[0280] To combine information from the histograms, the output of the arithmetic logic circuit of each of the neighboring light sensors 2904, 2908 can be sent to the arithmetic logic circuit of light sensor 2906. This can accumulate the photons received by any of light sensors 2904, 2906, 2908 into a single histogram to enhance the return signal. In some embodiments, this accumulation calculation can be extended to use a weighted sum that applies weights to the responses of the various light sensors. Using weights can enable a higher degree of accuracy in determining how many neighboring light sensors will influence the response of another light sensor. For example, since orthogonally adjacent neighboring light sensors can have light sensor view angles that are closer to the center light sensor view angle than diagonally adjacent light sensors, these orthogonally adjacent light sensors can have a higher weight applied to their photon counts when they are added to the histogram of the center light sensor than the diagonally adjacent light sensors. Using weights can also allow the optical measurement system to use light sensor responses that are not directly adjacent. As described above, object 2800 can include more light sensor view angles than the nine illustrated in FIG. 2. These additional light sensor view angles that are not necessarily immediately adjacent to the center light sensor can also be considered. For example, the photon counts from these non-adjacent light sensors can also be added to the histogram of the center light sensor with a weight that is less than the weight assigned to the directly adjacent light sensor responses. Figure 28 As described above, object 2800 can include more light sensor view angles than the nine illustrated in FIG. 2. These additional light sensor view angles that are not necessarily immediately adjacent to the center light sensor can also be considered. For example, the photon counts from these non-adjacent light sensors can also be added to the histogram of the center light sensor with a weight that is less than the weight assigned to the directly adjacent light sensor responses.

[0281] B. Combining Histograms / Peaks

[0282] Figure 30The histograms according to some embodiments can be combined for use with proximity optical sensors. In this example, the histogram of each of the at least nine proximity optical sensors in an array is illustrated using the optical sensor viewpoint on object 2800. It should be noted that the actual histograms may be stored in a memory block representing the histogram memory in the optical measurement system. They are displayed in the optical sensor viewpoint on object 2800 for illustrative purposes only.

[0283] Instead of simply generating a weighted sum of the ambient light sensor responses, some embodiments may use a more complex method that combines information from the histograms of the ambient light sensor views into the histogram of the central light sensor view. In this example, the light sensor view 2810 may be derived from, for example... Figure 30 The diagram illustrates eight orthogonally / diagonally adjacent optical sensor viewpoints. Because these optical sensor viewpoints are spatially adjacent and receive reflected photons from the same object 2800, each of the histograms associated with these viewpoints may contain a peak corresponding to the distance between the object 2800 and the optical measurement system. These peaks are... Figure 30 The peaks are graphically represented in each corresponding optical sensor viewpoint. It should be noted that some peaks are stronger than others, with the peak in the center optical sensor viewpoint 2810 being small enough that it does not pass the detection threshold.

[0284] Some embodiments may combine histogram information from surrounding light sensors to improve detection by the central light sensor. The information from the histograms may be specific peaks in a histogram memory. For example, when peaks are located at the same distance among multiple neighboring light sensors, the system may determine that the corresponding light sensors are spatially correlated and that the associated histograms are combined. Some embodiments may combine the entire histogram for each light sensor, while other embodiments may execute peak detection circuitry / algorithms and then combine only specific peak locations. Therefore, the information combined from the histograms may include the histogram values ​​themselves, portions of the histogram representing peaks, and / or additional values ​​derived from the histograms.

[0285] Instead of simply generating a weighted sum of surrounding histograms, some embodiments can use a Gaussian combination of neighboring histograms combined with the convolution process. For example, each of the histograms spatially adjacent to the central light sensor's viewpoint 2810 can have a Gaussian function applied before combination. Instead of summing the resulting histogram values ​​together, a matched filter can be convolved with them in the same way as the individual histogram convolutions. As described above, this approximates the spatial convolution between spatially adjacent light sensor responses.

[0286] XIII. Optical Sensor Array Configuration

[0287] The examples described above use a rectangular light sensor array pattern with light sensors arranged in a grid layout. However, not all embodiments are limited to rectangular grids. Other embodiments can use different light sensor array patterns that can result in higher light sensor density. Other patterns can also be more suitable for a rotating light sensor array. In any of these layout patterns, the techniques described above can be used to combine data from spatially proximate light sensor perspectives.

[0288] A. Solid State Array Configuration

[0289] Figure 31 An example of a rectangular light sensor layout is illustrated in accordance with some embodiments. This rectangular light sensor layout is similar to the rectangular grid pattern of light sensors illustrated in the above examples. The light sensor array 3100 can be implemented as a solid state array with light sensors 3102, 3104, etc. that are fixed relative to the rest of the optical measurement system, each of which can be scanned by a processor when taking a measurement.

[0290] To use the spatially related algorithms described above with the rectangular light sensor array 3100, it can be assumed that the histograms of proximate light sensors are spatially proximate or spatially related. For example, light sensor 3102 and light sensor 3104 are physically proximate in the light sensor array 3100. Therefore, the corresponding light sensor perspectives 3106, 3108 in the surrounding environment can also be proximate.

[0291] B. Rotating Array Configuration

[0292] Figure 32 A configuration for a rotating optical measurement system is illustrated in accordance with some embodiments. This optical measurement system can include one or more light sensor arrays 3204 that are physically coupled to a rotating element 3200. When the optical measurement system is operating, the rotating element 200 can rotate, and the light sensor array 3204 can continue to scan the surrounding area as it rotates around a central axis.

[0293] In contrast to Figure 31In a non-rotating optical sensor array, the physical proximity of the optical sensors in optical sensor array 3204 does not necessarily imply that the viewing angles of the optical sensors are spatially proximate in the surrounding environment. For example, when the optical measurement system rotates, optical sensor 3202 may perform optical measurements at a certain location. During subsequent optical measurements, optical sensor 3202 will be at a different rotation angle than it was during the previous optical measurements, and may therefore see different objects and different areas in the surrounding environment. Therefore, as optical sensor 3202 rotates, the histogram of a previous measurement for optical sensor 3202 may be "spatially proximate" to the histogram of a subsequent measurement for the same optical sensor 3202 during subsequent time intervals. Therefore, some embodiments may buffer previous histograms on-chip to use as spatially proximate histograms for the methods described above.

[0294] Figure 32 3204 optical sensor array and Figure 31 Another difference between the optical sensor arrays is the pattern in which the optical sensors are arranged in optical sensor array 3204. Instead of arranging them in a rectangular grid, the columns of optical sensors in optical sensor array 3204 are slightly offset from each other. This allows the optical sensors in array 3204 to be positioned closer to each other, thereby increasing the optical sensor density. In addition, by offsetting the optical sensors by less than the width of the optical sensors, the spatial resolution of optical sensor array 3204 can be increased.

[0295] This alternative arrangement of the optical sensors in array 3204 can influence which optical sensors are considered spatially adjacent to other optical sensors in array 3204. Physical proximity does not necessarily imply spatial proximity from the perspective of optical sensors in the surrounding environment. When the optical measurement system rotates, optical sensors spatially adjacent to optical sensor 3202 may include optical sensors on other optical sensor arrays on other sides of the rotating element 3200 (not shown), as well as histograms from previously buffered and not overwritten measurements. Spatial proximity can accommodate any geometric arrangement of optical sensors in the array configuration. Instead of relying on physical proximity, embodiments can actually determine the viewing angles of adjacent optical sensors in the surrounding environment during multiple measurements used to perform the operations described above.

[0296] XIV. Circuit Implementation Scheme for Combining Pixels

[0297] Figure 33Circuitry for combining information from spatially adjacent histograms according to some embodiments is described. As described in detail above, each photosensor 3202 can be coupled with an arithmetic logic circuit 3304 and a histogram memory 3308. The histogram memory 3308 for each photosensor 3302 can also be referred to as a memory block. Each memory block 3308 can store the photon counts of a histogram 3306 that is accumulated during a measurement to form a photon count. This combination of a photosensor 3302, an arithmetic logic circuit 3304, and a memory block 3308 can be referred to as a "pixel." Each pixel can operate independently to perform an optical measurement such that each pixel can receive its own start signal and accumulate photon counts independent of timing signals used to control other pixels.

[0298] To combine information from histograms as described above, an optical measurement system can include a sum / convolution circuit 3310 that uses mathematical operations to combine histograms together. The sum / convolution circuit 3310 can be configured to scan individual time bins for a number of spatially adjacent photosensors or receive the individual time bins from each of the memory blocks 3308. Each time bin from multiple histograms can be combined together into a time bin in a single histogram 3312 that represents a single photosensor like photosensor 3302b. This circuit can use an arithmetic logic unit to add values together. This circuit can also use a multiplier to apply a weight to each of the histograms 3306 when the histograms 3306 are combined together. The final histogram 3312 can pass through a peak detection circuit 3314 as described above to locate peaks in the histogram. Because the histograms 3306 have been combined from spatially adjacent photosensors, the resulting histogram 3312 is more likely to provide one or more peaks that exceed a detection threshold.

[0299] Any identified peaks can be sent to a processor 3316 for distance calculation as described above. The processor 3316 can be implemented on a separate and different integrated circuit 3320 from the integrated circuit 3322 on which the arithmetic logic circuit 3304 and / or the memory blocks 3308 are implemented.

[0300] In Figure 33 , multiple elements can be combined to form circuitry configured to combine information from a first histogram with information from one or more spatially adjacent histograms to generate a distance measurement for a photosensor. In this embodiment, the information from the histograms can include data values from the histograms themselves, and the circuitry can include a sum / convolution circuit 3310. In some embodiments, the circuitry can also include a processor 3316 that actually generates the distance measurement.

[0301] Figure 34An alternative circuit for combining information from histograms according to some embodiments is illustrated. This circuit is similar to the circuit in Figure 33 , except that in this case, peaks can be detected by a peak detection circuit 3411 for each individual histogram 3306 before they are combined. Instead of combining all histograms, this example can only combine the identified peaks in the histograms. Some embodiments can also determine whether the peaks identified in each of the histograms 3306 are spatially related before combining the information from the histograms. For example, if several spatially adjacent histograms illustrate a peak at a similar location. If so, it can be determined that the peaks were generated by the same object in the surrounding environment and the histograms can be considered to be spatially related at their locations. This can also be used to identify outliers. For example, if eight of nine adjacent histograms indicate a peak at a location, the ninth histogram can be excluded from the calculation as an outlier.

[0302] In the example of Figure 34 , the combination of spatially adjacent histograms can be done for the center pixel, like the pixel of histogram 3306b. In some embodiments, the histograms combined with histogram 3306b can be multiplied by a weighting factor before they are combined with the center pixel.

[0303] In this example, the circuit for combining information from histograms can be implemented in a processor in a second integrated circuit 3420 that is different from the integrated circuit 3422 in which the peaks are detected. As described in detail above, the peaks can be detected on-chip, and the time interval windows can be transferred to the processor for distance measurement. The processor can also execute a sum / convolution circuit 3425 to combine the individual peaks to form a single peak 3426 representation for the light sensor 3302b. This single peak 3426 can exceed a detection threshold and can be used by a distance calculation algorithm 3427 on the processor.

[0304] Figure 35Another circuit for combining information from histograms according to some embodiments is illustrated. In this example, the processor can calculate the distance for each light sensor based on its individual histogram without any combination of the histogram information. After the distance is calculated, the information from each pixel can be combined by a summation / convolution process 3504 on the processor. This operation can be significantly simpler than combining information from raw histograms and can be performed relatively faster than the examples described above. For example, if distances have been calculated with respect to several neighboring pixels, these distances can be compared to each other. Some embodiments can average these distances of surrounding pixels to determine the distance of the center pixel. This can improve the accuracy of each distance measurement for each pixel since it is based on several different pixel measurements rather than just a single pixel. Some embodiments can also use the surrounding pixels to calculate a confidence value. If the surrounding pixels contain similar distance measurement values to the center pixel, the confidence value for the center pixel can be relatively high.

[0305] XV. Methods for combining spatially proximate data

[0306] Figure 36 A flowchart of a method for using spatially proximate pixel information in an optical measurement system is illustrated. The following method describes a process for a single light sensor combined with one or more spatially proximate light sensors in an optical measurement system. Specifically, information from one or more spatially proximate light sensors can be combined with histogram information from a single light sensor to calculate a distance measurement. However, as described above, an optical measurement system can include many light sensors or "pixels" and this method can be performed for each light sensor in the optical measurement system. Thus, this method can be performed multiple times for each light sensor, once when considering the single light sensor in which the histogram information is combined, and multiple times according to the spatially proximate light sensors of other light sensors in the array.

[0307] At step 3602, the method can include transmitting one or more pulse trains for one or more first time intervals as part of an optical measurement. Each of the one or more time intervals can represent a "shot" that is repeated multiple times in the measurement. Each of the time intervals can include one or more pulse trains encoded and transmitted by the light source such that the pulse trains can be considered to reflect off of objects in the surrounding environment. Each of the time intervals can be subdivided into a plurality of time bins such that each of the time bins represents a bin in a histogram of photon counts received during the optical measurement. The above is described with respect to Figure 9 An example of how a single measurement can include multiple shots that are subdivided into time bins of aggregated photon counts is described.

[0308] At step 3604, the method can include detecting reflected photons from the one or more pulse trains. These reflected photons can be detected using a plurality of photosensors. The plurality of photosensors can include a first photosensor, and one or more photosensors spatially adjacent to the first photosensor. As described above, photosensors can be “spatially adjacent” to a first photosensor when the fields of view of the two photosensors are adjacent in the environment surrounding the optical measurement system. In some configurations, this can include physically adjacent photosensors on the optical measurement system, while other configurations (e.g., a rotating configuration) do not require physical adjacency. As described in detail above, each photosensor can include a plurality of photodetectors, such as a plurality of SPADs. The photosensors can receive reflected light as well as environmental background noise received from the surrounding environment. The reflected light received by the photosensors can include light reflected off of objects of interest in the surrounding environment. For example, these objects of interest can be at least 30 cm away from the photosensors, and can represent surrounding vehicles, buildings, pedestrians, and / or any other objects that can be encountered near the optical measurement system during use. These objects of interest can be distinguished from objects that are part of the optical measurement system itself, such as a housing. The photosensors can also be coupled to threshold detection circuitry and / or to arithmetic logic circuitry that accumulates photon counts. Figure 5 An example of how photon counts can be received from a photosensor and counted using threshold circuitry and a pixel counter (e.g., arithmetic logic circuitry) is illustrated.

[0309] At step 3606, the method can include accumulating photon counts received during one or more time intervals. The photons can be accumulated using arithmetic logic circuitry and can be accumulated into one or more memory blocks of a histogram memory as described above. The memory blocks can be implemented using registers in SRAM of a histogram data path, as described above in Figures 8-9 Each of the time intervals can be subdivided into a plurality of first time bins for the histogram. A corresponding time bin in each of the one or more time intervals can be accumulated in a single register in the memory block. Each of the time intervals can represent a shot, and the one or more time intervals together can represent a measurement for the optical measurement system. Each of the time intervals can be defined by a start signal or shot signal of a first register in the histogram that the memory resets back to.

[0310] At step 3608, the method can include combining information from the first histogram with information from one or more histograms to generate a distance measurement for the photosensor. As described above, the information from the first histogram can include raw data from the histogram itself, identified peaks in the histogram, calculated or statistical data derived from the histogram, distance measurements calculated based on the histogram, and / or any other information that can be calculated or derived from the photon counts in the histogram. Similarly, the information from one or more histograms of spatially adjacent photosensors can also include any of these types of information. Combining the information can include summoning the histogram information, applying numerical weights to the histogram information, convolving the histogram information, applying a Gaussian function to the histogram information, and / or any other mathematical operation that can combine information from different histogram information. The combining can result in a new histogram, a new peak in a histogram, a new distance measurement, and / or another numerical value. Circuitry for performing this combining and / or generating a distance measurement for the first photosensor can include on-chip circuitry for summing / convolving the histogram information, as well as all or part of a separate processor configured to calculate the distance measurement.

[0311] It should be appreciated that Figure 36 The particular steps illustrated in FIG. 16 provide particular methods according to various embodiments of using spatially adjacent pixel information in an optical measurement system. Other sequences of steps can also be performed according to alternative embodiments. For example, alternative embodiments of the present application can perform the steps outlined above in a different order. Moreover, Figure 36 The individual steps illustrated in FIG. 16 can include multiple sub-steps that can be performed in various sequences that are suitable for the individual steps. Moreover, additional steps can be added or removed depending on the particular applications. One of ordinary skill in the art would recognize many variations, modifications, and alternatives.

[0312] XVI. Additional Embodiments

[0313] While some embodiments disclosed herein focus on applying light ranging in the context of 3D sensing for automotive use cases, the systems disclosed herein can be used in any application without departing from the scope of the disclosure. For example, the system can have a small or even miniature form factor that enables several additional use cases, such as for a solid state light ranging system. For example, the system can be used for 3D cameras and / or depth sensors within a device such as a mobile phone, tablet PC, laptop computer, desktop PC, or other peripheral and / or user interface device. For example, one or more embodiments can be employed within a mobile device to support facial recognition and facial tracking capabilities, eye tracking capabilities, and / or 3D scanning for objects. Other use cases include a forward-facing depth camera for augmented and virtual reality applications in mobile devices.

[0314] Other applications include deploying one or more systems on board vehicles such as airplanes, helicopters, drones, and the like. Such instances can provide 3D sensing and depth imaging to aid in navigation (autonomous or otherwise) and / or generate 3D maps for later analysis, such as to support geophysical, architectural, and / or archaeological analysis.

[0315] Systems can also be mounted to fixed objects and structures such as buildings, walls, utility poles, bridges, scaffolding, and the like. In such cases, the systems can be used to monitor outdoor areas such as manufacturing facilities, assembly lines, industrial facilities, construction sites, excavation sites, roadways, railroads, bridges, and the like. Further, the systems can be mounted indoors and used to monitor movement of individuals and or objects within a building, such as movement of inventory within a warehouse or people, luggage, or cargo within an office building, airport, train station, and the like. As will be appreciated by one of ordinary skill in the art having benefit of the benefit of the present disclosure, many different applications of light ranging systems are possible, and thus, the examples provided herein are provided for illustrative purposes only and should not be interpreted as limiting use of such systems to only the explicitly disclosed examples.

[0316] XVII. Computer System

[0317] Any of the computer systems mentioned herein can utilize any suitable number or type of subsystems. Subsystems can communicate over a system bus or other suitable interface. By way of example, subsystems can include an input / output (I / O) device, a system memory, a storage device, and a network adapter (e.g., Ethernet, Wi-Fi, etc.) that can be used to connect the computer system to other devices, such as an engine control unit. The system memory and / or storage device can embody a computer readable medium.

[0318] A computer system can include a number of the same components or subsystems, each of which are the same as one another but for possibly differing physical locations within the computer system. A computer system can also contain one or more mass storage devices which can be removable (e.g., floppy disks, ZIP disks, etc.) or non-removable (e.g., hard disks). Also, a computer system can contain one or more user input devices which can be any of a keyboard, a mouse, a pen, a voice input device, a touch input device, etc. A computer system can also contain one or more output devices, such as a display, a printer, a speaker, etc.

[0319] Aspects of the embodiments can be implemented using hardware circuitry (e.g., an application specific integrated circuit or a field programmable gate array) and / or computer software, either in a modular or integrated manner, with control logic being implemented in a generally programmable processor under the control of computer software. As used herein, a processor can include a single core processor, multiple core processor on the same integrated chip, or multiple processing units on separate chips or networks, as well as specialized hardware. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will know and appreciate other ways and / or methods to implement embodiments of the present application using hardware and a combination of hardware and software.

[0320] Any of the software components or functions described in this application can be implemented as software code to be executed by a processor using any suitable computer language such as, for example, Java, C, C++, C#, Objective-C, Swift, or scripting language such as Perl or Python using, for example, conventional or object-oriented techniques. The software code can be stored as a series of instructions or commands on a computer readable medium for storage and / or transmission. A suitable non-transitory computer readable medium can include random access memory (RAM), read only memory (ROM), magnetic media such as a hard disk drive or floppy disk, or optical media such as a compact disk (CD) or DVD (digital versatile disk), flash memory, among others. The computer readable medium can be any combination of such storage or transmission devices.

[0321] Such programs can also be encoded and transmitted using carrier signals, which are appropriately designed for being transmitted via wired, optical, and / or wireless networks comprising the Internet. Thus, a computer readable medium can be created using a data signal encoded with such programs. Computer readable media encoded with the program code can be packaged with a compatible device or provided separately from other devices (e.g., via Internet downloading). Any such computer readable medium can reside on or within a single computer product (e.g. a hard drive, CD, or entire computer system), and can be present on or within different computer products within a system or network. A computer system can include a monitor, printer, or other suitable display for providing any of the results mentioned herein to a user.

[0322] Any of the methods described herein can be wholly or partly performed using a computer system including one or more processors that can be configured to perform the steps. Thus, the embodiments can involve a computer system configured to perform the steps of any of the methods described herein, possibly having different components performing the respective steps or groups of steps. While presented in a particular order, steps of a method can be performed in a different order or simultaneously. Additionally, some steps can be optional. Furthermore, the steps can be performed by different components, entities, or modules. Moreover, the steps of the steps of any of the methods can be performed by a computer system of a different embodiment.

[0323] The particular embodiments described herein can be combined in any suitable manner in the art. However, other embodiments of the application can be directed to specific embodiments relating to particular aspects of the individual aspects or specific combinations of these aspects.

[0324] The foregoing description of example embodiments of the application has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the application to the precise form disclosed. Many modifications and variations are possible in light of the above teachings.

[0325] The recitation "a" or "an" or "the" is intended to mean "one or more" unless specifically indicated to the contrary. The use of "or" is intended to mean "inclusive or" and not "exclusive or", unless expressly indicated to the contrary. Reference to a "first" component does not necessarily require a second component to be provided. Further, reference to a "first" or "second" component does not limit the referenced component to a particular location unless expressly indicated. The term "based on" is intended to mean "based, at least in part, on."

[0326] All patents, patent applications, publications, and descriptions mentioned herein are incorporated by reference in their entirety for all purposes. None is admitted to be prior art.

Claims

1. An optical measurement system comprising: a housing of the optical measurement system; a light source configured to transmit one or more pulse trains in one or more time intervals as part of an optical measurement, wherein each of the one or more first time intervals contains one of the one or more pulse trains; a light sensor configured to detect photons from the one or more pulse trains reflected off of the housing of the optical measurement system and to detect photons from the one or more pulse trains reflected off of an object in an environment surrounding the optical measurement system, wherein the housing is a known distance from the light sensor and the light source; a plurality of registers configured to accumulate counts of photons received from the light sensor during the one or more time intervals, wherein each of the one or more time intervals is subdivided into a plurality of time bins representing predicted time values based on an assumed clock period, and each of the plurality of registers is configured to accumulate counts of photons received during a corresponding one of the plurality of time bins in each of the one or more time intervals to represent a histogram of counts of photons received during the one or more time intervals; and circuitry configured to identify an initial peak in the histogram of counts of photons, wherein the initial peak represents the photons reflected off of the housing of the optical measurement system, and wherein the circuitry is further configured to calibrate distance measurements of subsequent peaks using the known distance of the housing from the light sensor and the light source by determining actual time values represented by each of the plurality of time bins based on the known distance, the actual time values being based on actual clock periods.

2. The system of claim 1, wherein the circuitry is configured to identify the initial peak by identifying a predetermined number of registers of the plurality of registers that first appear in the plurality of registers.

3. The system of claim 1, wherein the circuitry is configured to identify the initial peak by identifying one or more registers of the plurality of registers that store a highest number of counts of photons.

4. The system of claim 1, wherein the circuitry is configured to identify the initial peak by identifying a register of the plurality of registers that has a time bin corresponding to a distance between the light source and the housing of the optical measurement system.

5. The system of claim 1, wherein the circuitry is further configured to identify a subset of the plurality of registers representing the initial peak.

6. The system of claim 5, wherein, the subset of the plurality of registers is identified by selecting a predetermined number of registers near a register storing a maximum value of the initial peak.

7. The system of claim 5, wherein, the subset of the plurality of registers is identified by selecting registers near a register storing a maximum value of the initial peak that store values within a predetermined percentage of the maximum value.

8. The system of claim 1, wherein the circuitry is further configured to estimate a distance between the light source of the optical measurement system and the enclosure based on a location of the initial peak in the plurality of registers.

9. The system of claim 1, further comprising a processor configured to receive additional peaks in the histogram stored in the plurality of registers to calculate a distance to an object in the surrounding environment corresponding to the additional peaks, wherein the initial peak is excluded from the additional peaks received by the processor.

10. The system of claim 9, wherein the processor is implemented in an integrated circuit that is separate and distinct from an integrated circuit in which the plurality of registers are implemented.

11. A method of detecting a peak reflected from an enclosure in an optical measurement system, the method comprising: transmitting one or more pulse trains as part of an optical measurement in one or more time intervals, wherein each of the one or more first time intervals includes one of the one or more pulse trains; detecting photons from the one or more pulse trains reflected off of the enclosure of the optical measurement system and detecting photons from the one or more pulse trains reflected off of an object in an environment surrounding the optical measurement system; accumulating counts of the photons received during the one or more time intervals into a plurality of registers, wherein each of the one or more time intervals is subdivided into a plurality of time bins representing predicted time values based on an assumed clock period, and each of the plurality of registers accumulates counts of photons received during a corresponding one of the plurality of time bins in each of the one or more time intervals to represent a histogram of counts of photons received during the one or more time intervals; and identifying an initial peak in the histogram of counts of photons, wherein the initial peak represents the photons reflected off of the enclosure of the optical measurement system; and determining an actual time value represented by each of the plurality of time bins based at least in part on a known distance between the enclosure and the optical measurement system, the actual time value based on an actual clock period.

12. The method of claim 11, further comprising: identifying a second initial peak as part of a second optical measurement; and comparing the second initial peak to the initial peak.

13. The method of claim 12, further comprising: based on comparing the second initial peak to the initial peak, characterizing a change in transparency of a window in an enclosure of the optical measurement system.

14. The method of claim 11, further comprising: identifying a plurality of initial peaks detected by a plurality of different light sensors in the optical measurement system; and based on the plurality of initial peaks, determining a level of transparency of a corresponding segment of a window in an enclosure of the optical measurement system in front of each of the plurality of light sensors.

15. The method of claim 11, further comprising: comparing a maximum of the initial peak to a threshold value; and determining whether an obstruction is located outside of the optical measurement system based on comparing the maximum of the initial peak to the threshold value.

16. The method of claim 11, further comprising: identifying a plurality of initial peaks in a plurality of measurements; and storing a baseline initial peak based on a combination of the plurality of initial peaks in the plurality of measurements for comparison to further optical measurements.

17. The method of claim 16, further comprising: subtracting the baseline initial peak from the plurality of registers.

18. The method of claim 17, wherein a second peak at least partially overlaps the initial peak, and wherein subtracting the baseline initial peak enables the second peak to be detected by peak detection circuitry.

19. The method of claim 18, wherein the second peak corresponds to an object in an environment surrounding the optical measurement system that is within two feet of the optical measurement system.

Images