Time series measurement processing for lidar accuracy

Abstract

The optical measurement system may include a light source and a corresponding optical sensor configured to emit and detect photons reflected from objects in the surrounding environment for optical measurement. An initial peak may be identified as resulting from reflection from the housing of the optical measurement system. This peak may be removed or used to calibrate the measurement calculations of the system. Peaks resulting from reflection from surrounding objects may be processed using on-chip filters to identify potential peaks, and the unfiltered data may be passed to an off-chip processor for distance calculations and other measurements. Spatial filtering techniques may be used to combine values ​​from histograms for spatially adjacent pixels in the pixel array. This combination may be used to improve the reliability of distance measurements.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Provisional Patent Application No. 62 / 913,604, filed October 10, 2019, entitled "PROCESSING TIME-SERIES MEASUREMENTS FOR LIDAR ACCURACY," which is incorporated herein by reference. [Background technology]

[0002] Light Detection And Ranging (LIDAR) systems are used to detect and range objects, for example, for vehicles such as cars, trucks, and boats. LIDAR systems also have applications in mobile applications (e.g., for facial recognition), home entertainment (e.g., for capturing gestures for video game input), and augmented reality. LIDAR systems measure the distance to an object by shining a pulse from a laser onto a scene and measuring the time it takes for a photon to travel to the object and return after reflection, as measured by a receiver in the LIDAR system. The detected signal is analyzed to detect the presence of a reflected signal pulse in the background light. The distance to the object can be determined based on the time of flight between transmitting the pulse and receiving the corresponding reflected pulse.

[0003] Providing robust distance accuracy down to a few centimeters in all conditions can be difficult, especially at the economical cost of lidar systems. Promising new detector technologies such as single-photon avalanche diodes (SPADs) are attractive but have significant drawbacks due to their limited dynamic range, especially when used to measure time-of-flight and other signal characteristics over a wide range of ambient conditions and target distances. In addition, SPADs can be highly susceptible to ambient levels of background noise light due to their high sensitivity to even a small number of photons. Summary of the Invention

[0004] In some embodiments, the optical measurement system may include a housing of the optical measurement system and a light source configured to transmit one or more pulse trains over one or more time intervals as part of the optical measurement, wherein each of the one or more first time intervals may include one of the one or more pulse trains. The system may include an optical sensor configured to detect photons from the one or more pulse trains reflected from the housing of the optical measurement system and to detect photons from the one or more pulse trains reflected from objects in an environment surrounding the optical measurement system. The system may further include a plurality of registers configured to accumulate photon counts received from the optical sensor during the one or more time intervals. Each of the one or more time intervals may be subdivided into a plurality of time bins. Each of the plurality of registers may be configured to accumulate photon counts received during a corresponding one of the plurality of time bins in each of the one or more time intervals to represent a histogram of photon counts received during the one or more time intervals. The system may further include circuitry configured to identify an initial peak in the histogram of photon counts. The initial peak may represent photons reflected from the housing of the optical measurement system.

[0005] In any embodiment, any or all of the following features may be included in any combination, and without limitation: The circuit may be configured to identify the initial peak by identifying a predetermined number of registers in the plurality of registers that occurs first in the plurality of registers; The circuit may be configured to identify the initial peak by identifying one or more registers in the plurality of registers that store a maximum number of photon counts; The circuit may be configured to identify the initial peak by identifying a register in the plurality of registers having a time bin that corresponds to a distance between a light source and a housing of the optical measurement system; The circuit may be further configured to identify a subset of the plurality of registers that represents the initial peak; The subset of the plurality of registers may be identified by selecting a predetermined number of registers surrounding a register that stores a maximum value of the initial peak; The subset of the plurality of registers may be identified by selecting registers surrounding a register that stores a maximum value of the initial peak that store values ​​that are within a predetermined percentage of the maximum value; The circuit may be further configured to estimate a distance between a light source and a housing of the optical measurement system based on the location of the initial peak in the plurality of registers; The circuit may be further configured to calibrate distance measurements using the estimated distance between the light source and the housing. The system may also include a processor configured to receive additional peaks in the histogram stored in the plurality of registers and calculate distances to objects in the surrounding environment corresponding to the additional peaks, wherein the initial peak may be excluded from the additional peaks received by the processor. The processor may be implemented in an integrated circuit separate and different from the integrated circuit in which the plurality of registers are implemented.

[0006] In some embodiments, a method for detecting peaks reflected from a housing in an optical measurement system may include transmitting one or more pulse trains over one or more time intervals as part of the optical measurement. Each of the one or more first time intervals may include one of the one or more pulse trains. The method may also include detecting photons from the one or more pulse trains reflected from a housing of the optical measurement system and detecting photons from the one or more pulse trains reflected from an object in an environment surrounding the optical measurement system. The method may further include accumulating counts of photons received during the one or more time intervals in a plurality of registers. Each of the one or more time intervals may be subdivided into a plurality of time bins, and each of the plurality of registers accumulates photon counts received during a corresponding one of the plurality of time bins in each of the one or more time intervals to represent a histogram of photon counts received during the one or more time intervals. The method may further include identifying an initial peak in the histogram of photon counts. The initial peak may represent photons reflected from the housing of the optical measurement system.

[0007] In any embodiment, any or all of the following features may be included in any combination, without limitation. The method may also include identifying a second initial peak as part of the second optical measurement and comparing the second initial peak to the initial peak. The method may also include characterizing a change in transparency of a window in a housing of the optical measurement system based on comparing the second initial peak to the initial peak. The method may further include identifying multiple initial peaks detected by multiple different optical sensors in the optical measurement system and determining a transparency level of a corresponding section of a window in a housing of the optical measurement system in front of each of the multiple optical sensors based on the multiple initial peaks. The method may also include comparing a maximum value of the initial peaks to a threshold and determining whether an obstruction is located outside the optical measurement system based on comparing the maximum value of the initial peaks to the threshold. The method may further include identifying multiple initial peaks across multiple measurements and storing a baseline initial peak based on a combination of the multiple initial peaks across the multiple measurements for comparison with future optical measurements. The method may also include subtracting the baseline initial peak from the multiple registers. The second peak can at least partially overlap with the initial peak, and subtracting the baseline initial peak can make the second peak detectable by the peak detection circuitry. The second peak can correspond to an object in the environment surrounding the optical measurement system that is within two feet of the optical measurement system.

[0008] In some embodiments, the optical measurement system may include a light source configured to transmit one or more pulse trains over one or more time intervals as part of the optical measurement. Each of the one or more time intervals may include one of the one or more pulse trains. The system may also include an optical sensor configured to detect photons from the one or more pulse trains reflected from objects in an environment surrounding the optical measurement system. The system may further include a plurality of registers configured to accumulate photon counts received from the optical sensor during the one or more time intervals to represent an unfiltered histogram of the photon counts received during the one or more time intervals. The system may further include a filter circuit configured to provide a filtered histogram of the photon counts from the plurality of registers. The system may also include a peak detection circuit configured to detect locations of peaks in the filtered histogram and use the locations of the peaks in the filtered histogram to identify locations in the plurality of registers that store an unfiltered representation of the peaks.

[0009] In any embodiment, any or all of the following features may be included in any combination, and without limitation. The system may also include a processor configured to receive an unfiltered representation of the peaks and to calculate distances to objects in an environment surrounding the optical measurement system using the unfiltered representation of the peaks. A filter circuit may be configured to provide a filtered histogram by applying a matched filter corresponding to the one or more pulse trains. A pulse train in the one or more pulse trains may include a plurality of rectangular pulses. The filter circuit may be configured to low-pass filter the unfiltered histogram. The system may also include a second plurality of registers that store the filtered histogram. The filtered histogram may be generated in a single pass through the plurality of registers. Peaks may be detected during a single pass through the plurality of registers, such that the filtered histogram is not stored in its entirety. The peak detection circuit may be configured to detect the location of the peaks by detecting an increase in the values ​​of the plurality of registers followed by a decrease in the values. The processor may be implemented in an integrated circuit (IC) separate and distinct from the IC in which the plurality of registers are implemented. The light source and the light sensor may form a pixel within a plurality of pixels in the optical measurement system.

[0010] In some embodiments, a method for analyzing filtered and unfiltered data in an optical measurement system may include transmitting one or more pulse trains over one or more first time intervals as part of an optical measurement. Each of the one or more first time intervals may include one of the one or more pulse trains. The method may also include detecting photons from the one or more pulse trains reflected from an object in an environment surrounding the optical measurement system, populating a plurality of registers using the photons to represent an unfiltered histogram of photon counts received during the one or more first time intervals, filtering the unfiltered histogram in the plurality of registers to provide a filtered histogram of photons from the plurality of registers, detecting locations of peaks in the filtered histogram, using the locations of the peaks in the filtered histogram to identify locations in the plurality of registers that store unfiltered representations of the peaks, transmitting the unfiltered representations of the peaks to a processor, and calculating distances to objects in the environment surrounding the optical measurement system using the unfiltered representations of the peaks.

[0011] In any embodiment, any or all of the following features may be included, in any combination, and without limitation: Transmitting the unfiltered representation of the peak may include transmitting information identifying histogram time bins represented in the plurality of registers storing the unfiltered representation of the peak. Filtering the unfiltered histogram in the plurality of registers may include applying a convolution of the unfiltered histogram with at least one rectangular filter having a plurality of identical values. The plurality of identical values ​​may include a sequence of binary "1" values ​​and / or a sequence of "-1" values. Filtering the unfiltered histogram in the plurality of registers may include convolving the unfiltered histogram with at least one sequence including a non-zero value followed by a plurality of zero values. The at least one sequence may include a single binary "1" value or a single binary "-1" followed by a plurality of "0" values. The method may also include transmitting the filtered representation of the peak to a processor in addition to transmitting the unfiltered representation of the peak. The location of the peak in the filtered histogram may be detected as a single peak in the filtered histogram, and the unfiltered representation of the peak may include at least two peaks in the unfiltered histogram. One of the at least two peaks in the unfiltered histogram may represent a peak resulting from reflection of one or more pulse trains from a housing or window of the optical measurement system. Photons may be detected using multiple photodetectors in the optical sensor.

[0012] In some embodiments, the optical measurement system may include a plurality of light sources configured to emit one or more pulse trains over one or more time intervals as part of the optical measurement. The system may also include a plurality of light sensors configured to detect reflected photons from one or more pulse trains emitted from corresponding light sources in the plurality of light sources. The plurality of light sensors may include a first light sensor and one or more other light sensors spatially adjacent to the first light sensor. The system may further include a plurality of memory blocks configured to accumulate photon counts of photons received during one or more time intervals by corresponding light sensors in the plurality of light sensors to represent a plurality of histograms of photon counts. The plurality of histograms may include a first histogram corresponding to the first light sensor and one or more histograms corresponding to the one or more other light sensors. The system may further include circuitry configured to combine information from the first histogram with information from the one or more other histograms to generate a distance measurement for the first light sensor.

[0013] In any embodiment, any or all of the following features may be included in any combination, without limitation: the one or more light sensors may be physically adjacent to a first light sensor in the array of light sensors; the array of light sensors may include a solid-state array of light sensors; the one or more light sensors may include eight light sensors orthogonally or diagonally adjacent to the first light sensor; the one or more light sensors need not be physically adjacent to the first light sensor in the array of light sensors, but may be positioned to receive photons from a physical area adjacent to the physical area from which photons are received by the first light sensor; the multiple light sensors may be arranged in an array of light sensors that rotates around a central axis of the optical measurement system; the information from the first histogram may include a first distance measurement calculated based on the first histogram, the information from the one or more histograms may include one or more other distance measurements calculated based on one or more other histograms, and the distance measurement may include a combination of the first distance measurement with one or more other distance measurements. The first distance measurement may be below a detection limit of the optical measurement system before combining the first distance measurement with the plurality of other distance measurements. The distance measurement may be above a detection limit of the optical measurement system after combining the first distance measurement with the plurality of other distance measurements. The detection limit may represent a minimum number of photons received by a corresponding optical sensor. The circuit that combines information from the first histogram with information from the one or more histograms may include a processor implemented on an integrated circuit different from the integrated circuit on which the plurality of memory blocks are implemented. The circuit and the plurality of memory blocks may be implemented on the same integrated circuit.

[0014] In some embodiments, a method for using spatially adjacent pixel information in an optical measurement system may include transmitting one or more pulse trains over one or more first time intervals as part of an optical measurement and detecting reflected photons from the one or more pulse trains using multiple optical sensors. The multiple optical sensors may include a first optical sensor and one or more optical sensors spatially adjacent to the first optical sensor. The method may also include accumulating photon counts received by the multiple optical sensors during the one or more time intervals to represent multiple histograms of photon counts. The multiple histograms may include a first histogram corresponding to the first optical sensor and one or more histograms corresponding to the one or more optical sensors. The method may further include combining information from the first histogram with information from the one or more histograms to generate a distance measurement for the first optical sensor.

[0015] In any embodiment, any or all of the following features may be included in any combination, without limitation: the reflected photons received by the first light sensor and the one or more light sensors may be reflected from the same object in the surrounding environment; the information from the first histogram may include photon counts in the first histogram, and the information from the one or more histograms may include photon counts in the one or more histograms, and a distance measurement may be calculated based on an aggregation of the photon counts in the first histogram with the photon counts in the first histogram; the information from the first histogram may include one or more first peaks in the first histogram, and the information from the one or more histograms may include one or more second peaks in the one or more histograms, and a distance measurement may be calculated based on a combination of the first one or more peaks and the second one or more peaks; the distance measurement may be calculated based on a sum of the first one or more peaks and the second one or more peaks; or the distance measurement may be calculated based on a Gaussian combination of the first one or more peaks and the second one or more peaks. The distance measure may be calculated based on a convolution of the first one or more peaks and the second one or more peaks. The distance measure may be calculated based on a weighted combination of the first one or more peaks and the second one or more peaks.

[0016] A further understanding of the nature and advantages of various embodiments may be realized by reference to the remaining portions of the specification and the drawings, wherein like reference numerals are used throughout the several views to refer to like components. In some instances, a sub-label is associated with a reference numeral to indicate one of multiple similar components. When reference is made to a reference numeral without designation of the existing sub-label, it is intended to refer to all such multiple similar components. [Brief explanation of the drawings]

[0017] [Figure 1A] 1 illustrates an automotive optical ranging device, also referred to herein as a LIDAR system, according to some embodiments. [Figure 1B]1 illustrates an automotive optical ranging device, also referred to herein as a LIDAR system, according to some embodiments.

[0018] [Figure 2] FIG. 1 shows a block diagram of an exemplary lidar device for implementing various embodiments.

[0019] [Figure 3] 1 illustrates the operation of a typical lidar system that may be improved by embodiments.

[0020] [Figure 4] 1 illustrates an illustrative example of an optical transmission and detection process of an optical ranging system, according to some embodiments.

[0021] [Figure 5] 1 illustrates various stages of a sensor array and associated electronics according to an embodiment of the present invention.

[0022] [Figure 6] 1 shows a histogram according to an embodiment of the present invention.

[0023] [Figure 7] 10 illustrates histogram accumulation over multiple pulse trains for a selected pixel in accordance with an embodiment of the present invention.

[0024] [Figure 8] 1 illustrates a circuit for receiving photons and generating a set of signals stored in memory that represent a histogram, according to some embodiments.

[0025] [Figure 9] 10 illustrates timing associated with different shots in a measurement, according to some embodiments.

[0026] [Figure 10]1 illustrates a representation of a histogram memory containing peaks resulting from reflections from a housing of an optical measurement system.

[0027] [Figure 11] 1 illustrates a flowchart of a method for detecting peaks resulting from early reflections from a system housing using an optical measurement system.

[0028] [Figure 12] 1 illustrates the contents of a histogram memory that may be used to calibrate distance measurements for an optical measurement system, according to some embodiments.

[0029] [Figure 13A] 10 illustrates a portion of a histogram memory receiving an initial peak corresponding to a reflection from a system housing, according to some embodiments.

[0030] [Figure 13B] 10 illustrates the variation in the magnitude of a peak reflected from a housing according to some embodiments.

[0031] [Figure 14] 10 illustrates a portion of an array of light sensors that may be partially obscured by contamination of a window, according to some embodiments.

[0032] [Figure 15A] 10 illustrates an initial peak detected by an optical measurement system relative to an occlusion threshold, according to some embodiments.

[0033] [Figure 15B] 10 illustrates an initial peak later in the life cycle of an optical measurement system when an occlusion is present, according to some embodiments.

[0034] [Figure 16A]1 illustrates a histogram memory having an initial peak resulting from a housing reflection and a second peak resulting from a reflection from an object in the surrounding environment, according to some embodiments.

[0035] [Figure 16B] 10 illustrates the problem with near-field peak detection without compensation for reflections from the housing, according to some embodiments.

[0036] [Figure 17A] 16C illustrates the histogram memory from FIG. 16B.

[0037] [Figure 17B] 16C illustrates the histogram memory from FIG. 16B after the baseline peak has been subtracted from the registers in the histogram memory.

[0038] [Figure 18] 1 illustrates a circuit for removing the effects of reflections from the system housing during close-range measurements, according to some embodiments.

[0039] [Figure 19] 10 illustrates the contents of a histogram memory after a single pulse is transmitted and received by an optical measurement system, according to some embodiments.

[0040] [Figure 20] 20 illustrates a filtered version of the histogram data using the filter from FIG. 19 according to some embodiments.

[0041] [Figure 21A] Illustrates an example of two adjacent peaks that are relatively close together in time.

[0042] [Figure 21B] 21B illustrates the effect a low-pass filter may have on the unfiltered histogram data from FIG. 21A, according to some embodiments.

[0043] [Figure 22A] 10 illustrates how filtered data can be used to identify peaks in unfiltered data, according to some embodiments. [Figure 22B] 10 illustrates how filtered data can be used to identify peaks in unfiltered data, according to some embodiments.

[0044] [Figure 23] 1 illustrates a schematic diagram of a circuit for using filtered data and passing unfiltered data to a processor for distance calculations, according to some embodiments.

[0045] [Figure 24] 10 illustrates a portion of a histogram memory after receiving photons reflected from a multi-pulse code, according to some embodiments.

[0046] [Figure 25] 10 illustrates a filtered version of a peak received from a multi-pulse code, according to some embodiments.

[0047] [Figure 26] 1 illustrates a filter that uses only a single binary designator, according to some embodiments.

[0048] [Figure 27] 1 illustrates a flowchart of a method for analyzing filtered and unfiltered data in an optical metrology system, according to some embodiments.

[0049] [Figure 28] 1 illustrates an example of an object that can provide reflected photons to an adjacent photosensor, according to some embodiments.

[0050] [Figure 29] 10 illustrates an example of how spatially adjacent photosensor views may be used in the calculated distance measurement of one of the corresponding photosensors, according to some embodiments.

[0051] [Figure 30] 10 illustrates how histograms for adjacent photosensors can be combined, according to some embodiments.

[0052] [Figure 31] 1 illustrates an example of a rectangular photosensor layout, according to some embodiments.

[0053] [Figure 32] 1 illustrates an exemplary configuration for a rotational optical measurement system, according to some embodiments.

[0054] [Figure 33] 1 illustrates a circuit for combining information from spatially adjacent histograms according to some embodiments.

[0055] [Figure 34] 10 illustrates an alternative circuit for combining information from histograms, according to some embodiments.

[0056] [Figure 35] 10 illustrates another circuit for combining information from histograms, according to some embodiments.

[0057] [Figure 36] 1 illustrates a flowchart of a method for using spatially adjacent pixel information in an optical measurement system.

[0058] (term) The term “ranging,” particularly when used in the context of methods and devices for measuring the environment or assisting in vehicle operation, may refer to determining the distance or distance vector from one location or position to another. “Optical ranging” may refer to a type of ranging method that uses electromagnetic waves to perform a ranging method or function. Accordingly, an “optical ranging device” may refer to a device for performing an optical ranging method or function. “Lidar” (or LIDAR) may refer to a type of optical ranging method that measures the distance to a target by illuminating the target with pulsed laser light and then measuring the reflected pulses with a sensor. Accordingly, a “lidar device” or “lidar system” may refer to a type of optical ranging device for performing a lidar method or function. An “optical ranging system” may refer to a system that includes at least one optical ranging device, such as a lidar device. The system may further include one or more other devices or components in various configurations.

[0059] A "pulse train" may refer to one or more pulses transmitted together. The emission and detection of a pulse train may be referred to as a "shot." A shot may occur over a "detection time interval" (or "detection interval").

[0060] A "measurement" may include a train of N pulses emitted and detected over N shots, each lasting a detection time interval. An entire measurement may be equal to the N detection intervals of the measurement, or may exceed the measurement time interval (or simply "measurement interval"), which may be longer, for example, if pauses occur between the detection intervals.

[0061] An "optical sensor" or "photosensitive element" can convert light into an electrical signal. An optical sensor may include multiple "photodetectors," such as single-photon avalanche diodes (SPADs). An optical sensor may correspond to a specific pixel of the ranging measurement resolution.

[0062] A "histogram" may refer to any data structure representing a series of values ​​over time, discretized across time bins. A histogram may have a value assigned to each time bin. For example, a histogram may store a counter of the number of photodetectors activated during each particular time bin of one or more detection intervals. As another example, a histogram may correspond to the digitization of an analog signal at different times. A histogram may include a signal (e.g., a pulse) and noise. Thus, a histogram can be viewed as a combination of signal and noise as a time series or photon flux of photons. A raw / digitized histogram (or accumulated photon time series) may include the digitized signal and noise in memory without filtering. A "filtered histogram" may refer to the output of a raw histogram after it has passed through a filter.

[0063] An emitted signal / pulse may refer to an undistorted "nominal," "ideal," or "template" pulse or pulse train. A reflected signal / pulse may refer to a reflected laser pulse from an object, which may be distorted. A digitized signal / pulse (or raw signal) may refer to the digitized result from detecting one or more pulse trains over a detection interval, stored in memory, and thus may correspond to a portion of a histogram. A detected signal / pulse may refer to the location in memory where the signal was detected. A detected pulse train may refer to the actual pulse train found by a matched filter. A predicted signal profile may refer to the shape of a digitized signal that would result from a particular emitted signal with a particular distortion in the reflected signal. DETAILED DESCRIPTION OF THE INVENTION

[0064] The present disclosure relates generally to the field of object detection and ranging, and more particularly to the use of time-of-flight optical receiver systems for applications such as real-time 3D mapping and object detection, tracking and / or classification. Various improvements can be realized by various embodiments of the present invention.

[0065] The following sections introduce an exemplary automotive LIDAR system, followed by a description of exemplary techniques for detecting signals with an optical ranging system, and then a more detailed description of different embodiments.

[0066] I. Exemplary Automotive Lidar System 1A and 1B illustrate an optical ranging device for an automobile, also referred to herein as a LIDAR system, according to some embodiments. An automobile application for a LIDAR system is selected here for illustrative purposes only; the sensors described herein can be utilized in other types of vehicles, such as boats, airplanes, and trains, as well as in a variety of other applications where 3D depth images are useful, such as medical imaging, mobile phones, augmented reality, geodesy, geoinformatics, archaeology, geography, geology, topography, seismology, forestry, atmospheric physics, laser guidance, airborne laser swath mapping (ALSM), and laser altimetry. In some embodiments, a LIDAR system, such as a scanning LIDAR system 101 and / or a solid-state LIDAR system 103, can be mounted on the roof of a vehicle 105, as shown in FIGS. 1A and 1B.

[0067] The scanning lidar system 101 shown in FIG. 1A can employ a scanning architecture in which the orientation of the lidar light source 107 and / or detector circuitry 109 can be scanned around one or more fields of view 110 within an external field or scene outside the vehicle 105. With a scanning architecture, emitted light 111 can be scanned across the surrounding environment as shown. For example, the output beam of one or more light sources (such as infrared or near-infrared pulsed IR lasers, not shown) positioned within the lidar system 101 can be scanned, e.g., rotated, to illuminate the scene around the vehicle. In some embodiments, the scanning, represented by the rotation arrow 115, can be implemented by mechanical means, e.g., by mounting the light emitter on a rotating column or platform. In some embodiments, the scanning can be implemented through other mechanical means, such as through the use of a galvanometer. Chip-based steering techniques can also be employed, e.g., by using a microchip, using one or more MEMS-based reflectors, such as a digital micromirror (DMD) device, digital light processing (DLP) device, etc. In some embodiments, scanning can be achieved through non-mechanical means, for example, by steering one or more optical phased arrays using electronic signals.

[0068] For a fixed architecture, such as the solid-state lidar system 103 shown in FIG. 1B, one or more solid-state lidar subsystems (e.g., 103a and 103b) can be mounted on a vehicle 105. Each solid-state lidar unit can point in a different direction (possibly with partially overlapping and / or non-overlapping fields of view between units) to capture a larger combined field of view than each unit could capture by itself.

[0069] In either a scanning or fixed architecture, objects in the scene can reflect a portion of the light pulse emitted from the lidar light source. One or more reflected portions can then return to the lidar system and be detected by a detector circuit. For example, reflected portion 117 can be detected by detector circuit 109. The detector circuit can be disposed in the same housing as the emitter. Aspects of scanning and fixed systems are not mutually exclusive and can be used in combination. For example, the individual lidar subsystems 103a and 103b in FIG. 1B can use steerable emitters, such as optical phased arrays, or the entire combined unit can rotate via mechanical means to scan the entire scene in front of the lidar system, for example, from field of view 119 to field of view 121.

[0070] Figure 2 shows a more detailed block diagram of a rotating lidar system 200, according to some embodiments. More specifically, Figure 2 shows a rotating lidar system that can optionally employ a rotary actuator on a rotating circuit board that can receive (and transmit) power and data from a fixed circuit board.

[0071] The lidar system 200 can interact with one or more instantiations of a user interface 215. Different instantiations of the user interface 215 can vary and may include, for example, a computer system with a monitor, keyboard, mouse, CPU and memory, a touchscreen in an automobile, a handheld device with a touchscreen, or other suitable user interface. The user interface 215 may be local to the object to which the lidar system 200 is attached, or it may be a remotely operated system. For example, commands and data to or from the lidar system 200 can be routed over a cellular network (e.g., LTE), a personal area network (e.g., Bluetooth, Zigbee), a local area network (e.g., WiFi, IR), or a wide area network such as the Internet.

[0072] The hardware and software user interface 215 can present lidar data from the device to the user, but can also allow the user to control the lidar system 200 with one or more commands. Example commands include commands to activate or deactivate the lidar system, commands specifying photodetector exposure levels, biases, sampling periods, and other operating parameters (e.g., emission pulse patterns and signal processing), and commands specifying light emitter parameters such as brightness. Additionally, the commands can allow the user to select a method for displaying the results. The user interface can display the results of the lidar system, including, for example, a single-frame snapshot image, a continuously updated video image, and / or other light measurements for some or all pixels. In some embodiments, the user interface 215 can track the distance (proximity) of objects from the vehicle and potentially provide warnings to the driver or provide such tracking information for analysis of driver behavior.

[0073] In some embodiments, the lidar system can communicate with a vehicle control unit 217, and one or more parameters associated with controlling the vehicle can be modified based on the received lidar data. For example, in a fully autonomous vehicle, the lidar system can provide a real-time 3D image of the environment surrounding the vehicle to assist with navigation. In other cases, the lidar system can be used as part of an advanced driver assistance system (ADAS) or as part of a safety system that can provide 3D image data to any number of different systems, such as adaptive cruise control, automatic parking, driver drowsiness monitoring, blind spot monitoring, collision avoidance systems, etc. If the vehicle control unit 217 is communicatively coupled to the optical ranging device 210, it can provide warnings to the driver or track the nearby pursuit of an object.

[0074] The LIDAR system 200 shown in FIG. 2 includes an optical ranging device 210. The optical ranging device 210 includes a ranging system controller 250, an optical transmitting (Tx) module 240, and an optical sensing (Rx) module 230. Ranging data can be generated by the optical ranging device by transmitting one or more optical pulses 249 from the optical transmitting module 240 to an object within a field of view surrounding the optical ranging device. A reflected portion 239 of the transmitted light is then detected by the optical sensing module 230 after some delay time. Based on the delay time, the distance to the reflective surface can be determined. Other ranging methods, such as continuous wave, Doppler, etc., can also be used.

[0075] The Tx module 240 includes an emitter array 242, which may be a one- or two-dimensional array of emitters, and a Tx optical system 244, which, when combined, may form an array of micro-optical emitter channels. The emitter array 242 or individual emitters are examples of laser light sources. The Tx module 240 further includes a processor 245 and a memory 246. In some embodiments, pulse coding techniques, such as Barker codes, may be used. In such cases, the memory 246 may store pulse codes that indicate when to transmit light. In one embodiment, the pulse codes are stored as a sequence of integers stored in memory.

[0076] The Rx module 230 may include a sensor array 236, which may be, for example, a one-dimensional or two-dimensional array of optical sensors. Each optical sensor or photosensitive element (also referred to as a sensor) may include, for example, a collection of photodetectors such as APDs, or the sensor may be a single photon detector (e.g., a SPAD). Like the Tx module 240, the Rx module 230 includes an Rx optical system 237. The Rx optical system 237 and the sensor array 236 may combine to form an array of micro-optical receiver channels. Each micro-optical receiver channel measures light corresponding to an image pixel within a distinct field of view of the surrounding volume. Each sensor (e.g., a collection of SPADs) in the sensor array 236 may correspond to a specific emitter in the emitter array 242, for example, as a result of the geometric configuration of the optical sensing module 230 and the optical transmitting module 240.

[0077] In one embodiment, the sensor array 236 of the Rx module 230 is fabricated (e.g., using CMOS technology) as part of a monolithic device on a single substrate that includes both an array of photon detectors and an ASIC 231 for signal processing of raw histograms from individual photon detectors (or groups of detectors) in the array. As an example of signal processing, for each photon detector or group of photon detectors, the memory 234 (e.g., SRAM) of the ASIC 231 can accumulate counts of detected photons over successive time bins, which can be used in combination to recreate a time series of reflected light pulses (i.e., photon counts versus time). This time series of aggregated photon counts is referred to herein as an intensity histogram (or simply a histogram). The ASIC 231 can implement matched filters and peak detection processing to identify return signals in time. Additionally, ASIC 231 may accomplish (e.g., by processor 238) certain signal processing techniques, such as multi-profile matched filtering, that help recover a photon time series that is less susceptible to pulse shape distortions that may occur due to SPAD saturation and quenching. In some embodiments, all or part of such filtering may be performed by processor 258, which may be embodied in an FPGA.

[0078] In some embodiments, the Rx optical system 237 may be part of the same monolithic structure as the ASIC, with a separate substrate layer for each receiver channel layer. For example, the aperture layer, collimating lens layer, optical filter layer, and photodetector layer may be stacked and bonded at the wafer level before dicing. The aperture layer may be formed by placing a non-transparent substrate on a transparent substrate or by coating a transparent substrate with an opaque film. In yet other embodiments, one or more components of the Rx module 230 may be external to the monolithic structure. For example, the aperture layer may be implemented as a separate metal sheet with pinholes.

[0079] In some embodiments, the photon time series output from the ASIC is sent to the ranging system controller 250 for further processing; for example, the data may be encoded by one or more encoders in the ranging system controller 250 and then sent as data packets to the user interface 215. The ranging system controller 250 can be implemented in multiple ways, including, for example, by using a programmable logic device such as an FPGA as part of the ASIC or an ASIC, by using a processor 258 with memory 254, and any combination of the above. The ranging system controller 250 can operate in conjunction with a fixed base controller or independently of the base controller (via pre-programmed instructions) to control the light-sensing module 230 by sending commands to adjust light detector parameters, including starting and stopping light detection. Similarly, the ranging system controller 250 can control the light-transmitting module 240 by sending commands, including light emission start and stop controls, and controls that may adjust other light emitter parameters (e.g., pulse code), or by relaying commands from the base controller. In some embodiments, the ranging system controller 250 has one or more wired interfaces or connectors for exchanging data with the light-sensing module 230 and the light-transmitting module 240. In other embodiments, the ranging system controller 250 communicates with the light-sensing module 230 and the light-transmitting module 240 via a wireless interconnection, such as an optical communication link.

[0080] Electric motor 260 may be an optional component required when it is necessary to rotate system components such as Tx module 240 and / or Rx module 230. System controller 250 can control electric motor 260 to start rotation, stop rotation, and vary rotation speed.

[0081] II. Detection of reflected pulses The optical sensors can be arranged in various ways to detect the reflected pulses. For example, the optical sensors can be arranged in an array, and each optical sensor can include an array of photodetectors (e.g., SPADs). Different patterns of pulses (pulse trains) transmitted during the detection interval are also described below.

[0082] A. Time-of-Flight Measurement and Detectors FIG. 3 illustrates the operation of a typical lidar system that may be improved by some embodiments. A laser generates a short-duration optical pulse 310. The horizontal axis represents time, and the vertical axis represents power. An example laser pulse duration, characterized by its full width at half maximum (FWHM), is a few nanoseconds, with a single emitter having a peak power of a few watts. Embodiments using side-emitter lasers or fiber lasers may have much higher peak powers, while embodiments with small-diameter VCSELs may have peak powers in the tens to hundreds of milliwatts.

[0083] The start time 315 of the transmission of the pulse does not have to coincide with the leading edge of the pulse. As shown, the leading edge of the light pulse 310 can be after the start time 315. For example, for coded pulses, one may want the leading edge to be different in situations where different patterns of pulses are transmitted at different times.

[0084] The optical receiver system can begin detecting the received light simultaneously with, or at, the start of the laser. In other embodiments, the optical receiver system can begin at a later time, a known time after the start of the pulse. The optical receiver system first detects background light 330 and then detects the laser pulse reflection 320 some time later. The optical receiver system can compare the detected light intensity to a threshold to identify the laser pulse reflection 320. The threshold can distinguish the background light 330 from light corresponding to the laser pulse reflection 320.

[0085] The time of flight 340 is the time difference between when a pulse is transmitted and when it is received. The time difference can be measured by dividing the transmission time of the pulse (e.g., measured relative to the start time) from the reception time of the laser pulse reflection 320 (e.g., also measured relative to the start time). The distance to the target can be determined as half the product of the time of flight and the speed of light. Pulses from the laser device reflect off objects in the scene at different times, and the pixel array detects the pulses of radiation reflection.

[0086] B. Object Detection Using an Array Laser and an Array of Light Sensors Figure 4 shows an illustrative example of the optical transmission and detection process of an optical ranging system, according to some embodiments. Figure 4 shows an optical ranging system (e.g., solid-state and / or scanning) that collects three-dimensional distance data of a volume or scene surrounding the system. Figure 4 is an idealized diagram that emphasizes the relationship between the emitter and sensor, and therefore other components are not shown.

[0087] Optical ranging system 400 includes light emitter array 402 and light sensor array 404. Light emitter array 402 includes an array of light emitters, such as emitter 403 and emitter 409, such as an array of VCSELs. Light sensor array 404 includes an array of light sensors, such as sensors 413 and 415. The light sensors may be pixelated light sensors that use a set of discrete photodetectors, such as single-photon avalanche diodes (SPADs), per pixel. However, various embodiments may arrange for any type of photon sensor.

[0088] Each emitter may be slightly offset from adjacent emitters and configured to transmit light pulses into different fields of view from adjacent emitters, thereby illuminating the respective fields of view associated with only that emitter. For example, emitter 403 emits illumination beam 405 (formed from one or more light pulses) into circular field of view 407 (the size of which is exaggerated for clarity). Similarly, emitter 409 emits illumination beam 406 (also called an emitter channel) into circular field of view 410. Although not shown in FIG. 4 to avoid complexity, each emitter emits a corresponding illumination beam into a corresponding field of view, resulting in the illumination of a 2D array of fields of view (21 different fields of view in this example).

[0089] Each field of view illuminated by an emitter can be thought of as a pixel or spot in the corresponding 3D image generated from the ranging data. Each emitter channel can be distinct from the other emitter channels and non-overlapping with other emitter channels, meaning there is a one-to-one mapping between the set of emitters and the set of non-overlapping fields or views. Thus, in the example of FIG. 4, the system can sample 21 distinct points in 3D space. Higher-density sampling of points can be achieved by having a denser emitter array or by scanning the angular position of the emitter beam over time so that one emitter can sample multiple points in space. As noted above, scanning can be accomplished by rotating the entire emitter / sensor assembly.

[0090] Each sensor, like the emitters described above, can be slightly offset from adjacent sensors, such that each sensor can see a different field of view of the scene in front of the sensor. Further, the field of view of each sensor can, for example, substantially match, e.g., overlap, and be the same size as, the field of view of the respective emitter channel.

[0091] 4, the distance between corresponding emitter-sensor channels is exaggerated relative to the distance to objects in the field of view. In reality, the distance to objects in a small field of view is much greater than the distance between corresponding emitter-sensor channels, so the path of light from the emitter to the object is nearly parallel to the path of reflected light returning from the object to the sensor (i.e., the light is mostly "back-reflected"). Thus, there is a range of distances in front of system 400 where the field of view of each sensor and each emitter overlap.

[0092] Because the emitter fields of view overlap with the respective sensor fields of view, each sensor channel ideally can detect reflected illumination beams originating from the respective emitter channel and ideally without crosstalk, i.e., no reflected light from other illumination beams is detected. Thus, each light sensor may correspond to a respective light source. For example, emitter 403 emits illumination beam 405 within a circular field of view 407, and a portion of the illumination beam reflects off object 408. Ideally, reflected beam 411 is detected only by sensor 413. Thus, emitter 403 and sensor 413 share the same field of view, e.g., field of view 407, and form an emitter-sensor pair. Similarly, emitter 409 and sensor 415 form an emitter-sensor pair and share field of view 410. Although the emitter-sensor pairs are shown in FIG. 4 as being in the same relative positions within their respective arrays, any emitter can be paired with any sensor depending on the design of the optical system used in the system.

[0093] During ranging measurements, reflected light from different fields of view distributed around the volume surrounding the lidar system is collected and processed by various sensors to provide range information for any objects within their respective fields of view. As described above, time-of-flight techniques can be used, in which a precisely timed pulse is emitted by an optical emitter and, after a certain time, the reflection of the pulse is detected by each sensor. The time elapsed between emission and detection and the known speed of light are then used to calculate the distance to the reflecting surface. In some embodiments, in addition to range, additional information can be obtained by the sensors to determine other characteristics of the reflecting surface. For example, the Doppler shift of the pulse can be measured by the sensors and used to calculate the relative velocity between the sensor and the reflecting surface. The pulse intensity can be used to estimate the reflectivity of the target, and the pulse shape can be used to determine whether the target is a hard or diffuse material.

[0094] In some embodiments, the lidar system is configured with a relatively large 2D array of emitter and sensor channels and can operate as a solid-state lidar, i.e., the lidar system can acquire frames of range data without requiring the emitter and / or sensor to scan. In other embodiments, the emitters and sensors can be scanned, e.g., rotated about an axis, to ensure that the field of view of the set of emitters and sensors samples a complete 360-degree region of the surrounding volume (or some useful portion of the 360-degree region). For example, range data collected from a scanning system over a given period of time can then be post-processed into one or more frames of data, which can be further processed into one or more depth images or 3D point clouds. The depth images and / or 3D point clouds can be further processed into map files for use in 3D mapping and navigation applications.

[0095] C. Multiple photodetectors in each optical sensor 5 illustrates various stages of a sensor array and associated electronics according to an embodiment of the present invention. Array 510 illustrates photosensors 515, each corresponding to a different pixel. Array 510 may be a staggered array. In this particular example, array 510 is 18 x 4 photosensors. Because the implementation is suitable for sweeping, array 510 can be used to achieve high resolution (e.g., 72 x 1024).

[0096] Array 520 shows an expanded view of a portion of array 510. As shown, each photosensor 515 is made up of multiple photodetectors 525. The signals from a pixel's photodetectors collectively contribute to the measurement of that pixel.

[0097] In some embodiments, each pixel contains multiple single-photon avalanche diode (SPAD) units, which increase the dynamic range of the pixel itself. Each SPAD may contain analog front-end circuitry for biasing, quenching, and recharging. SPADs are typically biased at a bias voltage above the breakdown voltage. Suitable circuitry detects the leading edge of the avalanche current, generates a standard output pulse synchronized with the avalanche buildup, and quenches the avalanche by lowering the bias below the breakdown voltage, returning the photodiode to its operating level.

[0098] SPADs may be positioned to maximize the fill factor in their local area, or a microlens array may be used to enable a high optical fill factor at the pixel level. Thus, an imager pixel may contain an array of SPADs to increase the efficiency of the pixel detector. A diffuser may be used to diffuse the light passing through the aperture and collimated by the microlenses. The can diffuser serves to diffuse the parallel light so that all SPADs belonging to the same pixel receive some radiation.

[0099] FIG. 5 further illustrates a specific photodetector 530 (e.g., a SPAD) that detects a photon 532. In response to detection, the photodetector 530 generates an avalanche current 534 of charge carriers (electrons or holes). A threshold circuit 540 regulates the avalanche current 534 by comparing the avalanche current 534 to a threshold. If a photon is detected and the photodetector 530 is functioning properly, the avalanche current 534 rises above the comparator threshold, and the threshold circuit 540 generates a time-accurate binary signal 545 indicating the exact time of the SPAD current avalanche, which in turn is an accurate measure of photon arrival. The correlation between the current avalanche and photon arrival can occur with nanosecond resolution, thereby providing high timing resolution. The rising edge of the binary signal 545 can be latched by a pixel counter 550.

[0100] Binary signal 545, avalanche current 534, and pixel counter 550 are examples of data values ​​that may be provided by an optical sensor comprised of one or more SPADs. The data values ​​may be determined from respective signals from each of a plurality of photodetectors. Each signal may be compared to a threshold value to determine whether the corresponding photodetector has been triggered. Avalanche current 534 is an example of an analog signal, and therefore each signal may be an analog signal.

[0101] The pixel counter 550 can use the binary signal 545 to count the number of photodetectors for a given pixel that are triggered by one or more photons during a particular time bin (e.g., a time window of 1, 2, 3 ns, etc.) controlled by the periodic signal 560. The pixel counter 550 can store a counter for each of multiple time bins of a given measurement. The value of the counter for each time bin starts at zero and can be incremented based on the binary signal 545 indicating the detection of a photon. When any photodetector in the pixel provides such a signal, the counter can be incremented.

[0102] The periodic signal 560 can be generated by a phase-locked loop (PLL) or a delay-locked loop (DLL), or any other method for generating a clock signal. The coordination of the periodic signal 560 with the pixel counter 550 can function as a time-to-digital converter (TDC), a device for recognizing events and providing a digital representation of the time they occurred. For example, the TDC can output the arrival time of each detected photon or light pulse. The measured time may not be absolute time, but rather the elapsed time between two events (e.g., the start time and the detected photon or light pulse). The periodic signal 560 can be a relatively fast clock that switches between banks of memory containing the pixel counter 550. Each register in the memory may correspond to one histogram bin, and the clock may switch between them at the sampling interval. Thus, if the respective signal is greater than a threshold, a binary value indicating a trigger can be sent to a histogram circuit. The histogram circuit can aggregate the binary values ​​across multiple photodetectors to determine the number of photodetectors that were triggered during a particular time bin.

[0103] The time bins may be measured relative to a start signal, for example, at start time 315 in FIG. 3 . Thus, the counter for the time bin immediately following the start signal may have a low value corresponding to a background signal, e.g., background light 330. The final time bin may correspond to the end of the detection time interval (also called a shot) for a given pulse train, as described further in the next section. The number of cycles of the periodic signal 560 since the start time may serve as a timestamp for when the rising edge of the avalanche current 534 indicates a detected photon. The timestamp corresponds to a time bin for a particular counter in the pixel counter 550. This operation differs from a simple analog-to-digital converter (ADC) following a photodiode (e.g., an avalanche photodiode (APD)). Each of the time bin counters may correspond to a histogram, as described in more detail below. Thus, while an APD is a gain-limited linear amplifier for an input optical signal, a SPAD is a trigger device that provides a binary yes / no output for trigger events occurring within a time window.

[0104] D. Pulse train Ranging can be accomplished using a pulse train, defined as including one or more pulses, within which the number of pulses, the width of the pulses, and the duration between pulses (collectively referred to as the pulse pattern) can be selected based on several factors, some of which include: 1 - Maximum Laser Duty Cycle - Duty cycle is the percentage of time the laser is on. For pulsed lasers, this can be determined by the FWHM and the number of pulses emitted in a given period, as explained above. 2- Eye safety limit - This is determined by the maximum amount of radiation that the device can emit without damaging the eyes of a bystander looking in the direction of the lidar system. 3- Power Consumption-This is the power consumed by the emitter to illuminate the scene.

[0105] For example, the spacing between pulses in a pulse train can be on the order of single or tens of nanoseconds.

[0106] Multiple pulse trains may be emitted during a single measurement, each corresponding to a different time interval, e.g., each subsequent pulse train is not emitted until the expiration of the time limit for detecting the reflected pulse of the previous pulse train.

[0107] For a given emitter or laser device, the time between the emission of the pulse trains determines the maximum detectable range. For example, if pulse train A is emitted at time t0 = 0 ns and pulse train B is emitted at time t1 = 1000 ns, any reflected pulse trains detected after t1 are much more likely to be reflections from pulse train B and should not be assigned to pulse train A. Thus, the time between pulse trains and the speed of light define the maximum limit of the system's range, given by the following equation: Rmax=c×(t1-t0) / 2 The time between shots (emission and detection of the pulse train) can be as little as 1 μs, giving the entire pulse train enough time to travel to a distant object about 150 meters away and back.

[0108] III. Histogram signal from the photodetector One mode of operation for a lidar system is time-correlated single-photon counting (TCSPC), which is based on counting single photons in a periodic signal. This technique works well for low-level periodic radiation, which is preferred in lidar systems. This time-correlated counting can be controlled by the periodic signal 560 in FIG. 5 and can use time bins, as discussed in FIG. 5.

[0109] The frequency of the periodic signal may specify the time resolution at which the signal's data values ​​are measured. For example, one measurement may be taken for each optical sensor for each cycle of the periodic signal. In some embodiments, the measurement may be the number of optical detectors triggered during that cycle. The period of the periodic signal corresponds to a time bin, with each cycle being a different time bin.

[0110] FIG. 6 illustrates a histogram 600 according to an embodiment of the present invention. The horizontal axis corresponds to time bins measured relative to a start time 615. As described above, the start time 615 may correspond to the start time of a pulse train. Any offset between the rising edge of the first pulse in the pulse train and the start time of either or both of the pulse train and the detection time interval can be considered to determine the reception time used for the time-of-flight measurement. The vertical axis corresponds to the number of triggered SPADs. In certain embodiments, the vertical axis may correspond to the output of an ADC following an APD. For example, the APD may exhibit traditional saturation effects, such as a constant maximum signal, rather than the dead-time-based effects of the SPAD. Some effects may occur in both SPADs and APDs, for example, pulse smearing on highly tilted surfaces may occur in both SPADs and APDs.

[0111] Each counter in a time bin corresponds to a different bar in histogram 600. The counters in the early time bins are relatively low and correspond to background noise 630. At some point, a reflected pulse 620 is detected. The corresponding counter may be much larger and exceed the threshold that distinguishes the detected pulse from the background. The reflected pulse 620 (after digitization) is shown corresponding to four time bins, which may be due to laser pulses of similar width, e.g., a 4 ns pulse if the time bins are 1 ns each. However, as explained in more detail below, the number of time bins may vary based on the characteristics of a particular object, e.g., the angle of incidence of the laser pulse.

[0112] The temporal position of the time bin corresponding to the reflected pulse 620 can be used to determine the time of receipt relative to, for example, the start time 615. As described in more detail below, a matched filter can be used to identify pulse patterns, which can effectively increase the signal-to-noise ratio but also allow for a more accurate determination of the time of receipt. In some embodiments, the accuracy with which the time of receipt is determined can be less than the temporal resolution of a single time bin. For example, for a 1 ns time bin, that resolution would correspond to approximately 15 cm. However, it may be desirable to have an accuracy of only a few centimeters.

[0113] Thus, detected photons can result in a particular time bin of the histogram being incremented based on their arrival time relative to the start signal, as indicated by start time 615, for example. The start signal can be periodic, such that multiple pulse trains are transmitted during a measurement. Multiple start signals can also be synchronized to the laser pulse train, causing multiple pulse trains to be transmitted over multiple detection intervals. Thus, a time bin (e.g., 200-201 ns after the start signal) will occur for each detection interval. The histogram can accumulate counts, with the count for a particular time bin corresponding to the sum of all measurement data values ​​occurring within that particular time bin across multiple shots. When detected photons are histogrammed in this manner, a return signal is produced that has a signal-to-noise ratio greater than that from a single pulse train by the square root of the number of shots that occurred.

[0114] FIG. 7 illustrates a histogram accumulation over multiple pulse trains for a selected pixel, according to an embodiment of the present invention. FIG. 7 shows three detected pulse trains 710, 720, and 730. Each detected pulse train corresponds to a transmitted pulse train with the same pattern of two pulses separated by the same amount of time. Thus, each detected pulse train has the same pulse pattern, as indicated by the two time bins having appreciable values. For simplicity, the counters for the other time bins are not shown, although other time bins may have relatively low, non-zero values.

[0115] In the first detected pulse train 710, the counters for time bins 712 and 714 are the same. This may be due to the same number of detectors detecting photons during the two time bins. Alternatively, in other embodiments, approximately the same number of photons are detected during the two time bins. In other embodiments, more than one consecutive time bin may have consecutive non-zero values, but for ease of explanation, each non-zero time bin is shown.

[0116] Time bins 712 and 714 occur 458 ns and 478 ns, respectively, after start time 715. The counters shown for the other detected pulse trains occur in the same time bins relative to their respective start times. In this example, start time 715 is identified as occurring at time 0, but the actual time is arbitrary. The first detection interval for the first detected pulse train may be 1 μs. Thus, the number of time bins measured from start time 715 may be 1,000. After this first detection interval ends, a new pulse train may be transmitted and detected. The start and end of the different time bins may be controlled by a clock signal, which may be part of a circuit operating as a time-to-digital converter (TDC), as described in FIG. 5, for example.

[0117] For the second detected pulse train 720, the start time 725 is at 1 μs, for example, when the second pulse train may be emitted. Because these distinct detection intervals may occur such that all pulses transmitted at the beginning of the first detection interval have already been detected, there is no confusion regarding the pulses detected in the second time interval. For example, if there is no extra time between shots, the circuit may confuse a retroreflective stop sign at 200 m with a much less reflective object at 50 m (assuming a shot duration of approximately 1 μs). The two detection time intervals for pulse trains 710 and 720 may be the same length and have the same relationship to their respective start times. Time bins 722 and 724 occur at the same relative times, 458 ns and 478 ns, as time bins 712 and 714. Thus, when an accumulation step occurs, the corresponding counters may be incremented. For example, the counter values ​​in time bins 712 and 722 may be incremented together.

[0118] For the third detected pulse train 730, the start time 735 is at 2 μs, for example, when the third pulse train may be emitted. Time bins 732 and 734 also occur at 458 ns and 478 ns relative to the respective start time 735. For example, due to the stochastic nature of the scattering process of light pulses leaving an object, the counters in different time bins may have different values ​​even if the emitted pulses have the same power.

[0119] Histogram 740 shows the accumulation of counters from three detected pulse trains in time bins 742 and 744, which also correspond to 458 ns and 478 ns. Histogram 740 may have a fewer number of time bins measured during each detection interval or having values ​​below the threshold, for example, as a result of time bins falling at the beginning or end. In some implementations, approximately 10-30 time bins may have appreciable values, depending on the pattern of the pulse train.

[0120] By way of example, the number of pulse trains emitted during a measurement to create a single histogram can be approximately 1 to 40 (e.g., 24), but can be much higher, e.g., 50, 100, or 500. Once the measurement is complete, the histogram's counter can be reset, and a new set of pulse trains can be emitted to perform a new measurement. In various embodiments, measurements can be performed every 25, 50, 100, or 500 μs, depending on the number of detection intervals in their respective durations. In some embodiments, measurement intervals may overlap, e.g., so that a given histogram corresponds to a particular sliding window of pulse trains. In such an example, there may be memory for storing multiple histograms, each corresponding to a different time window. Any weighting applied to the detected pulses may be the same for each histogram, or such weighting may be individually controlled.

[0121] IV. Histogram Datapath FIG. 8 shows a circuit for receiving photons and generating a set of signals stored in memory representing a histogram, according to some embodiments. As described above in connection with FIG. 5, an array of photosensors can be used to receive reflected pulses and background photons from ambient light in an optical measurement system. A single photosensor 802 can include multiple photodetectors. Each photodetector can be implemented with a SPAD or other photosensitive sensor, and the photodetectors can be arranged in a grid pattern for the photosensor 802, as illustrated in FIG. 8. A filter can be used for each photosensor to block light outside of a range centered on the light source of the lidar system from being received by the photosensor. However, even with this filter, some ambient light at or near the emission wavelength of the light source may pass through the filter. This can result in photons emitted from the lidar light source being received by the photosensor along with photons from the ambient light.

[0122] Each photodetector in optical sensor 802 may include analog front-end circuitry for generating an output signal indicating when a photon is received by the photodetector. For example, referring back to FIG. 5 , avalanche current 534 from the SPAD may trigger threshold circuit 540 to generate output binary signal 545. Returning to FIG. 8 , each photodetector in optical sensor 802 may generate its own signal corresponding to a received photon. Thus, optical sensor 802 may generate a set of signals 816 corresponding to the number of photodetectors in optical sensor 802. Optical sensor 802 may also be referred to as a “pixel” or “pixel sensor” because it may correspond to a single pixel of information when displayed or analyzed in later stages of the optical measurement system. When a signal is generated in response to a received photon (e.g., a transition from a logical “0” to a logical “1”), this may be referred to as a “positive” signal.

[0123] An arithmetic logic unit (ALU) 804 may be used to implement the functionality of pixel counter 550 from FIG. 5. Specifically, ALU 804 may receive set of signals 816 from individual photodetectors of photosensor 802 and tally the number of these signals, each indicating the detection of a photon. ALU 804 may include combinatorial digital electronic circuitry that performs arithmetic and / or other bitwise operations on set of signals 816. For example, ALU 804 may receive each of set of signals 816 as a binary signal (i.e., “0” or “1”) as an input or operand to ALU 804. By tallying or adding the inputs together, ALU 804 may count the number of positive signals in set of signals 816 that indicate a photon was received within a particular time bin. For example, by adding each of the signals that indicate a “1” signal level, the output of ALU 804 may indicate the number of signals in set of signals 816 associated with photodetectors that sequentially received a photon during the time bin.

[0124] ALU 804 is specially designed to receive at least a number of inputs corresponding to the number of photodetectors in photosensor 802. In the example of FIG. 8, ALU 804 may be configured to receive 32 parallel inputs that are single-bit wide. Internally, ALU 804 may be implemented with digital logic gates to form a ripple-carry adder, a carry-lookahead adder, a carry-save adder, and / or any other type of adder that can sum a relatively large number of inputs with low propagation time. The output of ALU 804 may be referred to as a "total signal count" and may be represented as an n-bit binary number output from ALU 804 or a stage of ALU 804.

[0125] As described above, the output of ALU 804 may characterize the total number of photons received by photosensor 802 during a particular time bin. Each time ALU 804 completes an aggregation operation, the total signal count may be added to a corresponding memory location in memory 806 representing histogram 818. In some embodiments, memory 806 may be implemented using SRAM. Thus, over the course of multiple shots (each shot including a pulse train), the total signal count from ALU 804 may be aggregated with existing values ​​in corresponding memory locations in memory 806. A single measurement may consist of multiple shots that populate memory 806 to generate a histogram 818 of values ​​in time bins that may be used to detect reflected signals, background noise, peaks, and / or other signals of interest.

[0126] The ALU 804 may also perform a second aggregation operation, adding the total signal count to an existing value in a memory location of the memory 806. Recall from FIG. 7 that with each shot, a new total signal count may be added to an existing value in the corresponding time bin of the memory 806. In this way, the histogram 818 may be gradually built up in the memory 806 over several shots. As the total signal count is generated by the ALU 814, a current value 820 of the corresponding memory location of that time bin may be obtained from the memory 806. The current value 820 may be provided as an operand to the ALU 804, which may be combined with the total signal count from the set of signals 816. In some embodiments, the ALU 804 may be comprised of a first stage and a second stage, where the first stage calculates the total signal count from the photosensor 802 and the second stage combines the total signal count with the current value 820 from the memory location of the time bin in the memory 806. In some embodiments, the aggregation of the set of signals 816 and the aggregation of the total signal count and current value 820 may be performed as a single operation. Thus, although these two operations may be described as functionally separate "aggregations," they may actually be implemented together using a combination of parallel and serial circuitry within ALU 804.

[0127] As described above in connection with FIG. 5 , the ALU 804 may receive a periodic signal 560 that triggers the counting operation. The periodic signal 560 may be generated using any of the techniques described above. The periodic signal 560 may define the length of each time bin. In some embodiments, the periodic signal 560 and corresponding time bins may be measured relative to a start signal, as illustrated in FIG. 3 . Each cycle of the periodic signal 560 may cause the ALU 804 to perform the counting operation and increment the memory address of the memory 860 to the next time bin. For example, a rising edge of the periodic signal 560 may cause the ALU 804 to generate a result that counts the total signal count and the current value 820 together. The corresponding periodic signal 808 may also be sent to a memory interface circuit that increments the address to the memory location of the current time bin, such that each cycle moves to the next time bin in the memory 806.

[0128] A clock circuit 810 may be used to generate the periodic signal 560 based on inputs defining shots and measurements of the optical measurement system. For example, the shot input 814 may correspond to the start signal illustrated in FIG. 3 . The shot input 814 may reset the address of the memory 806 to a starting memory location corresponding to the first time bin of the histogram 818. The shot input 814 may also cause the clock circuit 810 to begin generating the periodic signal 560 of the ALU 804 and / or the periodic signal 808 that increments the address of the memory 806. Additionally, the clock circuit 810 may receive a measurement input 812 that defines the start / end of a measurement. A measurement may consist of multiple shots that gradually build the histogram 818. The measurement signal 812 may be used to reset the values ​​in the memory 806 so that the histogram can start over with each new measurement.

[0129] The memory 806 may include multiple registers that accumulate photon counts from the photodetectors. By accumulating photon counts in respective registers corresponding to time bins, the registers in the memory 806 may store photon counts based on the arrival time of the photon. For example, photons arriving in a first time bin may be stored in a first register in the memory 806, photons arriving in a second time bin may be stored in a second register in the memory 806, and so on. Each "shot" may include one traversal through each of the registers in the memory 806 that correspond to the time bins of that photosensor. The shot signal 814 may be referred to as an "enable" signal for the multiple registers in the memory 806, in that the shot signal 814 enables the registers in the memory 806 to store the result from the ALU 804 during the current shot.

[0130] The periodic signal 560 may be generated so as to be configured to capture the set of signals 816 as they are provided asynchronously from the photosensor 802. For example, the threshold circuit 540 may be configured to hold its output signal high for a predetermined time interval. The periodic signal 560 may be timed so that it has a period that is less than or equal to the hold time of the threshold circuit 540. Alternatively, the period of the periodic signal 560 may be a percentage of the hold time of the threshold circuit 540, such as 90%, 80%, 75%, 70%, 50%, 110%, 120%, 125%, 150%, 200%, etc. Some embodiments may use a rising edge detection circuit, such as that illustrated in FIG. 5, to convert the asynchronous signal from the photodetector into a single-clock strobe that uses the same clock that runs the ALU 804. This may ensure that photons are not counted more than once. Other embodiments may alternatively oversample the asynchronous pulses from the photodetector or use brute-force rising edge detection.

[0131] 9 illustrates the timing associated with different shots in a measurement, according to some embodiments. The vertical axis represents the total number of photodetector signals measured by a single optical sensor. For example, the vertical axis may represent total photon counts. The horizontal axis represents time. The first shot may begin with the receipt of shot signal 814-1 as the starting signal when a new optical measurement begins, as indicated by measurement signal 812. Periodic signal 506 illustrates how each clocking of ALU 804 corresponds to a single time bin and corresponding memory location in memory 806.

[0132] With each subsequent shot, a histogram may be built in memory 806, as illustrated in FIG. 9. Each time a shot input 814 is received, the addressing of memory 806 is reset, thereby adding a new total signal count to the existing signal count. In the example of FIG. 9, there is a non-zero time interval separation between each shot. For example, a non-zero time interval elapses before the shot initiated by shot signal 814-1 ends and the shot defined by shot signal 814-2 begins. Alternatively, some embodiments may not have a delay between subsequent shots, such that the periodic signal 506 clocks continuously throughout the entire measurement. Subsequent shot signals 814-2, 814-3 may then define both the end of the previous shot and the start of the subsequent shot.

[0133] In some embodiments, the timing of the measurement signal 812, the shot signal 814, and the periodic signal 506 that clocks the ALU 804 may all be generated in a coordinated manner. Thus, the clocking of the ALU may be triggered by and dependent on the start signal of each shot. Additionally, the period of the periodic signal 506 may define the length of each time bin associated with each memory location in the histogram.

[0134] The data path illustrated in FIG. 8 is primarily configured to build the histogram 818 over several shots. However, the histogram may, in some embodiments, be populated only during the shot. Any photons received between shots will not be aggregated into time bins by the ALU 804 and stored in the memory 806. Additionally, the contents of the memory 806 may be reset after each measurement is completed and before the next measurement begins. Thus, photons received before and / or after a current measurement may not be saved or immediately available. Furthermore, the total count of all photons received in the histogram is not immediately available in the histogram data path, and if a non-zero interval exists between shots when the histogram is disabled, the total count of all photons received during a measurement cycle is not available. In addition to the histogram data path illustrated in FIG. 8, a second parallel data path may be used to record received photons in a continuous manner independent of shot / measurement timing.

[0135] V. Identifying early reflections from the system housing The above examples illustrate a single pulse or multiple pulses as part of a pulse train transmitted from a light source, reflected from objects in the surrounding environment, and then detected by an optical sensor. These reflections are recorded in a histogram memory, and after several repeated shots in the measurement, form “peaks” in the histogram. The locations of these peaks can then be used to determine the distance between the optical measurement system and objects in the surrounding environment. However, these examples are not meant to be limiting. In addition to the intended peaks caused by the pulse train reflecting from objects of interest in the surrounding environment, the histogram memory may also include other peaks resulting from more immediate, unintended reflections of the pulse train and peaks not necessarily resulting from the pulse train. For example, additional peaks may correspond to external light sources, sunlight, reflections, or other ambient light phenomena in the surrounding environment. In these cases, instead of detecting only a single peak, the optical measurement system may detect multiple peaks that may result in the histogram memory. In addition to extraneous peaks resulting from external light sources, an initial peak may result from reflections of the pulse train from the internal housing of the optical measurement system.

[0136] FIG. 10 illustrates a representation of a histogram memory including a peak 1002 resulting from reflection from the housing of an optical measurement system. When a light source generates one or more pulses for an emitted pulse train, these pulses may pass through a window or other transparent material within the housing of the optical measurement system. However, some of the light emitted by the light source may reflect from the window of the housing back to the light sensor instead of passing completely through the window. In addition, some of the light emitted by the light source may reflect from the interior surface of the housing itself. For example, some incident light may reflect from portions of the housing surrounding the transparent window or from other internal materials of the optical measurement system. A portion of the light reflected from the housing of the optical measurement system may be received by the light sensor and stored in the histogram memory in the same way that reflected pulses are detected and stored from objects of interest in the surrounding environment.

[0137] In FIG. 10 , peaks 1002 resulting from early reflections of an emitted pulse train from the optical measurement system housing may be stored in an initial set of registers in a histogram memory. Because the reflections occur immediately after the pulse train is emitted from the optical measurement system's light source, the photon counts from these early reflections may be stored in the first register in the histogram memory for the measurement. For example, FIG. 10 illustrates peaks 1002 corresponding to early reflections from the system housing stored in the first nine registers of the histogram memory. However, this example is not meant to be limiting. The number and location of time bins representing reflections from the system housing may depend on the intensity of the light pulse, the distance between the housing and the light source, the distance between the housing and the photodetector, and / or other characteristics of the physical design of the optical measurement system. Methods for determining the location of peaks 1002 resulting from reflections from the system housing are discussed in more detail below.

[0138] As photon counts are received for the remainder of the optical measurement, additional peaks resulting from photon reflections from objects of interest in the surrounding environment may also be detected. For example, peak 1004 in FIG. 10 may be received later in the measurement after reflecting from an external object within range of the optical measurement system. Typically, these objects of interest in the surrounding environment are far enough away from the optical measurement system that the resulting peak 1004 can be easily distinguished from peak 1002 reflected from the system housing. Some embodiments may ignore or remove peak 1002 from the histogram memory and / or from any calculations performed on the histogram memory to calculate the distance to the external object corresponding to peak 1004. For example, some embodiments may mask out an initial set of histogram registers that are likely to contain a large peak 1002 from the housing reflection. Some embodiments may insert a delay between when the pulse train is emitted by the light source and when the light sensor begins accumulating photon counts in the histogram memory to avoid recording peak 1002 from the housing reflection. Because peak 1002 occurs very early in the measurement compared to when peak 1004 is received, peak 1002 can often be ignored when using peak 1004 to perform distance calculations.

[0139] However, some embodiments may receive and store the peaks 1002 resulting from the housing reflection in a histogram memory and then use the peaks to improve the near-field performance of the optical measurement system. These improvements may include calibrating the optical measurement system, characterizing the opacity of a window in the optical measurement system, detecting blockages in the optical measurement system, improving the near-field performance of the optical measurement system, etc. Each of these various improvements that may be realized by first identifying the peaks 1002 resulting from the housing reflection is described in more detail below.

[0140] FIG. 11 illustrates a flowchart of a method for detecting peaks resulting from initial reflections from the system housing using an optical measurement system. The method can be used to transmit / receive pulses and identify specific locations in a histogram memory that correspond to peaks from the housing reflections. The following sections of this disclosure describe different ways in which the identified peaks can be used to improve the performance of the optical measurement system. The following method describes a process for a single optical sensor in an optical measurement system. However, as noted above, an optical measurement system may include many optical sensors, and the method can be performed for each optical sensor in the optical measurement system.

[0141] In step 1102, the method may include transmitting one or more pulse trains from the light source over one or more first time intervals as part of the optical measurement. Each of the one or more first time intervals may represent a "shot" that is repeated multiple times in the measurement. Each of the first time intervals may include one or more pulse trains that are coded and transmitted by the light source so that the pulse trains can be recognized when reflected from objects in the surrounding environment. Each of the time intervals may be subdivided into multiple time bins, whereby each time bin represents a bin of a histogram of photon counts received during the optical measurement. An example of how a single measurement may include multiple shots subdivided into time bins that aggregate photon counts is described above in connection with FIG. 9.

[0142] In step 1104, the method may also include detecting photons from the one or more pulse trains using an optical sensor. As described in detail above, the optical sensor may include multiple photodetectors, such as multiple SPADs. The optical sensor may receive reflected light received from the surrounding environment and ambient background noise. The reflected light received by the optical sensor may include light reflected from objects of interest in the surrounding environment. For example, these objects of interest may be at least 30 cm away from the optical sensor and may represent surrounding vehicles, buildings, pedestrians, and / or any other objects that may be encountered around the optical measurement system during use. These objects of interest may be distinguished from objects that are part of the optical measurement system itself, such as the housing. The reflected light received by the optical sensor may also include light reflected from the housing or other parts of the optical measurement system. These reflections may include direct primary reflections from a window or housing of the optical measurement system and secondary reflections reflected around the interior of the optical measurement system. The optical sensor may also be coupled to a threshold detection circuit and / or an arithmetic logic circuit that accumulates photon counts. This combination may be referred to above as a "pixel." FIG. 5 illustrates one example of how photon counts may be received from a photosensor and counted using a threshold circuit and a pixel counter (eg, an arithmetic logic circuit).

[0143] In step 1106, the method may also include accumulating photon counts from the optical sensor in multiple registers to represent a histogram of photon counts received during the current measurement. These photon counts may be accumulated in multiple registers in a memory block corresponding to the optical sensor. The multiple registers may be implemented using registers in the SRAM of the histogram data path, as described above in FIGS. 8 and 9. Each time interval may be subdivided into multiple first time bins for the histogram. Corresponding time bins in each of the one or more time intervals may be accumulated in a single register in the multiple registers. Each time interval may represent a shot, and together the one or more time intervals may represent a measurement of the optical measurement system. Each time interval may be defined by a start signal or shot signal that resets the memory representing the histogram back to the first register of the histogram. Received photons reflected from the optical measurement system housing may be stored in the histogram memory in the same manner as other received photons reflected from ambient objects outside the optical measurement system.

[0144] In step 1108, the method may further include identifying a peak in the histogram representing photons reflected from the system housing. Detecting this peak may be performed using several different techniques, depending on the particular embodiment. In some embodiments, a pass may be made through the histogram memory to identify the first peak occurring in time. The system may sequentially access registers in the plurality of registers to identify the first peak occurring in time, starting from the beginning of the optical measurement. The center of the peak may be identified by identifying values ​​in the histogram having smaller values ​​in the time bin on either side. In the example of FIG. 10 , the register representing the fourth and fifth time bins may be identified as a peak because the time bin values ​​in the registers on either side of the fourth and fifth registers have significantly smaller values ​​stored therein. In some embodiments, an initial peak may be identified by identifying a register in the plurality of registers during a maximum number of photon counts. These embodiments may assume that the initial peak may have a higher intensity than other peaks. Some embodiments may also identify the initial peak by identifying a register in the plurality of registers corresponding to the distance between the light source and the housing of the optical measurement system. For example, when this distance is known, the distance between the light sensor and the housing may be used to identify a register that may receive the peak reflected from the housing. Some embodiments may identify the initial peak by identifying a predetermined number of registers in the plurality of registers that first occur. For example, based on an initial calibration and a known distance between the light sensor and the housing, some embodiments may identify the first 15 time bins as storing the initial peak.

[0145] Identifying a peak in a histogram may involve locating a window of time bins that occurs near the center of the peak. This process may involve extending outward from the peak location to determine a range of registers in the histogram memory that includes the entire peak. As illustrated in FIG. 10 , photons resulting from reflection from the system housing may be received across multiple time bins rather than being isolated to a single time bin. Accordingly, some embodiments may identify surrounding time bins that should also be designated in representing the reflection peak from the system housing. For example, some embodiments may identify a predetermined number of time bins around a maximum value that may be designated as a peak. For example, a predetermined number of time bins, such as three time bins, five time bins, nine time bins, fifteen time bins, seventeen time bins, etc., may be identified and centered around the maximum value to represent the entire peak. Some embodiments may identify surrounding time bins having values ​​within a percentage of the maximum value in the peak register. This may result in a varying number of time bins that may be used to represent the peak, depending on the width of the peak. For example, time bins surrounding a maximum value may be included in the peak if their values ​​are within 25% of the maximum value.

[0146] Instead of detecting a variable peak location for each measurement, some embodiments may assume a known location of the optical measurement housing and designate a specific window of registers that are likely to contain values ​​from early reflections from the system housing. In the example of FIG. 10, the system may designate registers corresponding to time bins 2 through 6 as registers that store peaks related to reflections from the system housing. Physical measurements may be taken to determine the distance between the light source and the housing and the distance between the housing and the light sensor. This distance may then be used in combination with the speed of light to determine where in the histogram memory reflections from the system housing are likely to occur.

[0147] It should be understood that the specific steps illustrated in FIG. 11 provide a particular method for identifying peaks representing reflections from a system housing, according to various embodiments. Other sequences of steps may be performed according to alternative embodiments. For example, alternative embodiments of the present invention may perform the steps outlined above in a different order. Furthermore, individual steps illustrated in FIG. 11 may include multiple sub-steps that may be performed in various sequences appropriate to the individual step. Furthermore, additional steps may be added or removed depending on the particular application. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0148] VI. Using Housing Reflection to Improve Measurements After identifying the location of the pulse representing the reflection from the system housing, the location and / or peak value of that reflection can be used to improve or characterize the optical measurement system in several different ways, including calibrating distances known to us in additional distance measurements by the optical measurement system, characterizing the transparency of windows to the optical measurement system, detecting occlusions or other obstacles that obstruct the field of view of the optical measurement system, and / or other similar improvements.

[0149] A. Calibration using housing reflection FIG. 12 illustrates the contents of a histogram memory that may be used to calibrate distance measurements for an optical measurement system, according to some embodiments. As described above, the histogram may include a peak 1002 corresponding to a reflection from the housing of the optical measurement system. The histogram may also include a peak 1004 resulting from a photon reflection from an object of interest in the surrounding environment. Typically, the center location of the peak 1004 is identified and used to calculate the distance between the optical measurement system and the object from which the peak 1004 was reflected. This distance calculation may use the speed of light along with a known time interval for each of the time bins represented by the histogram register. For example, for a system operating at 500 MHz, the resulting time bin may represent approximately 2 ns of received photon counts. The distance of the peak 1004 may be calculated using the number of time bins (e.g., 32) multiplied by the time of each time bin (e.g., 2 ns) multiplied by the speed of light.

[0150] However, if any of the values ​​assumed above are inaccurate or vary between different optical measurement systems, the distance calculations may be inaccurate. For example, the actual time represented by each time interval may not exactly match that predicted based on the assumed clock cycle. Therefore, it may be useful in some systems to implement a self-calibration process that can be used to accurately determine the different constants used by the distance calculations.

[0151] Some embodiments may use the location of peak 1002, which represents the housing reflection, to calibrate values ​​for other distance calculations of the optical measurement system. The distance between the light source and the housing of the optical measurement system may be known precisely based on the manufacturing process of the optical measurement system. Similarly, the distance between the housing and the optical sensor of the system may also be known precisely. These distances may be used in combination with peak 1002 to determine precise values ​​for constants used in the distance calculations, such as the time represented by each time bin and / or any offsets in the system. These constants may be part of a transformation function that receives the time of the peak and provides an accurate distance measurement. Such a transformation may be constant (e.g., a uniform delay), linear (e.g., later time bins are offset more or less than earlier time bins by an amount proportional to the time bin number), or nonlinear.

[0152] In the example of FIG. 12 , the location of peak 1002 can be determined using any of the techniques described above. The location of peak 1002 can be used to determine the number of time bins 1202 between the start of the measurement and the center of the peak from the housing. The known distance between the light source, housing, and light sensor described above can be divided by the speed of light to determine the total time between when light is emitted from the light source and when peak 1002 occurs in the histogram memory. This value can then be divided by the number of time bins 1202 to calculate the exact amount of time represented by each time bin. Note that this procedure assumes that the histogram memory begins accumulating photon counts in a register when light is emitted from the light source. If this is not the case, any delay between the emission of light from the light source and the start of an accumulation cycle by the histogram memory can be subtracted from the time estimate.

[0153] The exact value of the constant, such as the time represented by each time bin, can then be used to calculate the distance of the other peaks in the measurement. In the example of Figure 12, the distance between the optical measurement system and the object represented by peak 1004 can be calculated as above, but instead of using the time value assumed from the clock cycle, the system can instead calculate the distance to the object using the calibrated value calculated using peak 1002.

[0154] Therefore, using calibrated values ​​from the peak reflected from the housing of the optical measurement system can be used to generate more accurate distance measurements. These techniques can overcome problems resulting from process variations and sensor drift over time. These techniques can also be used to detect movement of the system housing relative to the light source and / or light sensor. For example, if the location of peak 1002 changes over the lifecycle of the optical measurement system, this can indicate movement of the housing relative to the rest of the system.

[0155] B. Transparency characteristics of windows In addition to calibrating values ​​used in distance calculations, identifying the location of reflections from the system housing can also be used to characterize various aspects of an operating optical measurement system. In general, the peak reflected from the system housing may be larger than the peak received from an object of interest in the surrounding environment. The size of this initial peak may be due to the proximity of the housing to the optical sensor. Because the optical sensor is so close to the reflection, the intensity of the reflected light may be relatively large. However, assuming the size of the reflected peak is within the saturation limit of the histogram memory, specific measurements can be made based on the magnitude of the peak reflected from the housing to characterize portions of the housing itself.

[0156] FIG. 13A illustrates a portion of a histogram memory receiving an initial peak 1302 corresponding to reflection from a system housing, according to some embodiments. The peak 1302 may have an initial magnitude 1308. This magnitude may characterize the transparency of a window on the housing of the optical measurement system. As described above, the housing may include a window through which light may be transmitted from a light source and through which reflected light may be received by a light sensor. Assuming the transparency of the remainder of the housing is substantially opaque, the magnitude 1308 of the peak 1302 may be used to characterize the transparency of the window itself. For example, baseline transparency may be characterized during the initial portion of the life cycle of the optical measurement system by recording the average magnitude 1308 of the housing reflection. This value may be stored over time and used to detect changes in the transparency of the window.

[0157] Many practical applications of optical measurement systems may include use in open environments where the optical measurement system may be exposed to pollution, rain, snow, and / or other environmental elements that may damage, obscure, and / or otherwise affect the transparency of the window housing the optical measurement system. For example, when an optical measurement system is installed in an automotive application, the system may be located on top of the vehicle. The system may also be located in an area surrounding the vehicle's periphery, such as the bumper or side of the vehicle. In these locations, the optical measurement system may be subjected to weather elements, rocks and debris, exhaust or fog, and / or other influences that may affect the transparency of the window. For example, a rock from the road may strike the optical measurement system and scratch or otherwise damage the window. A humid environment may cause moisture to temporarily accumulate on the window. Rain or snow may accumulate on the outside of the window. Dirt, pollution, or dust from the road may accumulate on the outside of the window. Many other adverse conditions in an automotive environment may also affect the window of an optical measurement system.

[0158] Each of these different environmental influences can change the transparency of the window. For example, as dirt or scratches accumulate on the window, less light may be transmitted through the window. As a result, more light may be reflected from the window back to the light sensor. Therefore, considering the window to be part of the housing of the optical measurement system, the techniques described above for monitoring the magnitude of the peak reflected from the housing of the optical measurement system over time can also be used to track differences in the transparency of the window over time. The transparency of the rest of the housing can be assumed to remain constant (e.g., the housing is completely opaque). Therefore, any change in the magnitude of the peak reflecting from the housing can be attributed to changes in the transparency of the window itself.

[0159] FIG. 13B illustrates the change in magnitude of peak 1304 reflected from a housing, according to some embodiments. Peak 1304 may be recorded from the same optical measurement system that recorded peak 1302 in FIG. 13A, but peak 1304 may be recorded later in the life cycle of the optical measurement system. For example, peak 1304 may be recorded after several months of use in an automotive application. When comparing magnitude 1310 of peak 1304 to magnitude 1308 of peak 1302, FIG. 13B illustrates how the magnitude of peak 1304 has increased over time. The increase may be due to a change in the transmission characteristics of a window on the housing of the optical measurement system.

[0160] As described above, the magnitude 1308 of the initial peak 1302 may be recorded as a baseline measurement when the housing window is clean. This initial value may be stored over time and used as a baseline for comparison to future peak magnitudes to track window transparency over time. For example, some embodiments may track the difference 1306 between the initial magnitude 1308 and the current magnitude 1310 of the peaks 1302, 1304 over time. As the difference 1306 increases, various outputs may be provided by the optical measurement system. For example, some embodiments may provide an output that triggers a warning or message to a user or vehicle system indicating that the window is dirty ("Please clean the lidar window"). Some embodiments may automatically trigger a system on the vehicle to clean the optical measurement system, such as a system that sprays a detergent on the optical measurement system.

[0161] FIG. 14 illustrates a portion of an array of optical sensors 1402 that may be partially obscured by window contamination, according to some embodiments. As described above, these methods for detecting reflected pulses from the housing of an optical measurement system may be implemented for each optical sensor in the array of optical sensors 1402. Therefore, each individual optical sensor may generate a baseline magnitude of the reflected peak and a current magnitude of the reflected peak. Each individual optical sensor may then report the percentage or degree to which the corresponding portion of the window is obscured. The transparency of each optical sensor may be characterized relative to one of the optical sensors or relative to the baseline level of transparency. In this example, a portion 1404 of the window may be covered with contaminants such as mud. Individual optical sensors transmitting / receiving light through the portion 1404 of the window may report an increase in the magnitude of the reflected pulse, while optical sensors outside the portion 1404 of the window may continue to report a stable magnitude of the reflected pulse. This may be used to generate a mapping or characterization of the window. For example, the image of FIG. 14 may be generated and provided to a user interface illustrating the current state of the window. A message may be generated characterizing the window as dirty, partially obscured, or completely blocked.

[0162] In some embodiments, the optical measurement system may shut down light sensors behind a dirty or blocked portion 1404 of the window, or in the case of a rotating system, light sensors may be shut down when they are behind portion 1404. Due to the intensity of the reflected pulses, the histogram data paths of these light sensors may saturate such that the magnitude of the pulse reflected from the window has a magnitude greater than the bit limit of the histogram data path. Reflections from the window in this portion 1404 may also be scattered by the blockage and interfere with photons received by other nearby light sensors. Because saturation, scattering, or other effects may distort measurements, the system may take corrective action, such as preventing these light sensors from accumulating photon counts.

[0163] C. Window Blockage Detection In addition to detecting dirty or partially obscured windows, the initially detected peak can also be used to determine whether the optical measurement system is obstructed by a foreign object. As noted above, many operating environments, such as an automotive environment, can provide situations in which the optical measurement system can be exposed to adverse conditions. Some of these environments can completely obstruct the optical measurement system from the surrounding environment. For example, a plastic bag may be sprayed onto the optical measurement system. A person may place their hand in front of the optical measurement system. The optical measurement system may be intentionally disabled by tape or other object placed in front of the optical measurement system. In any of these environments, it can be useful to detect when such obstructions occur and where such obstructions may be located.

[0164] 15A illustrates an initial peak 1502 detected by an optical measurement system relative to an occlusion threshold 1506, according to some embodiments. The initial peak 1502 may be the result of a reflection from the housing of the optical measurement system, as described above. Based on this baseline measurement of the magnitude of the peak 1502, an occlusion threshold 1506 may be established that is higher than the magnitude of the peak 1502. The occlusion threshold 1506 may represent a magnitude for the peak 1502 that may indicate an occlusion in front of the optical measurement system. The occlusion threshold 1506 may be set to be a predetermined distance and / or percentage above the magnitude of the initial peak 1502 so that it is crossed only when an object is in front of the optical measurement system.

[0165] 15B illustrates an initial peak 1504 later in the life cycle of an optical measurement system when an obstruction is present, according to some embodiments. When an object is placed in front of the optical measurement system, a large portion of the pulse may be instantly reflected from the obstructing object. If the obstructing object is very close to the optical measurement system, the resulting peak resulting from the reflection from the obstructing object may merge with other reflections from the housing of the optical measurement system, as described above. In this example, the obstructing object is close enough to the optical measurement system that the reflection from the housing and the reflection from the obstructing object are received together, resulting in a peak 1504 that is substantially higher than the peak 1502 resulting from the reflection from the housing alone.

[0166] When an occluding object is sufficiently close to the optical measurement system, thereby preventing a significant portion of the light emitted by the optical measurement system from being reflected off other objects in the surrounding environment, the system may determine that an occlusion has occurred. This determination may be made when the magnitude of the initial peak 1504 meets or exceeds an occlusion threshold 1506. As illustrated in FIG. 15B , reflections from the housing and the occluding object may combine to increase the magnitude of the peak 1504 above the threshold 1506. When this occurs, any of the corrective actions described above may be triggered. For example, some embodiments may generate a warning to the user and / or the vehicle system. Some embodiments may shut down any optical sensors in the array that detect the occlusion. Some embodiments may activate other emergency or safety systems, such as indicating that the optical measurement system is no longer providing an accurate depiction of objects in the surrounding environment.

[0167] VII. Housing Reflection Compensation The use of peaks generated by reflections from the housing of the optical measurement system described above generally improves the performance of the optical measurement system by calibrating the system and compensating for external effects. Some embodiments may alternatively or additionally improve the performance of the optical measurement system by removing peaks caused by housing reflections from the histogram memory signal analyzed by a peak detection circuit or by an auxiliary processor. These embodiments may remove initial peaks from the housing reflection to improve the near-field accuracy of the optical measurement system.

[0168] A. Near-field detection 16A illustrates a histogram memory having an initial peak 1602 resulting from a housing reflection and a second peak 1604 resulting from a reflection from an object in the surrounding environment, according to some embodiments. As described above, when the second peak 1604 is sufficiently removed from the initial peak 1602, the distance calculation of the second peak 1604 need not be affected by the presence of the initial peak 1602. For example, when an object is 10 feet away from the optical measurement system, the peak resulting from photons reflected from the object may occur far enough away from the initial peak 1602 in the histogram memory that the shape of the second peak 1604 is affected. In general, the mid- and long-range accuracy of the optical measurement system need not be affected by the presence of the initial peak 1602 from the housing reflection.

[0169] However, as the object moves closer to the optical measurement system, residual effects of the initial peak 1602 may begin to affect the shape of the second peak 1604. For example, as photons begin to be received from reflections from the object, scattered photons from the optical measurement system's internal housing may still be received by the optical sensor. Because additional photon counts from the reflected photons may continue to be stored in a register representing the time bin of the second peak 1604, the shape of the second peak 1604 may become distorted. As the shape changes, the peak detection circuit may identify an incorrect location of the peak. For example, if the beginning of the second peak 1604 increases due to photons reflected from the housing, the center of the second peak 1604 may shift to the left as detected by the peak detection circuit.

[0170] FIG. 16B illustrates a problem with near-field peak detection without compensation for reflection from the housing, according to some embodiments. In this example, an object in the surrounding environment moved close enough to the optical measurement system that a second peak 1604 resulting from reflection from the object began to merge with the initial peak 1604 resulting from reflection from the housing. As illustrated in FIG. 16B , this has the effect of changing the size and / or shape of the initial peak 1602, altering or even obscuring the presence of the second peak 1604. Because the size of the second peak 1604 is generally substantially smaller than the size of the initial peak 1602, the shape and / or location of the second peak 1604 may be substantially subsumed by the initial peak 1602 to the point that it may no longer be accurately recognized as a peak by the optical measurement system.

[0171] The overall effect of this combination of the initial peak 1602 and the second peak 1604 is to significantly reduce the close-range accuracy of the optical measurement system. Reflections from objects several feet away from the optical measurement system can be affected by these housing reflections to the point that "dead zones" can occur in the immediate vicinity of the optical measurement system.

[0172] B. Eliminating housing reflections To compensate for the effects of reflections from the housing, the optical measurement system may use a baseline measurement of the initial peak as described above. Specifically, the optical measurement system may record the baseline shape and / or location of the initial peak as observed during measurements over time. In addition to using changes to the initial peak to characterize the window of the optical measurement system, the shape and / or location of the baseline measurement of the initial peak may be used to compensate for reflections from the housing for near-field measurements.

[0173] Figure 17A illustrates the histogram memory from Figure 16B. In addition, Figure 17A also overlays values ​​in registers representing a baseline peak 1702 established by averaging previous measurements. Essentially, baseline peak 1702 represents the magnitude of photon counts that can be attributed to reflections from the housing rather than reflections from nearby objects of interest in the surrounding environment. Because the values ​​of each of these registers at baseline peak 1702 are known, those values ​​can be removed from the histogram registers for the current measurement.

[0174] FIG. 17B illustrates the histogram memory from FIG. 16B after the baseline peak 1702 has been subtracted from the registers in the histogram memory. For example, the register value from the baseline peak 1702 can be subtracted from each of the registers in the histogram memory for the current measurement. The resulting contents of the histogram memory show a second peak 1604 resulting from a near-field object in the surrounding environment. Note that the shape of the second peak 1604 can be preserved by removing the effect of the initial peak 1602. This allows for significant improvements in the near-field performance and accuracy of the optical measurement system. Instead of creating a "dead zone" around the optical measurement system, near-field objects can be accurately detected even when they are very close to the optical measurement system.

[0175] Figure 18 illustrates a circuit for removing the effects of reflections from the system housing during near-field measurements, according to some embodiments. Figure 18 is similar to Figure 8 in that it includes an arithmetic logic circuit 804, a histogram memory 806, and various periodic signals that control the timing of the arithmetic logic circuit 804 and the histogram memory 806. This basic circuit can be used to generate a histogram in the histogram memory 806 over a shot or multiple shots defined by the start signal 814 and the measurement signal 812, as described above.

[0176] The histogram data path illustrated in FIG. 18 also includes a baseline peak 1804 that stores histogram register values ​​for the initial peak resulting from reflections from the housing, as described above. This baseline peak 1804 may be the average peak value recorded over multiple previous measurements. The value at each register location may be averaged over a sliding window of previous measurements to eliminate any outliers, blockages, or other effects that may only temporarily affect the accuracy of the baseline peak 1804. Other mathematical operations may be used to combine multiple initial peaks recorded in previous measurements to generate the baseline initial peak, including maintaining a moving average, identifying minimum or maximum peaks, using a stepping average finder, and other similar techniques.

[0177] The baseline peak 1804 may be provided to the subtraction circuit 1802 so that it can be subtracted from the corresponding register in the histogram memory 806. For example, a memory interface may provide the first nine register values ​​from the histogram memory 806 to the subtraction circuit 1802, and subtract the corresponding value from the baseline peak 1804 from those register values. The register values ​​may then be returned to the histogram memory 806 so that the histogram memory 806 has the effect of reflections from the housing removed from the histogram. The subtraction circuit 1802 may be implemented using an arithmetic logic unit (ALU) or digital logic gates that implement the subtraction function.

[0178] In some embodiments, the baseline peak 1804 may additionally or alternatively be provided to the processor 1806. Instead of using the subtraction circuit 1802 to compensate for reflections from the housing, the baseline peak 1804 may be passed to the processor 1806 along with the raw and / or filtered data in the histogram memory 806, and the processor 1806 may then remove the baseline peak 1804 from the current measurement. The processor 1806 may be on the same integrated circuit as the rest of the optical measurement system, including the histogram memory 806 and the arithmetic and logic circuit 804. Alternatively, the processor 1806 may be located on a separate integrated circuit different from the integrated circuit implementing the optical measurement system. The processor 1806 may receive the filtered and / or raw data from the histogram memory 1806 and then use a set of instructions to perform operations to remove the effects of reflections from the system housing, as described above.

[0179] VIII. Peak Detection Using Matched Filters As described in detail above, the optical measurement system may generate a pulse train emitted from a light source and directed toward the surrounding environment. Light from these pulse trains may reflect off objects in the surrounding environment and be received by corresponding light sensors of the optical measurement system. As photon counts are received by the light sensors, they may be accumulated in registers in a histogram memory (e.g., SRAM) to form a histogram representation of the photons received during the optical measurement. The photon counts accumulated in the histogram memory may represent raw measurement data, which may include reflected photons from the light source and photons from background noise in the surrounding environment. Depending on the noise level of the background environment, this raw data in the histogram memory may have a relatively low signal-to-noise ratio (SNR). This level of noise in the histogram may increase the difficulty associated with performing on-chip peak detection. To improve the reliability of the on-chip peak detection circuitry, the optical measurement system may apply one or more filters to the raw data in the histogram memory, after which a peak detection algorithm using the filtered data may be performed.

[0180] While filtering the data improves the SNR of the data in the histogram memory, it may also mask characteristics of the histogram of reflected photons that may be useful for calculating distance measurements, performing statistical analysis of the histogram data, detecting edge cases, detecting close-range objects, distinguishing between adjacent reflected peaks, curve fitting, etc. The embodiments described below take advantage of filtering the data in the histogram memory and preserving additional information provided by the raw data received from the light sensor. The filtered data can be used to perform on-chip peak detection and then identify time windows in the unfiltered histogram memory that may contain the detected peaks. After the time windows are identified using the filtered data, the integrated circuit in which the histogram memory is implemented can pass the time bins containing the unfiltered histogram data during those time windows to an external processor for distance calculations and other analysis of the raw histogram data.

[0181] The following sections first explain how the filtering process can use a matched filter for a single pulse scenario to detect a time window of unfiltered data passed off-chip. Then, more complex examples are presented using pulse trains with multi-pulse codes and more complex matched filters. Some filters can be designed to compress the data for these longer pulse codes without sacrificing the shape information of the original data.

[0182] A. Long-distance peak detection FIG. 19 illustrates the contents of a histogram memory after a single pulse is transmitted and received by an optical measurement system, according to some embodiments. As described above, the initial time bin in the histogram memory may contain photon counts resulting from an initial pulse reflected from the housing and / or window of the optical measurement system. When the light source of the optical measurement system emits photons into the surrounding environment, a portion of these photons may reflect back to the corresponding light sensor without leaving the housing of the optical measurement system. This initial reflection of photons may result in a relatively large peak 1902 stored in the initial time bin of the histogram memory. Some embodiments may process the contents of the histogram memory so that the initial peak 1902 can be distinguished from subsequent peaks reflected from objects in the surrounding environment. For example, some embodiments may ignore peaks above a predetermined intensity threshold that is likely exceeded only by the initial peak 1902. Some embodiments may ignore peaks occurring in the initial time bins (e.g., in the first 20 time bins). Some embodiments may average multiple signals to estimate the background level of noise, including the initial peak 1902, and this average signal may be subtracted from the measured signal, effectively removing the initial peak 1902. Thus, the initial peak 1902 may be recognized and distinguished from subsequent peaks that represent reflections from objects of interest in the surrounding environment.

[0183] After the initial peak 1902, the histogram memory may subsequently include one or more peaks resulting from photon reflections from objects in the surrounding environment. For example, peak 1904 may occur after the initial peak 1902 and represent photons reflected from an object in the surrounding environment. Both peak 1902 and peak 1904 may result from photons emitted as part of the same pulse train, but peak 1902 may be reflected from a housing / window of the optical measurement system, while peak 1904 may be reflected from an object in the surrounding environment. The shapes of peaks 1902, 1904 may approximately correspond to the shape of the emitted pulse from the optical measurement system. For example, some embodiments may emit pulses having a square shape. As illustrated in FIG. 19 , the photon counts received by the histogram memory may have an approximately square shape corresponding to the emitted pulse shape. Background noise, reflection patterns, and other environmental influences may cause the shape of the received peaks 1902, 1904 to vary somewhat from the ideal square shape of the emitted pulse.

[0184] As noted above, the photon counts of the histogram illustrated in FIG. 19 may contain a nontrivial amount of background noise and other effects that can cause difficulties when identifying peaks in the histogram memory. Some embodiments may include on-chip peak detection circuitry to identify peaks in the histogram memory. Because the total length and amount of data stored in the histogram memory can be relatively large, this peak detection circuitry may identify time windows in the histogram memory that contain the identified peaks. The integrated circuit on which the histogram memory is implemented may then pass the histogram data from these time windows to an off-chip processor for distance calculations. Transmitting windows of peak data instead of the entire contents of the histogram memory significantly reduces the data bandwidth that needs to be transmitted between the integrated circuit and the processor. However, these on-chip peak detection algorithms / circuitry may best perform when the raw data in the histogram memory is first filtered to improve SNR.

[0185] Some embodiments may use a matched filter to improve the SNR of the raw data in the histogram memory. The shape of the matched filter may correspond to the shape of the initial pulse emitted by the optical measurement system. For example, a matched filter 1906 may be applied to raw data in the histogram memory having a generally square shape, as illustrated in FIG. 19 . The filter 1906 may be composed of a series of constant values, such as “1” or other numeric values, which may be convolved with the raw, unfiltered histogram data to generate a filtered version of the histogram data. Other filter shapes may be used, such as a pattern having one or more zeros between a set of one or more non-zero values ​​(e.g., 1 and −1). Such a pattern may occur when different shots are assigned different weights, as described in U.S. Patent Publication No. 2018 / 0259645, which is incorporated by reference in its entirety.

[0186] FIG. 20 illustrates a filtered version of the histogram data using the filter 1906 from FIG. 19 , according to some embodiments. In general, the filter 1906 may act as a low-pass filter, reducing the effect of background noise in the signal and improving the SNR. The low-pass filter also has the effect of increasing the width of any reflection peaks in the histogram data. In some embodiments, the width of the matched filter 1906 may be approximately the same as the width of the pulse transmitted from the light source of the optical measurement system. For example, for a pulse that is approximately four time bins wide (e.g., 8 ns), the corresponding matched filter 1906 may also be approximately four time bins wide. This may have the effect of approximately doubling the width of the peaks in the filtered histogram data (e.g., 8 bins wide).

[0187] In Figure 20, the first peak 2002 may represent a filtered version of the initial peak 1902 of Figure 19 resulting from reflections from the housing / window of the optical measurement system. Similarly, the second peak 2004 may represent a filtered version of the second peak 1902 of Figure 19 resulting from reflections from an object of interest in the surrounding environment. Note that the low-pass filtering operation of the matched filter 1906 has changed the shape of the peaks 2002, 2004 by increasing their width and smoothing their overall shape. This may help to more clearly distinguish the peaks 2002, 2004 from any noise accidentally stored in the histogram memory.

[0188] While the matched filter 1906 may make the peak 2002 more easily identifiable in the filtered histogram data, this filtering operation also distorts the shape of the histogram data. This results in a higher SNR and a higher confidence level for peak detection, but it also removes important information that may be present in the unfiltered histogram data. This information may be useful to a processor performing distance calculations, statistical analysis, and other calculations that may best be performed using the detail available in the unfiltered data. While some embodiments may pass the peak 2002 and / or peak 2004 from the filtered histogram data to a processor for distance calculations, some embodiments may use the locations of the peaks 2002, 2004 identified in the filtered histogram data to identify the corresponding peak locations in the unfiltered histogram data and then send the unfiltered peaks from the unfiltered histogram data to the processor. This process will be described in more detail below.

[0189] B. Adjacent peak detection In addition to losing signal information through the filtering process described above, some embodiments may also find it difficult to distinguish between adjacent peaks without using unfiltered histogram data.

[0190] FIG. 21A illustrates an example of two adjacent peaks that are relatively close together in time. The first peak 2102 may correspond to an initial pulse reflection from the housing / window of the optical measurement system, as described above. The second peak 2104 may correspond to an object near the optical measurement system. FIG. 21A represents unfiltered histogram data in a histogram memory after receiving photon counts from these two different reflections. Note that in the unfiltered histogram data, the two peaks 2102, 2104 can be distinguished from one another. Although they begin to merge, the first peak 2102 can be distinguished from the second peak 2104 using a peak detection algorithm. For example, one peak detection algorithm may scan through a histogram memory register and identify a time bin pattern that increases and then decreases by a threshold amount. Although the peaks 2102, 2104 are close together in time and at least partially overlap, the second peak 2104 can still be distinguished using this type of peak detection algorithm.

[0191] FIG. 21B illustrates the effect a low-pass filter may have on the unfiltered histogram data from FIG. 21A , according to some embodiments. The low-pass filter may be a matched filter, such as matched filter 1906 of FIG. 19 . The filter may increase the overall width of the first pulse 2102 and smooth any abrupt changes in the histogram data. In doing so, the first peak 2102 may blend in with the time bin of the second peak 2104. FIG. 21B illustrates how these two peaks 2102, 2104 may be encapsulated within a single peak 2106 such that they are no longer distinguishable within the unfiltered histogram data. When the filtered histogram data is analyzed and used to calculate distance measurements, close-range objects in the surrounding environment, represented by peak 2104, may be lost in the filtered histogram data. As a result, the close-range accuracy of the optical measurement system may be affected when the filtered histogram data is used without the unfiltered histogram data.

[0192] 21A and 21B illustrate the problems associated with close-range object detection, the same principles can also be applied to any range of objects that are close to each other. For example, peak 2102 and peak 2104 may both represent objects in the surrounding environment that are a significant distance (e.g., 15 feet, 20 feet, etc.) from the optical measurement system. However, if both of these objects are very close to each other (e.g., 1 foot, 2 feet, etc.), the resulting peaks in the unfiltered histogram data may be relatively close to each other. When the histogram data is filtered, these two peaks may merge together into a single peak, which may cause the optical measurement system to be unable to distinguish between these two objects in the surrounding environment.

[0193] IX. Using Filtered Data to Provide a Window into Unfiltered Data To solve these and other technical problems, some embodiments may filter the histogram data but pass the unfiltered data to a processor for distance calculations, statistical analysis, curve interpolation, peak fitting, etc. The filtered data may be used to identify the location of a histogram peak, but instead of simply passing a window of time bins containing the filtered peak data, these embodiments may alternatively or additionally use the identified location in the filtered histogram data to identify a corresponding location in the unfiltered histogram data. The window of time bins may then be sent from the unfiltered histogram data to a processor, and the processor may use the unfiltered histogram data to analyze the peak for distance calculations.

[0194] A. Window location in unfiltered data 22A and 22B illustrate how filtered data can be used to identify peaks in unfiltered data, according to some embodiments. The curve in FIG. 22A represents the filtered histogram data from FIG. 20. Similarly, the curve in FIG. 22B illustrates the unfiltered histogram data from FIG. 19. A filter can be applied to the unfiltered histogram data of FIG. 22B to obtain the filtered histogram data of FIG. 22A. A peak detection circuit / algorithm can be performed on the filtered histogram data to identify any peaks in the filtered histogram data. As described above, the peak detection algorithm can be performed on the filtered histogram data because the filtered histogram data may have a higher SNR, cleaner curves, fewer transient spikes, and / or produce better overall results than using unfiltered histogram data.

[0195] The peak detection circuit / algorithm may initially identify the first peak 2002. However, as described above, the peak detection circuit may be configured to exclude the first peak 2002 detected in the unfiltered histogram data under the assumption that the first peak 2002 may correspond to an early reflection of the light pulse from the housing / window of the optical measurement system. The peak detection algorithm may continue to scan through the unfiltered histogram data until a second peak 2004 is detected. Generally, the peak detection circuit / algorithm may identify the location of the maximum value of the peak 2004. For example, the peak detection circuit / algorithm may identify a time bin having a local maximum point around which the peak 2004 is centered.

[0196] After detecting the center of the peak 2004, the peak detection circuit / algorithm may identify surrounding time bins that may be considered part of the peak 2004. For example, some embodiments may identify a predetermined number of time bins (e.g., 17 time bins) centered around the peak location. Some embodiments may identify time bins that are within a threshold amount of the peak's maximum value (e.g., within 50% of the maximum value). Some embodiments may select a number of time bins centered around the maximum value based on the width of the light pulse emitted by the optical measurement system and the width of the filter. For example, if the emitted light pulse is four time bins wide and the corresponding matched filter is also four time bins wide, the resulting peak in the filtered histogram data may be expected to be at least eight time bins wide. As a result, the peak detection circuit / algorithm may identify at least eight time bins (e.g., 10 time bins, 12 time bins, etc.) centered around the peak's maximum value as representing a peak in the unfiltered histogram data.

[0197] Some embodiments may pass one or more windows of time bins containing the filtered peak data identified above to an off-chip processor. For example, an initial peak may be identified by the algorithm described above, along with three additional peaks corresponding to objects in the surrounding environment. A window of time bins centered around these peaks may be identified, and the filtered data within those time bins may be sent to the processor for processing. Using filtered data may be acceptable in situations where distance information about the surrounding environment can be adequately captured.

[0198] Alternatively or additionally, some embodiments may use the location of the identified peaks in the filtered data to identify windows of time bins in the unfiltered data, which may be passed to a processor for processing. These windows of unfiltered data may be passed in addition to or instead of the windows of filtered data described above.

[0199] To identify a window of unfiltered data in the unfiltered histogram, the optical measurement system may use the location of the maximum value of the peak identified in the filtered histogram data. For example, the location of the maximum value of peak 2004 in the filtered histogram data may be used as the location of the maximum value of peak 1904 in the unfiltered histogram data. Once this location is identified in the unfiltered histogram data, a window of time bins surrounding the maximum value may be identified around the corresponding location in the unfiltered data using the techniques described above (e.g., a predetermined number of time bins, time bins within a percentage range of the maximum value, etc.). As described above, the number of time bins identified in the time bin window of the filtered histogram data may be based on the width of the emitted light pulse and the width of the applied filter. This may result in a time bin window that is at least twice the width of the emitted light pulse. However, a fewer number of time bins may be used when identifying a time bin window in the unfiltered histogram data. Instead of considering how a low pass filter increases the width of peak 2004, the width of unfiltered peak 1904 may be smaller. For example, some embodiments may use a number of time bins that are slightly larger than the width of the emitted light pulse (e.g., two time bins, four time bins, six time bins, eight time bins, etc., that are larger than the emitted light pulse).

[0200] This method of using filtered data to identify time windows in unfiltered data preserves the benefits of using unfiltered data while also preserving the benefits of using filtered data. A higher SNR and smoother peak profile may enable on-chip peak detection circuitry to accurately identify peaks in the histogram. This allows a chip implementing a histogram memory to pass only a small subset of the information stored in the histogram memory to a processor for processing. However, by passing unfiltered data within these time bin windows to the processor, the processor may retain all the benefits of using the complete unfiltered data for processing.

[0201] 21A and 21B, the peak 2106 in FIG. 21B can be easily identified by on-chip peak detection circuitry, and the two peaks 2102, 2104 in FIG. 21A can be passed within a time window to a processor using the method described above. The processor can then use various techniques to analyze the data within the unfiltered time window to recognize that the time window contains not just a single peak, but two peaks 2102, 2104. The processor can also use the unfiltered data to more accurately identify the centers of these peaks 2102, 2104 when calculating distance calculations.

[0202] B. Circuitry for passing unfiltered data off-chip Figure 23 illustrates a schematic diagram of a circuit for using filtered data to pass unfiltered data to a processor for distance calculations, according to some embodiments. Figure 23 is similar to Figure 8 above in that it includes an arithmetic logic circuit 804 that receives signals 816 from the photodetectors in the optical sensor and aggregates those signals into photon counts that are aggregated in a histogram memory 806 over multiple shots in the optical measurement. The values ​​stored in the histogram memory 806 may represent the unfiltered histogram data described above.

[0203] FIG. 23 also includes a filter 2302 that may be applied to the photon counts in the histogram memory 806. The filter 2302 may comprise a matched filter with a shape that corresponds to the shape of the emitted light pulses from the optical measurement system. The filter 2302 may also act as a low-pass filter to smooth the overall shape of the histogram data. In some embodiments, the filter 2302 may be applied gradually to the time bins in the histogram memory 806. For example, the filter 2302 may be comprised of a series of numerical values ​​(e.g., 1 1 1 1) that may be convolved with the time bins in the histogram memory 806. In some embodiments, the filtered values ​​as they are generated by the filter 2302 may be passed to a peak detection circuit 2308. The peak detection circuit 2308 may then evaluate the filtered values ​​as they are received to detect peaks as they occur in the histogram memory 806. As described above, the peak detection circuit 2308 may operate by identifying peak values ​​in the filtered data characterized by increasing values ​​followed by decreasing values ​​(and vice versa). The peak analysis circuit 2308 may record the locations of local maxima identified as the filter 2302 passes through the unfiltered histogram memory 806. This method allows the filtered histogram data to be calculated in a single pass without requiring the entire filtered histogram data to be stored in a separate memory. When the filtered histogram data is passed to the processor 2314 for processing, the peak detection circuit 2308 may store the filtered values ​​around the detected peak as a time bin window, as described above.

[0204] In some embodiments, the output of filter 2302 may be stored in a separate buffer 2306. As noted above, in embodiments in which peak detection circuit 2308 operates on the direct output of filter 2302, a separate buffer 2306 may not be necessary. However, embodiments that store the output of filter 2302 in a separate buffer 2306 may then simultaneously store both unfiltered histogram data in histogram memory 806 and filtered histogram data in buffer 2306. Peak detection circuit 2308 may then pass through buffer 2306 to detect peaks in the filtered histogram data. Storing the filtered histogram data in buffer 2306 allows peak detection circuit 2308 to make multiple passes through the buffer and perform various iterative techniques to identify peaks in the filtered histogram. For example, each pass through buffer 2306 may detect a maximum peak that is different from the maximum value detected during a previous iteration of the peak detection algorithm.

[0205] 23 may operate as a multi-stage pipeline. For example, the first filter 2302 may use values ​​in the histogram memory 806 to populate the buffer 2306 after the current measurement is completed. During a subsequent measurement, the peak detection circuit 2308 may operate on values ​​stored in the buffer 2306 from the previous measurement. At the same time, the histogram memory 806 may be reset and begin receiving photon counts from the current measurement. In other embodiments, the peak detection circuit 2308 may identify peaks and transmit values ​​from the histogram memory 806 before new values ​​from the next measurement are received.

[0206] Once a maximum value in the filtered data is identified by the peak detection circuit 2308, one or more time bin windows 2310 may be identified in the unfiltered data in the histogram memory 806. These windows 2310 may be composed of multiple time bins surrounding the maximum value of the peak identified by the peak detection circuit 2308. The windows 2310 may be populated with data from the histogram memory 806 as unfiltered values ​​2312. These unfiltered values ​​2312 may then be passed to a processor 2314 for processing, such as distance calculations, curve fitting, etc. In some embodiments, a bin identifier may be passed to the processor 2314 along with, or instead of, the photon count of the bin itself.

[0207] The processor 2314 may be implemented on an integrated circuit that is separate and different from the integrated circuit on which the histogram memory 806 is implemented. In some embodiments, a first integrated circuit, such as an application specific integrated circuit (ASIC), may be fabricated, which includes the arithmetic logic circuit 804 and the histogram memory 806. The ASIC also includes the peak detection circuit 2308 and may include the optional buffer 2306, if that is part of the design. Thus, when referring to operations performed "on-chip," these operations may be performed on the first integrated circuit.

[0208] Some embodiments may also include a second integrated circuit comprising a processor 2314. The processor 2314 may include a microcontroller, a microprocessor, a field programmable gate array (FPGA) implementing a processor core, an ASIC implementing a processor core, etc. The unfiltered value 2312 may be transmitted via printed leads on a circuit board between the first integrated circuit and the second integrated circuit comprising the processor 2314.

[0209] 23, only unfiltered values ​​2312 from histogram memory 806 are shown as being passed to processor 2314. However, other embodiments may also pass filtered data to processor 2314. In some embodiments, filtered data need not be passed to processor 2314, as processor 2314 can instead re-run filter 2302 on unfiltered data to achieve similar results. In these embodiments, reducing the bandwidth of information passed between the first and second integrated circuits may be more important than ensuring that the number of processing operations performed by processor 2314 is minimized.

[0210] X. Multipulse Code Filtering The above embodiments use a single-pulse code as a simplified example of how filtered data can be used to identify and pass unfiltered data to a processor. However, these techniques described above are equally applicable to more complex multi-pulse codes. Multi-pulse codes may include a pulse train or multiple pulses transmitted as part of a single shot during a measurement. These pulse codes may be replicated during each shot during a measurement. Using a multi-pulse code provides a more distinct pattern for detection purposes, which may minimize false positives and other false peaks that may result from ambient noise rather than photons reflected from an object of interest in the surrounding environment. The following example illustrates how these techniques can be used with a multi-pulse code to transmit unfiltered data to a processor.

[0211] A. Low-pass addition filter 24 illustrates a portion of a histogram memory after receiving photons reflected from a multi-pulse code, according to some embodiments. Multi-pulse codes can be significantly longer than single-pulse codes, and therefore filtering operations can be even more important to correctly identify peaks in a data set that characterize reflection patterns from objects in the surrounding environment. FIG. 24 illustrates a multi-pulse code having the following format: 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 -1 -1 -1 -1 0 0 0 0 1 1 1 1 0 0 0 0

[0212] This particular pulse code can be constructed from multiple shots from the optical measurement system's light source during each shot of measurement. For example, the first shot can include pulses with positive "1" peaks, as shown in FIG. 24. When these peaks are received by the optical sensor, they can be given a positive weight and added to a histogram memory time bin. The second shot can include pulses with negative "-1" peaks, as shown in FIG. 24. When these peaks are received by the optical sensor, they can be given a negative weight and added to a histogram memory time bin. A combination of these two shot types can be combined to produce positive and negative peaks in the histogram memory, as shown in FIG. 24. Due to the length of the entire pulse code, the time bin window 2402 in the histogram memory that contains the entire pulse code can be significantly longer than the time bin window described above. For example, a multi-pulse code can receive up to approximately 200 time bins in the histogram memory.

[0213] When a multi-pulse code is received as reflected photons in a histogram memory, this can be represented as positive / negative peaks in the photon count. In the example of Figure 24, the multi-pulse code can result in peaks 2404, 2406, 2408, and 2409 in the histogram memory. The histogram memory can then be processed with a matched filter 2416 based on the shape of the multi-pulse code transmitted from the light source of the optical measurement system.

[0214] 25 illustrates a filtered version of a peak received from a multi-pulse code, according to some embodiments. When convolving the matched filter 2416 with the peaks 2404, 2406, 2408, 2409 in the histogram memory, the resulting filtered version of the histogram data may include a large peak 2504 rather than multiple individual peaks. A window of time bins 2502 may be specified around the filtered peak of the filtered data, and this peak 2504 may be identified by a peak detection circuit as described above.

[0215] Using the above techniques, the location of the peak 2504 in the filtered data may be used to identify the time bin window 2402 of the unfiltered data of Figure 24. The unfiltered peaks 2404, 2406, 2408, 2409 may then be passed to a processor for processing within the time bin window 2402. As described above for other embodiments, the filtered data of Figure 23 may also be passed to the processor in addition to the unfiltered data.

[0216] B. Compression Filter Figure 26 illustrates a filter that uses only a single binary designator, according to some embodiments. The matched filter 2416 of Figure 24 uses the same binary designator for each value in the filter. For example, the sequence "1 1 1 1" was used in filter 2416 to correspond to the first peak of the multi-pulse code. However, using this uniform sequence of values ​​in filter 2416 creates a low-pass effect in the filtered data. As noted above, this changes the shape of the filtered data, causing many of the characteristics of the unfiltered data to be lost in the filtered data.

[0217] In the example of Figure 26, the reflection peaks 2604, 2606, 2608, and 2609 in the unfiltered histogram data may not have the same precise square shape as the multi-pulse code transmitted from the light source of the optical measurement system. For example, the photodetector may exhibit a "pile-up" effect, where photons received at the beginning of the peak reflection may create an avalanche effect, after which it may take some time for the photodetector to reset. This causes the peaks to have the shape shown in Figure 26, whereby the peaks 2604, 2606, 2608, and 2609 have a more pointed or sloped shape instead of the ideal square shape illustrated in the other examples above.

[0218] To preserve this shape in the filtered data, some embodiments may use a single binary designator that may still sum peaks 2604, 2606, 2608, 2609 into a single peak without implementing a low-pass filter operation. Filter 2616 may use only a single value or designator at the beginning of each pulse instead of a uniform set of values ​​throughout the duration of each pulse. In the example of FIG. 26, filter 2616 may have the following values: 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 -1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0

[0219] This filter may produce peaks similar to those illustrated in Figure 25, but the general shape of the peaks appears to more closely resemble the shapes of peaks 2604, 2606, 2608, 2609 in the unfiltered data. Therefore, the filtered data may more accurately preserve the shapes of the peaks illustrated in Figure 26. Some embodiments may then pass the filtered data, which preserves the shapes of the peaks, to a processor for processing.

[0220] In some embodiments, the filters illustrated in both Figures 24 and 26 may be operated simultaneously in tandem. For example, unfiltered data may be processed using filter 2416 and filter 2616 to generate two filtered versions of the data. Because the first portion of each filter may use the same arithmetic operations, circuitry may be shared between the two filters when they are run in parallel. For example, the first stage of the summation step of each of these filters may be shared. Some embodiments may then use a low-pass filtered version of the histogram data to identify peaks and then pass a compressed version of the histogram data to a processor that does not lose its shape due to the low-pass filtering.

[0221] Note that the example filters 2416, 2616 described above with respect to multi-pulse codes use a single bit as a filter tap designator. However, this is done merely as an example in this disclosure. The actual values ​​of each filter may include multi-bit values, such as 10-bit values, rather than a single bit. As noted above, these values ​​may be convolved and calculated as a sliding window to detect resulting peaks in the filtered data.

[0222] XI. Methods for Providing Unfiltered Data 27 illustrates a flowchart of a method for analyzing filtered and unfiltered data in an optical measurement system, according to some embodiments. The following method describes the process for a single optical sensor in an optical measurement system. However, as noted above, an optical measurement system may include many optical sensors or "pixels," and the method may be performed for each optical sensor in the optical measurement system.

[0223] In step 2702, the method may include transmitting one or more pulse trains from the light source over one or more first time intervals as part of the optical measurement. Each of the one or more first time intervals may represent a "shot" that is repeated multiple times in the measurement. Each of the first time intervals may include one or more pulse trains that are coded and transmitted by the light source so that the pulse trains can be recognized when reflected from objects in the surrounding environment. Each of the time intervals may be subdivided into multiple time bins, whereby each time bin represents a bin of a histogram of photon counts received during the optical measurement. An example of how a single measurement may include multiple shots subdivided into time bins that aggregate photon counts is described above in connection with FIG. 9.

[0224] In step 2704, the method may also include detecting photons from the one or more pulse trains using a light sensor. As described in detail above, the light sensor may include multiple light detectors, such as multiple SPADs. The light sensor may receive reflected light received from the surrounding environment and ambient background noise. The reflected light received by the light sensor may include light reflected from objects of interest in the surrounding environment. For example, these objects of interest may be at least 30 cm away from the light sensor and may represent surrounding vehicles, buildings, pedestrians, and / or any other objects that may be encountered around the optical measurement system during use. These objects of interest may be distinguished from objects that are part of the optical measurement system itself, such as the housing. In some embodiments, the reflected light received by the light sensor may also include light reflected from the housing or other parts of the optical measurement system. These reflections may include direct primary reflections from a window or housing of the optical measurement system and secondary reflections reflected around the interior of the optical measurement system. The light sensor may also be coupled to a threshold detection circuit and / or an arithmetic logic circuit that accumulates photon counts. This combination may be referred to above as a “pixel.” Figure 5 illustrates one example of how photon counts may be received from a photosensor and counted using a threshold circuit and a pixel counter (e.g., an arithmetic logic circuit).

[0225] In step 2706, the method may also include accumulating photon counts from the optical sensor in multiple registers to represent a histogram of photon counts received during the current measurement. These photon counts may be accumulated in multiple registers in a memory block corresponding to the optical sensor. The multiple registers may be implemented using registers in the SRAM of the histogram data path, as described above in FIGS. 8 and 9. Each time interval may be subdivided into multiple first time bins for the histogram. Corresponding time bins in each of the one or more time intervals may be accumulated in a single register in the multiple registers. Each time interval may represent a shot, and together the one or more time intervals may represent a measurement of the optical measurement system. Each time interval may be defined by a start signal or shot signal that resets the memory representing the histogram back to the first register of the histogram. Received photons reflected from the optical measurement system housing may be stored in the histogram memory in the same manner as other received photons reflected from ambient objects outside the optical measurement system.

[0226] In step 2708, the method may additionally include filtering the histogram in the multiple registers to provide a filtered histogram of photons from the multiple registers. The filter may include a matched filter based on the shape of one or more pulse trains transmitted from the optical measurement system. The filter may be convolved with the unfiltered histogram in the multiple registers and stored in a separate buffer. The filter may consist of repeated non-zero valued square pulses. The filter may also consist of a single non-zero value followed by multiple approximately zero values. The filter may operate as described above with respect to FIGS. 19, 24, and / or 26.

[0227] In step 2710, the method may further include detecting the location of peaks in the filtered histogram. Detecting this peak may be performed using several different techniques, depending on the particular embodiment. In some embodiments, a pass may be made through the filtered histogram memory to identify peaks. The system may sequentially access values ​​in the filtered histogram and identify peaks, starting from the beginning of the measurement. Peaks may be identified by identifying increasing values ​​followed by decreasing values ​​in the filtered histogram. Similarly, the center of a peak may be identified by identifying values ​​in the histogram that have smaller values ​​in the time bins on either side.

[0228] In step 2712, the method may also include identifying locations within the plurality of registers storing and unfiltered representations of the peak. These locations may be identified using the location of the peak in the filtered histogram. Thus, the filtered histogram data may be used to identify the location of the peak in the unfiltered histogram data. Identifying a peak in the unfiltered histogram may involve locating a window of time bins occurring near the center of the peak. This process may involve extending outward from the peak location to determine a range of registers in the histogram memory that includes the entire peak. For example, some embodiments may identify a predetermined number of time bins around the location of the maximum value in the unfiltered histogram that may be designated as a peak. For example, a predetermined number of time bins, such as three time bins, five time bins, nine time bins, fifteen time bins, seventeen time bins, etc., may be identified and centered on the maximum value to represent the entire peak. Some embodiments may identify surrounding time bins having values ​​within a percentage of the maximum value in the peak register. This may result in a varying number of time bins that can be used to represent the peak, depending on the width of the peak. For example, time bins surrounding a maximum value may be included in the peak if their values ​​are within 25% of the maximum value. This process of identifying representations of peaks in unfiltered histograms is described above in connection with Figures 22A and 22B.

[0229] In step 2714, the method may also include transmitting the unfiltered representation of the peaks to a processor to calculate the distance to the object. The distance to the object may represent the distance between the optical measurement system and an object in the surrounding environment. This calculation may be performed using the unfiltered representation of the peaks transmitted to the processor. The processor may be physically separate and distinct from the integrated circuit on which the histogram memory is implemented. Thus, the histogram memory may represent a first integrated circuit, and the processor may represent a second integrated circuit that communicate with each other via a printed circuit board. An example of this circuit is described above in connection with FIG. 23.

[0230] It should be understood that, according to various embodiments, the specific steps illustrated in FIG. 27 provide a particular method for using filtered histogram data to identify the locations of peaks in unfiltered histogram data and passing the unfiltered data to a separate processor. According to alternative embodiments, other sequences of steps may be performed. For example, alternative embodiments of the present invention may perform the steps outlined above in a different order. Furthermore, individual steps illustrated in FIG. 27 may include multiple sub-steps that may be performed in various sequences appropriate to the individual step. Furthermore, additional steps may be added or removed depending on the particular application. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0231] XII. Combining spatially adjacent pixels In some environmental conditions, optical measurement systems may have difficulty providing reliable responses above a predetermined confidence level when measuring distances to some objects. These environmental conditions may include weather phenomena such as rain, mist, and fog. These conditions may also include objects that are simply too far from the optical measurement system to generate reliable measurements based on the number of photons accurately reflected by objects in the environment and received by the optical sensor. In any of these situations, the optical measurement system may use a detection threshold to prevent false positives from being detected by the system.

[0232] To improve the confidence level of some measurement conditions, some embodiments may use a form of spatial filtering. These embodiments may assume that spatial correlation exists between photons received by spatially adjacent optical sensors. For example, physically adjacent optical sensors may receive photons reflected from the same object in the surrounding environment. Techniques are described below for combining the responses of these adjacent optical sensors so that the combined response is above the detection threshold, even if the responses of these individual optical sensors are below the detection threshold, thereby improving the reliability of the distance calculation.

[0233] FIG. 28 illustrates an example of an object that may provide reflected photons to an adjacent optical sensor, according to some embodiments. In this example, the object 2800 in the surrounding environment may include a stop sign as illustrated in FIG. 28. Other example objects may include other road signs, buildings, vehicles, pedestrians, etc. Depending on the distance of the object 2800 from the optical measurement system, the object 2800 may be large enough to cause photons from multiple light sources to be reflected back to corresponding optical sensors in the optical measurement system. Thus, more than one adjacent optical sensor may receive photons reflected from the object 2800, and therefore, the multiple optical sensors may independently use the photon counts reflected from the object 2800 to calculate the distance to the object 2800. For example, each of these optical sensors may be associated with an independent histogram data path and histogram memory, as described above in connection with FIG. 8.

[0234] FIG. 28 illustrates how multiple optical sensors can "see" different portions of the same object 2800. In this example, nine optical sensors may have views of the object 2800. These views are represented by the circles illustrated on the stop sign object 2800. The circles may be referred to herein as "optical sensor views" to distinguish what is viewed by the optical sensors from the optical sensors themselves. Note that additional optical sensors may also have views of the object 2800 beyond the nine optical sensor views explicitly illustrated in FIG. 28.

[0235] Based on environmental conditions or distance to object 2800, some of the optical sensor views may fail to produce results that meet or exceed the detection threshold. The detection threshold may be implemented in different manners depending on the embodiment. For example, the detection threshold may be based on receiving at least a threshold number of photon counts reflected from object 2800. The detection threshold may be based on a calculated distance to object 2800. The detection threshold may be based on a confidence level in distinguishing peaks in a corresponding histogram from noise in the histogram. For example, a particularly noisy environment with a low SNR may fail to produce results that meet or exceed the detection threshold. The detection threshold may also be implemented as a peak detection threshold for detecting peaks in a histogram.

[0236] In the example of FIG. 28 , optical sensor views 2802, 2810, and 2816 may produce results that exceed the detection threshold. These optical sensor views 2802, 2810, and 2816 are shaded in a darker color to distinguish them from other optical sensor views 2804, 2806, 2808, 2812, 2814, and 2818 of object 2800, which produce results that do not exceed the detection threshold. Note that optical sensors with adjacent optical sensor views may produce different results. This difference may be due to slightly different environmental conditions for each optical sensor and / or slightly different reflective characteristics in different areas of object 2800. This situation may become more likely at the detection limit of the optical measurement system. For example, as object 2800 begins to obscure or move out of range of the optical measurement system, optical sensors in the array may begin to “drop out” of detecting object 2800 as their results begin to fall below the detection limit.

[0237] Some embodiments may ignore any light sensors below the detection limit. However, other embodiments may exploit the spatial correlation of adjacent pixels. Often, adjacent light sensor views may be assumed to be spatially correlated. In the example of FIG. 28 , each of the light sensor views is spatially correlated because they reflect from the same object 2800. Because of this spatial correlation, adjacent light sensors may be considered together when evaluating optical measurements. Instead of relying solely on a single light sensor, these embodiments may instead consider a combination of light sensor responses to characterize and / or generate distance measurements for each light sensor.

[0238] 28 , central optical sensor view 2810 may be spatially adjacent to eight additional optical sensor views 2802, 2804, 2806, 2808, 2812, 2814, 2816, and 2818. These additional optical sensor views may be considered spatially adjacent when they are diagonally adjacent, such as optical sensor views 2802, 2806, 2814, and 2818, and when they are orthogonally adjacent, such as optical sensor views 2804, 2808, 2812, and 2816. The characterization and / or calculation of a distance measurement for central optical sensor view 2810 may utilize information derived from the histograms of each of these other spatially adjacent optical sensors.

[0239] A. Combination of distance measurements Figure 29 illustrates one example of how spatially adjacent photosensor views 2808, 2812 may be used in the calculated distance measurement of one of the corresponding photosensors 2810, according to some embodiments. Figure 29 may be a simplified view of Figure 28, focusing for clarity on how a photosensor may use spatially adjacent views from two other photosensors. However, as described below, these techniques may be applied to any number of adjacent photosensor views.

[0240] In this example, the photosensor array may include a photosensor 2906 comprising one or more individual photodetectors, as illustrated in FIG. 29 . The photosensor 2906 may be physically positioned next to two orthogonally adjacent photosensors 2904, 2908 in a photosensor array 2902 of the optical measurement system. Because the photosensors 2904, 2906, 2908 are physically adjacent in the photosensor array 2902, the corresponding photosensor views 2808, 2810, 2812 may also be considered spatially adjacent in the surrounding environment. However, this may not always be the case. Some embodiments described below may use a rotating photosensor array, where the spatial adjacency of photosensor views does not necessarily depend on the physical adjacency of the corresponding photosensors.

[0241] To calculate or characterize a distance measurement for the optical sensor 2906, some embodiments may rely solely on photon counts reflected from the object 2800 from the optical sensor view 2810. If the resulting histogram and / or distance calculation meets or exceeds the detection threshold, it may be sufficient to rely on the optical sensor 2906 response itself. However, in some cases, the resulting histogram and / or resulting distance calculation for the optical sensor 2906 may fall below the detection threshold based on the various factors described above (e.g., environmental conditions, distance to the object 2800, ambient noise, etc.). In these situations, the optical measurement system may utilize responses from adjacent optical sensors 2904, 2908 to improve the reliability of the measurement obtained by the optical sensor 2906, such that the resulting peak or calculation meets or exceeds the detection threshold.

[0242] Some embodiments may be configured to combine information from a histogram associated with light sensor 2906 with information from histograms of neighboring light sensors, such as light sensors 2904 and 2908. In some embodiments, this process may share data from neighboring light sensors with each other. For example, photon counts from the histograms of light sensors 2904 and 2908 may be added to the histogram of light sensor 2906. This may have the effect of boosting the reflected signal in the histogram of light sensor 2906, since the photons are all reflecting from the same object 2800. Note that if the light sensor views 2808, 2810, and 2812 are not spatially adjacent as expected, accumulating photons from light sensors 2904 and 2908 is likely to have no adverse effect on the histogram of light sensor 2906, since there are no reflections at that distance. In effect, this may approximate a spatial matched filter over the histograms of neighboring light sensors. If the two peaks coincide in time, the responses of the photosensors can be considered spatially correlated, and the two signals can be combined to produce a larger resulting peak in the histogram.

[0243] To combine information from the histograms, the output of the arithmetic logic circuit for each of the adjacent photosensors 2904, 2908 may be sent to the arithmetic logic circuit for photosensor 2906. This may accumulate photons received by any of the photosensors 2904, 2906, 2908 into a single histogram to boost the return signal. In some embodiments, this accumulation calculation may be extended to use a weighted sum that applies weights to the responses of the various photosensors. Using weights may allow for a more accurate determination of how much an adjacent photosensor affects the response of another photosensor. For example, orthogonally adjacent photosensors may have photosensor views that are closer to the central photosensor view than diagonally adjacent photosensors, so these orthogonally adjacent photosensors may be weighted higher than diagonally adjacent photosensors when their photon counts are added to the central photosensor's histogram. Using weights may also allow the optical measurement system to use photosensor responses that are not directly adjacent. As noted above, object 2800 may include more than the nine photosensor views illustrated in FIG. 28. These additional photosensor views, which are not necessarily immediately adjacent to the central photosensor, may also be considered. For example, the photon counts from these non-adjacent photosensors may also be added to the central photosensor's histogram with a weight less than the weight assigned to the immediately adjacent photosensor responses.

[0244] B. Histogram / Peak Combination 30 illustrates how histograms for adjacent light sensors may be combined, according to some embodiments. In this example, a histogram for each light sensor in an array of at least nine adjacent light sensors is illustrated along with a light sensor view on object 2800. Note that the actual histograms may be stored in a memory block representing a histogram memory in an optical measurement system. They are displayed in the light sensor view of object 2800 for illustrative purposes only.

[0245] Instead of simply generating a weighted sum of the surrounding light sensor responses, some embodiments may use more complex methods that combine information from the histograms of the surrounding light sensor views into the histogram of the central light sensor view. In this example, light sensor view 2810 may be surrounded by eight orthogonally / diagonally adjacent light sensor views, as illustrated in FIG. 30 . Because these light sensor views are spatially adjacent and receive reflected photons from the same object 2800, each of the histograms associated with these views may include a peak corresponding to the distance between object 2800 and the optical measurement system. These peaks are graphically represented in FIG. 30 for each of the corresponding light sensor views. Note that some peaks are stronger than others, and the peak of the central light sensor view 2810 may be small enough that it does not pass the detection threshold.

[0246] Some embodiments may combine information from histograms of surrounding light sensors to improve the detection of the central light sensor. The information from the histograms may be specific peaks in the histogram memory. For example, when peaks are located at the same distance from multiple adjacent light sensors, the system may determine that the corresponding light sensor views are spatially correlated and combine the associated histograms. Some embodiments may combine the entire histogram for each light sensor, while other embodiments may implement a peak detection circuit / algorithm and then combine only the specific peak locations. Therefore, the information combined from the histograms may include the histogram values ​​themselves, portions of the histograms representing the peaks, and / or additional values ​​derived from the histograms.

[0247] Instead of simply generating a weighted sum of surrounding histograms, some embodiments may use a Gaussian combination of adjacent histograms combined with a convolution process. For example, each of the histograms spatially adjacent to the histogram of the central photosensor view 2810 may have a Gaussian function applied before combination. Instead of summing the resulting histogram values ​​together, they may be convolved together in the same way that a matched filter is convolved with a single histogram. As noted above, this may approximate spatial convolution between spatially adjacent photosensor responses.

[0248] XIII. Optical Sensor Array Configuration The above example uses a rectangular photosensor array pattern with photosensors arranged in a grid layout. However, not all embodiments are limited to a rectangular grid. Other embodiments may use different photosensor array patterns that may result in higher photosensor densities. Other layout patterns may also be more suitable for rotating the photosensor array. In any of these layout patterns, the techniques described above may be used to combine data from spatially adjacent photosensor views.

[0249] A. Solid-state array configuration 31 illustrates an example of a rectangular photosensor layout, according to some embodiments. This rectangular photosensor layout is similar to the rectangular grid pattern of photosensors illustrated in the examples above. Photosensor array 3100 may be implemented as a solid array with photosensors 3102, 3104, etc., stationary relative to the rest of the optical measurement system. Each of the photosensors in photosensor array 3100 may be scanned by a processor as measurements are made.

[0250] To use the above algorithm for spatial correlation with a rectangular photosensor array 3100, the histograms for adjacent photosensors may be assumed to be spatially adjacent or spatially correlated. For example, photosensor 3102 and photosensor 3104 are physically adjacent in photosensor array 3100. As a result, corresponding photosensor views 3106, 3108 of the surrounding environment may also be adjacent.

[0251] B. Rotating array configuration 32 illustrates a configuration for a rotating optical measurement system, according to some embodiments. The optical measurement system may include one or more optical sensor arrays 3204 physically coupled to a rotating member 3200. When the optical measurement system operates, the rotating member 200 may rotate and the optical sensor arrays 3204 may continuously scan the surrounding area as they rotate about a central axis.

[0252] In contrast to the non-rotating optical sensor array of FIG. 31 , physical proximity of optical sensors in optical sensor array 3204 does not necessarily mean that optical sensor views are spatially adjacent in the surrounding environment. For example, optical sensor 3202 may perform optical measurements at a position as the optical measurement system rotates. During a subsequent optical measurement, optical sensor 3202 may be at a different rotation angle than during the previous optical measurement and thus view different objects and areas in the surrounding environment. Therefore, a histogram for a previous measurement of optical sensor 3202 may be “spatially adjacent” to a histogram for a subsequent measurement for the same optical sensor 3202 during a subsequent time interval as optical sensor 3202 rotates. Therefore, some embodiments may buffer previous histograms on-chip for use as spatially adjacent histograms for the above methods.

[0253] Another difference between the photosensor array 3204 of FIG. 32 and the photosensor array of FIG. 31 is the pattern in which the photosensors are arranged in the photosensor array 3204. Instead of being arranged in a rectangular grid, the columns of photosensors in the photosensor array 3204 are slightly offset from one another. This allows the photosensors in the array 3204 to be positioned closer to one another, thereby increasing photosensor density. Additionally, by offsetting the photosensors by less than the width of the photosensors, the spatial resolution of the photosensor array 3204 may be improved.

[0254] This alternative pattern for light sensor placement within the array 3204 may affect which light sensors are considered spatially adjacent to other light sensors in the array 3204. Physical adjacency does not necessarily mean spatial adjacency from the light sensor's view in the surrounding environment. As the optical measurement system rotates, light sensors spatially adjacent to light sensor 3202 may include light sensors on other light sensor arrays on the other side of the rotating member 3200 (not shown), as well as histograms from previous measurements that have not been buffered and have not yet been overwritten. Spatial adjacency may correspond to any geometric arrangement of light sensors in an array configuration. Instead of relying on physical adjacency, embodiments may instead determine adjacent light sensor views in the surrounding environment during multiple measurements to perform the above operations.

[0255] XIV. Circuit Implementation for Combining Pixels FIG. 33 illustrates circuitry for combining information from spatially adjacent histograms, according to some embodiments. As described in detail above, each photosensor 3202 may be coupled with an arithmetic logic circuit 3304 and a histogram memory 3308. The histogram memory 3308 for each photosensor 3302 may be referred to as a memory block. Each memory block 3308 may store photon counts aggregated during measurement to form a histogram 3306 of photon counts. This combination of photosensor 3302, arithmetic logic circuit 3304, and memory block 3308 may be referred to as a "pixel." Each pixel may operate independently to perform optical measurements, such that each pixel receives its own start signal and may aggregate photon counts, independent of the timing signals used to control other pixels.

[0256] To combine information from the histograms as described above, the optical measurement system may include a summation / convolution circuit 3310 that combines the histograms together using mathematical operations. The summation / convolution circuit 3310 may be configured to scan or receive individual time bins from each of the memory blocks 3308 for several spatially adjacent photosensors. Each time bin from multiple histograms may be combined together into a single histogram 3312 representing a single photosensor, such as photosensor 3302b. This circuit may use an arithmetic logic unit to add values ​​together. This circuit may also use multipliers to apply weights to each of the histograms 3306 as they are combined together. The final histogram 3312 may be passed through a peak detection circuit 3314, as described above, to locate peaks within the histogram. Because the histogram 3306 is combined from spatially adjacent photosensors, the resulting histogram 3312 is likely to provide one or more peaks that exceed a detection threshold.

[0257] Any identified peaks may be sent to processor 3316 for distance calculation as described above. Processor 3316 may be implemented on an integrated circuit 3320 that is physically separate and different from the integrated circuit 3322 on which arithmetic logic circuit 3304 and / or memory block 3308 are implemented.

[0258] 33, multiple elements may be combined to form a circuit configured to combine information from a first histogram with information from one or more spatially adjacent histograms to generate a distance measurement for the light sensor. In this embodiment, the information from the histograms may include data values ​​from the histograms themselves, and the circuit may include a summation / convolution circuit 3310. In some embodiments, the circuit may also include a processor 3316 that actually generates the distance measurement.

[0259] FIG. 34 illustrates an alternative circuit for combining information from histograms, according to some embodiments. This circuit is similar to the circuit of FIG. 33, except that in this case, peaks may be detected by peak detection circuit 3411 for each individual histogram 3306 before they are combined. Instead of combining all of the histograms, this example may combine only the peaks identified in the histograms. Some embodiments may also determine whether the peaks identified in each of the histograms 3306 are spatially correlated before combining the information from the histograms. For example, if several spatially adjacent histograms demonstrate peaks in similar locations, a determination may be made that the peaks result from the same object in the surrounding environment, and the histograms may be treated as spatially correlated at that location. This may also be used to identify outliers. For example, if eight of nine adjacent histograms demonstrate a peak at a certain location, the ninth histogram may be excluded from the calculation as an outlier.

[0260] 34, combining spatially adjacent histograms may be performed on a central pixel, such as the pixel of histogram 3306b. In some embodiments, the histograms to be combined with histogram 3306b may be multiplied by a weighting factor before they are combined with the central pixel.

[0261] In this example, the circuitry for combining information from the histograms may be implemented in a processor in a second integrated circuit 3420 different from the integrated circuit 3422 on which the peaks are detected. As described in detail above, the peaks may be detected on-chip and a window of time bins may be passed to the processor for distance measurement. The processor may also execute summation / convolution circuitry 3425 to combine the individual peaks to form a representation of a single peak 3426 for the optical sensor 3302b. This single peak 3426 may exceed a detection threshold and may be used by a distance calculation algorithm 3427 on the processor.

[0262] FIG. 35 illustrates another circuit for combining information from histograms, according to some embodiments. In this example, the processor may calculate the distance for each photosensor based on its individual histogram, without any combination of histogram information. After the distances are calculated, the information from each pixel may be combined via a summation / convolution process 3504 on the processor. This operation may be significantly simpler than combining information from raw histograms and may be performed relatively faster than the above example. For example, if distances have been calculated for a number of neighboring pixels, these distances may be compared to each other. Some embodiments may average these distances for surrounding pixels to determine the distance of the central pixel. This may improve the accuracy of each distance measurement for each pixel because each pixel is based on several different pixel measurements, not just a single pixel. Some embodiments may also use surrounding pixels to calculate a confidence value. If the surrounding pixels include similar distance measurements as the central pixel, the confidence value of the central pixel may be relatively high.

[0263] XV. Methods for Combining Spatially Adjacent Data FIG. 36 illustrates a flowchart of a method for using spatially adjacent pixel information in an optical measurement system. The following method describes a process for a single optical sensor combined with one or more spatially adjacent optical sensors in an optical measurement system. Specifically, information from one or more spatially adjacent optical sensors may be combined with histogram information from the single optical sensor to be used in calculating distance measurements. However, as noted above, an optical measurement system may include many optical sensors or "pixels," and this method may be performed for each optical sensor in the optical measurement system. Thus, this method may be performed multiple times for each optical sensor: once when considered as a single optical sensor with its combined histogram information, and multiple times as a spatially adjacent optical sensor of other optical sensors in the array.

[0264] In step 3602, the method may include transmitting one or more pulse trains over one or more first time intervals as part of the optical measurement. Each of the one or more time intervals may represent a "shot" that is repeated multiple times in the measurement. Each of the time intervals may include one or more pulse trains that are coded and transmitted by the light source so that the pulse trains can be recognized when reflected from objects in the surrounding environment. Each of the time intervals may be subdivided into multiple time bins, whereby each time bin represents a bin of a histogram of photon counts received during the optical measurement. An example of how a single measurement may include multiple shots subdivided into time bins that aggregate photon counts is described above in connection with FIG. 9.

[0265] In step 3604, the method may include detecting reflected photons from the one or more pulse trains. These reflected photons may be detected using multiple optical sensors. The multiple optical sensors may include a first optical sensor as well as one or more optical sensors spatially adjacent to the first optical sensor. As described above, an optical sensor may be "spatially adjacent" to a first optical sensor when the views of the two optical sensors are adjacent in the environment surrounding the optical measurement system. In some configurations, this may include optical sensors that are physically adjacent on the optical measurement system, while other configurations (e.g., rotating configurations) do not require physical proximity. As described in detail above, each optical sensor may include multiple photodetectors, such as multiple SPADs. The optical sensors may receive reflected light received from the surrounding environment and ambient background noise. The reflected light received by the optical sensors may include light reflected from objects of interest in the surrounding environment. For example, these objects of interest may be at least 30 cm away from the light sensor and may represent surrounding vehicles, buildings, pedestrians, and / or any other objects that may be encountered around the optical measurement system during use. These objects of interest may be distinguished from objects that are part of the optical measurement system itself, such as the housing. The light sensor may also be coupled to a threshold detection circuit and / or an arithmetic logic circuit that accumulates the photon counts. Figure 5 illustrates one example of how photon counts may be received from the light sensor and counted using a threshold circuit and a pixel counter (e.g., an arithmetic logic circuit).

[0266] In step 3606, the method may include accumulating photon counts received during one or more time intervals. The photons may be accumulated using arithmetic and logic circuits and may be accumulated in one or more memory blocks, such as the histogram memory described above. The memory blocks may be implemented using registers in the SRAM of the histogram data path, as described above in FIGS. 8 and 9. Each time interval may be subdivided into multiple first time bins for the histogram. Corresponding time bins in each of the one or more time intervals may be accumulated in a single register in the memory block. Each time interval may represent a shot, and together the one or more time intervals may represent a measurement of the optical measurement system. Each time interval may be defined by a start signal or shot signal that resets the memory representing the histogram back to the first register of the histogram.

[0267] In step 3608, the method may include combining information from the first histogram with information from one or more histograms to generate a distance measurement for the light sensor. As described above, the information from the first histogram may include raw data from the histogram itself, peaks identified within the histogram, calculations or statistics derived from the histogram, distance measurements calculated based on the histogram, and / or other information that may be calculated or derived from the photon counts in the histogram. Similarly, information from one or more histograms of spatially adjacent light sensors may also include any of these types of information. Combining the information may include calling the histogram information, applying numerical weights to the histogram information, convolving the histogram information, applying a Gaussian function to the histogram information, and / or other mathematical operations that may combine information from different histogram information. This combination may result in a new histogram, a new peak in the histogram, a new distance measurement, and / or another numerical value. Circuitry for performing this combination and / or generating the distance measurement for the first photosensor may include all or part of on-chip circuitry for summing / convolving the histogram information and a separate processor configured to calculate the distance measurement.

[0268] It should be understood that the specific steps illustrated in FIG. 36 provide a particular method for using spatially adjacent pixel information in an optical measurement system, according to various embodiments. Other sequences of steps may be performed according to alternative embodiments. For example, alternative embodiments of the present invention may perform the steps outlined above in a different order. Furthermore, individual steps illustrated in FIG. 36 may include multiple sub-steps that may be performed in various sequences appropriate to the individual step. Furthermore, additional steps may be added or removed depending on the particular application. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0269] XVI. Additional Embodiments Although some embodiments disclosed herein focus on optical ranging applications within the context of 3D sensing for automotive use cases, the systems disclosed herein can be used in any application without departing from the scope of the present disclosure. For example, the systems can have small or even miniature form factors, enabling several additional use cases, such as for solid-state optical ranging systems. For example, the systems can be used in 3D cameras and / or depth sensors within devices such as mobile phones, tablet PCs, laptops, desktop PCs, or other peripherals and / or user interface devices. For example, one or more embodiments can be used within mobile devices for facial recognition and tracking capabilities, support for gaze tracking capabilities, and / or 3D scanning of objects. Other use cases include front-facing depth cameras for augmented reality and virtual reality applications in mobile devices.

[0270] Other applications include deployment of one or more systems on an airborne vehicle, such as an airplane, helicopter, unmanned aerial vehicle, etc. Such examples can provide 3D sensing and depth imaging to aid in navigation (autonomous or otherwise) and / or generate 3D maps for later analysis, e.g., to support geophysical, architectural, and / or archaeological analysis.

[0271] The system can also be mounted on stationary objects and structures such as buildings, walls, poles, bridges, scaffolding, etc. In such cases, the system can be used to monitor outdoor areas such as manufacturing facilities, assembly lines, industrial facilities, construction sites, excavation sites, roadways, railways, bridges, etc. Additionally, the system can be mounted indoors and used to monitor the movement of people and / or objects within a building, such as the movement of inventory within a warehouse, or the movement of people, luggage, or goods within an office building, airport, train station, etc. As will be recognized by those of ordinary skill in the art with the benefit of this disclosure, many different applications of optical ranging systems are possible, and therefore the examples provided herein are provided by way of example only and should not be construed as limiting the use of such systems to only the examples expressly disclosed.

[0272] XVII. Computer Systems Any of the computer systems or circuits referred to herein may utilize any suitable number of subsystems. The subsystems may be connected via a system bus 75. By way of example, the subsystems may include input / output (I / O) devices, system memory, storage devices, and network adapters (e.g., Ethernet, Wi-Fi, etc.) that may be used to connect the computer system to other devices (e.g., an engine control unit). The system memory and / or storage devices may incorporate computer-readable media.

[0273] A computer system may include multiple identical components or subsystems connected to each other by an internal interface or by an external interface, for example, through a removable storage device connecting one component to another. In some embodiments, the computer systems, subsystems, or devices may communicate over a network.

[0274] Aspects of the embodiments may be implemented using hardware circuitry in the form of control logic (e.g., application-specific integrated circuits or field-programmable gate arrays) and / or using computer software with general-purpose programmable processors in a modular or integrated fashion. As used herein, a processor may include a single-core processor, a multi-core processor on the same integrated chip, or multiple processing units on a single circuit board or networked, as well as dedicated hardware. Based on this disclosure and the teachings provided herein, those skilled in the art will recognize and understand other ways and / or methods for implementing embodiments of the present invention using hardware and combinations of hardware and software.

[0275] Any of the software components or functions described in this application may be implemented as software code executed by a processing device using any suitable computer language, such as, for example, Java, C, C++, C#, Objective-C, Swift, or a scripting language, such as, for example, Perl or Python, using conventional or object-oriented techniques. The software code may be stored as a series of instructions or commands on a computer-readable medium for storage and / or transmission. Suitable non-transitory computer-readable media may include random access memory (RAM), read-only memory (ROM), magnetic media (such as a hard drive or floppy disk), or optical media (such as a compact disc (CD) or DVD (digital versatile disc)), flash memory, or the like. The computer-readable medium may also be any combination of such storage or transmission devices.

[0276] Such programs may also be coded and transmitted using carrier signals adapted for transmission over wired, optical, and / or wireless networks according to various protocols, including the Internet. Thus, computer-readable media may be created using data signals coded with such programs. Computer-readable media coded with program code may be packaged with a compatible device or provided separately from other devices (e.g., via Internet download). Any such computer-readable medium may reside on or within a single computer product (e.g., a hard drive, CD, or entire computer system), or may reside on or within different computer products within a system or network. A computer system may include a monitor, printer, or other suitable display for providing a user with any of the results described herein.

[0277] Any of the methods described herein may be implemented in whole or in part using a computer system including one or more processors that can be configured to perform the steps. Accordingly, embodiments may be directed to a computer system configured to perform the steps of any of the methods described herein, with potentially different components performing each step or group of steps. While presented as numbered steps, steps of the methods herein may be performed simultaneously or at different times, or in different orders. In addition, portions of these steps may be combined with portions of other steps from other methods. Also, all or portions of a step may be optional. Additionally, any steps of any of the methods may be performed by a module, unit, circuit, or other means of a system for performing these steps.

[0278] The specific details of the particular embodiments may be combined in any suitable manner without departing from the spirit and scope of the embodiments of the invention, however, other embodiments of the invention may be directed to particular embodiments relating to each individual aspect or particular combinations of these individual aspects.

[0279] The above description of exemplary embodiments of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form described, and many modifications and variations are possible in light of the above teachings.

[0280] The use of "a," "an," or "the" is intended to mean "one or more" unless specifically stated to the contrary. The use of "or" is intended to mean "inclusive or" rather than "exclusive or" unless specifically stated to the contrary. A reference to a "first" element does not necessarily require that a second element be provided. Furthermore, a reference to a "first" or "second" element does not limit the referenced elements to a particular location unless explicitly stated. The term "based on" is intended to mean "based at least in part on."

[0281] All patents, patent applications, publications, and specifications mentioned herein are incorporated by reference in their entirety for all purposes. None is admitted to be prior art. The scope of the claims at the time of filing is as follows: <Claim 1> 1. An optical measurement system comprising: a housing for the optical measurement system; and a light source configured to transmit one or more pulse trains over one or more time intervals as part of an optical measurement, each of the one or more first time intervals including one of the one or more pulse trains; an optical sensor configured to detect photons from the one or more pulse trains reflected from a housing of the optical measurement system and to detect photons from the one or more pulse trains reflected from objects in an environment surrounding the optical measurement system; and a plurality of registers configured to accumulate photon counts received from the photosensor during the one or more time intervals, each of the one or more time intervals being subdivided into a plurality of time bins, each of the plurality of registers configured to accumulate photon counts received during a corresponding one of the plurality of time bins in each of the one or more time intervals to represent a histogram of photon counts received during the one or more time intervals; a circuit configured to identify an initial peak in the histogram of the photon counts, the initial peak representing the photons reflected from the housing of the optical measurement system; and An optical measurement system comprising: <Claim 2> The circuitry is configured to identify the initial peak by identifying a predetermined number of registers within the plurality of registers that occur first within the plurality of registers. 2. The system of claim 1 . <Claim 3> The circuitry is configured to identify the initial peak by identifying one or more registers within the plurality of registers that store a maximum number of photon counts. 2. The system of claim 1 . <Claim 4> The circuitry is configured to identify the initial peak by identifying a register in the plurality of registers having a time bin corresponding to a distance between the light source and the housing of the optical measurement system. 2. The system of claim 1 . <Claim 5> The circuitry is further configured to identify a subset of the plurality of registers that represents the initial peak. 2. The system of claim 1 . <Claim 6> The subset of the plurality of registers is identified by selecting a predetermined number of registers surrounding a register storing a maximum value of the initial peak. 6. The system of claim 5. <Claim 7> The subset of the plurality of registers is identified by selecting a plurality of registers surrounding a register storing a maximum value of the initial peak, the plurality of registers storing values ​​within a predetermined percentage of the maximum value. 6. The system of claim 5. <Claim 8> The circuitry is further configured to estimate a distance between the light source and the housing of the optical measurement system based on a location of the initial peak in the plurality of registers. 2. The system of claim 1 . <Claim 9> The circuitry is further configured to calibrate distance measurements using the estimated distance between the light source and the housing. 9. The system of claim 8. <Claim 10> a processor configured to receive additional peaks in the histogram stored in the plurality of registers and calculate distances to objects in the environment corresponding to the additional peaks; Further provided with The initial peak is excluded from the additional peaks received by the processor. 2. The system of claim 1 . <Claim 11> The processor is implemented on an integrated circuit separate and different from the integrated circuit on which the plurality of registers are implemented. 11. The system of claim 10. <Claim 12> 1. A method for detecting a peak reflected from a housing in an optical measurement system, comprising: transmitting, as part of the optical measurement, one or more pulse trains over one or more time intervals, each of the one or more first time intervals including one of the one or more pulse trains; detecting photons from the one or more pulse trains that are reflected from the housing of the optical measurement system and detecting photons from the one or more pulse trains that are reflected from objects in an environment surrounding the optical measurement system; accumulating in a plurality of registers the counts of the photons received during the one or more time intervals, wherein each of the one or more time intervals is subdivided into a plurality of time bins, and each of the plurality of registers accumulates the photon counts received during a corresponding one of the plurality of time bins in each of the one or more time intervals such that the histogram of the photon counts received during the one or more time intervals is represented; identifying an initial peak in the histogram of the photon counts, the initial peak representing the photons reflected from the housing of the optical measurement system; A method comprising: <Claim 13> identifying a second initial peak as part of the second optical measurement; comparing the second initial peak to the initial peak; 13. The method of claim 12, further comprising: <Claim 14> characterizing a change in transparency of a window within a housing of the optical measurement system based on comparing the second initial peak to the initial peak. 14. The method of claim 13, further comprising: <Claim 15> identifying a plurality of initial peaks detected by a plurality of different optical sensors in the optical measurement system; determining a level of transparency of a corresponding section of a window in a housing of the optical measurement system in front of each of the plurality of optical sensors based on the plurality of initial peaks; 13. The method of claim 12, further comprising: <Claim 16> comparing the maximum value of the initial peak with a threshold; determining whether an obstruction is located outside the optical measurement system based on comparing the maximum value of the initial peak with the threshold value; 13. The method of claim 12, further comprising: <Claim 17> identifying a plurality of initial peaks across a plurality of measurements; storing a baseline initial peak based on a combination of the initial peaks across the plurality of measurements for comparison with future optical measurements; 13. The method of claim 12, further comprising: <Claim 18> subtracting the baseline initial peak from the plurality of registers. 18. The method of claim 17, further comprising: <Claim 19> a second peak at least partially overlapping the initial peak; The step of subtracting the baseline initial peak makes the second peak detectable by peak detection circuitry. 19. The method of claim 18. <Claim 20> The second peak corresponds to an object in the environment surrounding the optical measurement system that is within two feet of the optical measurement system. 20. The method of claim 19. <Claim 21> 1. An optical measurement system comprising: a light source configured to transmit one or more pulse trains over one or more time intervals as part of an optical measurement, each of the one or more time intervals including one of the one or more pulse trains; an optical sensor configured to detect photons from the one or more pulse trains reflected from objects in an environment surrounding the optical measurement system; a plurality of registers configured to accumulate photon counts received from the photosensor during the one or more time intervals to represent an unfiltered histogram of photon counts received during the one or more time intervals; a filter circuit configured to provide a filtered histogram of the photon counts from the plurality of registers; A peak detection circuit comprising: locating peaks in the filtered histogram; using the location of the peak in the filtered histogram to identify a location within the plurality of registers for storing an unfiltered representation of the peak; a peak detection circuit configured to: An optical measurement system comprising: <Claim 22> a processor configured to receive the unfiltered representation of the peak and to calculate a distance to the object in the environment surrounding the optical measurement system using the unfiltered representation of the peak. 22. The optical measurement system of claim 21 further comprising: <Claim 23> The filter circuit is configured to provide the filtered histogram by applying a matched filter corresponding to the one or more pulse trains. 22. Optical measurement system according to claim 21. <Claim 24> A pulse train in the one or more pulse trains includes a plurality of rectangular pulses. 22. Optical measurement system according to claim 21. <Claim 25> The filter circuit is configured to low-pass filter the unfiltered histogram. 22. Optical measurement system according to claim 21. <Claim 26> a second plurality of registers for storing the filtered histogram; 22. The optical measurement system of claim 21 further comprising: <Claim 27> The filtered histogram is generated in one pass through the plurality of registers. 22. Optical measurement system according to claim 21. <Claim 28> The peak is detected during the single pass through the plurality of registers so that the filtered histogram is not stored in its entirety. 28. Optical measurement system according to claim 27. <Claim 29> The peak detection circuit is configured to detect the location of the peak by detecting an increase in the values ​​of the plurality of registers followed by a decrease in the values. 22. Optical measurement system according to claim 21. <Claim 30> The processor is implemented in an integrated circuit (IC) separate and different from the integrated circuit (IC) in which the plurality of registers are implemented. 22. Optical measurement system according to claim 21. <Claim 31> The light source and the light sensor form a pixel within a plurality of pixels in the optical measurement system. 22. Optical measurement system according to claim 21. <Claim 32> 1. A method for analyzing filtered and unfiltered data in an optical measurement system, comprising: transmitting, as part of the optical measurement, one or more pulse trains over one or more first time intervals, each of the one or more first time intervals including one of the one or more pulse trains; detecting photons from the one or more pulse trains reflected from objects in an environment surrounding the optical measurement system; using the photons to populate a plurality of registers to represent an unfiltered histogram of photon counts received during the one or more first time intervals; filtering the unfiltered histogram in the plurality of registers to provide a filtered histogram of the photons from the plurality of registers; locating peaks in the filtered histogram; using the location of the peak in the filtered histogram to identify a location within the plurality of registers for storing an unfiltered representation of the peak; sending the unfiltered representation of the peaks to a processor and using the unfiltered representation of the peaks to calculate distances to the objects in the environment surrounding the optical measurement system; A method comprising: <Claim 33> transmitting the unfiltered representation of the peak includes transmitting information identifying histogram time bins represented in the plurality of registers storing the unfiltered representation of the peak. 33. The method of claim 32. <Claim 34> Filtering the unfiltered histogram in the plurality of registers includes applying a convolution of the unfiltered histogram with at least one rectangular filter having a plurality of identical values. 33. The method of claim 32. <Claim 35> The plurality of identical values ​​includes a sequence of binary "1" values ​​and / or a sequence of "-1" values. 35. The method of claim 34. <Claim 36> Filtering the unfiltered histogram in the plurality of registers includes convolving the unfiltered histogram with at least one sequence including a non-zero value followed by a plurality of zero values. 33. The method of claim 32. <Claim 37> The at least one sequence includes a single binary "1" value or a single binary "-1" value followed by multiple "0" values. 37. The method of claim 36. <Claim 38> transmitting a filtered representation of the peak to the processor in addition to transmitting the unfiltered representation of the peak. 33. The method of claim 32, further comprising: <Claim 39> the locations of the peaks in the filtered histogram are detected as a single peak in the filtered histogram; The unfiltered representation of the peaks includes at least two peaks in the unfiltered histogram. 33. The method of claim 32. <Claim 40> One of the at least two peaks in the unfiltered histogram represents a peak resulting from reflection of the one or more pulse trains from a housing or window of the optical measurement system. 40. The method of claim 39. <Claim 41> The photons are detected using a plurality of photodetectors in an optical sensor. 33. The method of claim 32. <Claim 42> 1. An optical measurement system comprising: a plurality of light sources configured to emit one or more pulse trains over one or more time intervals as part of the optical measurement; a plurality of optical sensors configured to detect reflected photons from the one or more pulse trains emitted from corresponding light sources in the plurality of light sources, the plurality of optical sensors including a first optical sensor and one or more other optical sensors spatially adjacent to the first optical sensor; a plurality of memory blocks configured to accumulate photon counts of the photons received during the one or more time intervals by corresponding photosensors in the plurality of photosensors to represent a plurality of histograms of photon counts, the plurality of histograms including a first histogram corresponding to the first photosensor and one or more other histograms corresponding to the one or more other photosensors; a circuit configured to combine information from the first histogram with information from the one or more other histograms to generate a distance measurement for the first photosensor; An optical measurement system comprising: <Claim 43> The one or more photosensors are physically adjacent to the first photosensor in the photosensor array. 43. Optical measurement system according to claim 42. <Claim 44> The photosensor array includes a solid-state photosensor array. 44. Optical measurement system according to claim 43. <Claim 45> The one or more photosensors include eight photosensors that are orthogonally adjacent or angularly adjacent to the first photosensor. 44. Optical measurement system according to claim 43. <Claim 46> The one or more photosensors are not physically adjacent to the first photosensor in the photosensor array, but the one or more photosensors are positioned to receive photons from a physical area adjacent to a physical area from which photons are received by the first photosensor. 43. Optical measurement system according to claim 42. <Claim 47> The plurality of optical sensors are arranged in an optical sensor array that rotates about a central axis of the optical measurement system. 47. Optical measurement system according to claim 46. <Claim 48> the information from the first histogram includes a first distance measure calculated based on the first histogram; the information from the one or more histograms includes one or more other distance measures calculated based on the one or more other histograms; The distance measurement includes a combination of the one or more other distance measurements with the first distance measurement. 43. Optical measurement system according to claim 42. <Claim 49> The first distance measurement is below a detection limit of the optical measurement system before combining the first distance measurement with the plurality of other distance measurements. 49. Optical measurement system according to claim 48. <Claim 50> The distance measurement, after combining the first distance measurement with the plurality of other distance measurements, exceeds the detection limit of the optical measurement system. 50. Optical measurement system according to claim 49. <Claim 51> The detection limit represents the minimum number of photons received by the corresponding optical sensor. 50. Optical measurement system according to claim 49. <Claim 52> The circuitry for combining the information from the first histogram with the information from the one or more histograms includes a processor implemented on an integrated circuit different from an integrated circuit on which the plurality of memory blocks are implemented. 43. Optical measurement system according to claim 42. <Claim 53> The circuit and the plurality of memory blocks are implemented on the same integrated circuit. 43. Optical measurement system according to claim 42. <Claim 54> 1. A method of using spatially adjacent pixel information in an optical measurement system, comprising: transmitting one or more pulse trains over one or more first time intervals as part of the optical measurement; detecting reflected photons from the one or more pulse trains using a plurality of optical sensors, the plurality of optical sensors including a first optical sensor and one or more optical sensors spatially adjacent to the first optical sensor; accumulating photon counts received by the plurality of light sensors during the one or more time intervals to represent a plurality of histograms of photon counts, the plurality of histograms including a first histogram corresponding to the first light sensor and one or more histograms corresponding to the one or more light sensors; combining information from the first histogram with information from the one or more histograms to generate a distance measurement for the first photosensor; A method comprising: <Claim 55> The reflected photons received by the first light sensor and received by the one or more light sensors are reflected from the same object in the surrounding environment. 55. The method of claim 54. <Claim 56> the information from the first histogram includes photon counts in the first histogram; the information from the one or more histograms includes photon counts in the one or more histograms; The distance measurement is calculated based on an aggregation of the photon counts in the first histogram and the photon counts in the one or more histograms. 55. The method of claim 54. <Claim 57> the information from the first histogram includes a first peak or peaks in the first histogram; the information from the one or more histograms includes a second one or more peaks in the one or more histograms; The distance measurement is calculated based on a combination of the first peak(s) and the second peak(s). 55. The method of claim 54. <Claim 58> The distance measurement is calculated based on a sum of the first peak(s) and the second peak(s). 58. The method of claim 57. <Claim 59> The distance measure is calculated based on a Gaussian combination of the first one or more peaks and the second one or more peaks. 58. The method of claim 57. <Claim 60> The distance measure is calculated based on a convolution of the first one or more peaks and the second one or more peaks. 58. The method of claim 57. <Claim 61> The distance measure is calculated based on a weighted combination of the first one or more peaks and the second one or more peaks. 58. The method of claim 57.

Claims

1. 1. An optical measurement system comprising: a housing for the optical measurement system; and a light source configured to transmit one or more pulse trains over one or more time intervals as part of an optical measurement, each of the one or more first time intervals including one of the one or more pulse trains; an optical sensor configured to detect photons from the one or more pulse trains reflected from a housing of the optical measurement system and to detect photons from the one or more pulse trains reflected from objects in an environment surrounding the optical measurement system; a plurality of registers configured to accumulate photon counts received from the photosensor during the one or more time intervals, each of the one or more time intervals being subdivided into a plurality of time bins, each of the plurality of registers configured to accumulate photon counts received during a corresponding one of the plurality of time bins in each of the one or more time intervals to represent a histogram of photon counts received during the one or more time intervals; a circuit configured to identify an initial peak in the histogram of the photon counts, the initial peak representing the photons reflected from the housing of the optical measurement system; and Equipped with the housing is a known distance from the light sensor and the light source; The circuitry is further configured to calibrate distance measurements of subsequent peaks following the initial peak by using the known distance of the housing from the light sensor and the light source to determine a time value represented by each of the plurality of time bins based on the known distance. A system characterized by:

2. The circuitry is configured to identify the initial peak by identifying a predetermined number of registers within the plurality of registers that occur first within the plurality of registers.

2. The system of claim 1.

3. The circuitry is configured to identify the initial peak by identifying one or more registers within the plurality of registers that store a maximum number of photon counts.

2. The system of claim 1.

4. The circuitry is configured to identify the initial peak by identifying a register in the plurality of registers having a time bin corresponding to a distance between the light source and the housing of the optical measurement system.

2. The system of claim 1.

5. The circuitry is further configured to identify a subset of the plurality of registers that represents the initial peak.

2. The system of claim 1.

6. The subset of the plurality of registers is identified by selecting a predetermined number of registers surrounding a register storing a maximum value of the initial peak.

6. The system of claim 5.

7. The subset of the plurality of registers is identified by selecting a plurality of registers surrounding a register storing a maximum value of the initial peak, the plurality of registers storing values ​​within a predetermined percentage of the maximum value.

6. The system of claim 5.

8. a processor configured to receive the subsequent peaks in the histogram stored in the plurality of registers and calculate distances to objects in the environment corresponding to the subsequent peaks; Further provided with 2. The system of claim 1.

9. The processor is implemented on an integrated circuit separate and different from the integrated circuit on which the plurality of registers are implemented.

9. The system of claim 8.

10. 1. A method for detecting a peak reflected from a housing in an optical measurement system, comprising: transmitting, as part of the optical measurement, one or more pulse trains over one or more time intervals, each of the one or more first time intervals including one of the one or more pulse trains; detecting photons from the one or more pulse trains that are reflected from the housing of the optical measurement system and detecting photons from the one or more pulse trains that are reflected from objects in an environment surrounding the optical measurement system; accumulating, in a plurality of registers, counts of the photons received during the one or more time intervals, wherein each of the one or more time intervals is subdivided into a plurality of time bins, and each of the plurality of registers accumulates photon counts received during a corresponding one of the plurality of time bins in each of the one or more time intervals, such that the histogram of photon counts received during the one or more time intervals is represented; identifying an initial peak in the histogram of the photon counts, the initial peak representing the photons reflected from the housing of the optical measurement system; determining a time value represented by each of the plurality of time bins based at least in part on a known distance between the housing and the optical measurement system; A method comprising:

11. identifying a second initial peak as part of the second optical measurement; comparing the second initial peak to the initial peak; 11. The method of claim 10, further comprising:

12. characterizing a change in transparency of a window within a housing of the optical measurement system based on comparing the second initial peak to the initial peak.

12. The method of claim 11 further comprising:

13. identifying a plurality of initial peaks across a plurality of measurements; storing a baseline initial peak based on a combination of the initial peaks across the plurality of measurements for comparison with future optical measurements; 11. The method of claim 10, further comprising:

14. subtracting the baseline initial peak from the plurality of registers.

14. The method of claim 13 further comprising:

15. a second peak at least partially overlapping the initial peak; The step of subtracting the baseline initial peak makes the second peak detectable by peak detection circuitry.

15. The method of claim 14.

16. The second peak corresponds to an object in the environment surrounding the optical measurement system that is within two feet of the optical measurement system.

16. The method of claim 15.

Images