Time series measurements to handle LIDAR accuracy

Abstract

A time series measurement to handle LIDAR accuracy is provided. An optical measurement system may include a light source and a corresponding light sensor configured to emit and detect photons reflected from an object in an ambient environment for optical measurement. Initial peaks may be identified as caused by reflections from an enclosure of the optical measurement system. This peak may be removed or used to calibrate the measurement calculation of the system. Peaks caused by reflections from surrounding objects may be processed using an on-chip filter to identify potential peaks, and unfiltered data may be communicated to an off-chip processor for distance calculations and other measurements. Spatial filtering techniques may be used to combine values from histograms of spatially adjacent pixels in an array of pixels. This combination can be used to improve the confidence of distance measurements.

Description

[0001] This application is a divisional application of the invention patent application filed on October 12, 2020, with application number 202080067525.0 and titled "Time Series Measurement for Processing LiDAR Accuracy".

[0002] Cross-citation of related applications This application claims the benefit of U.S. Provisional Patent Application No. 62 / 913,604, filed October 10, 2019, entitled “Proceduring time-series measurements for LIDAR accuracy,” which is incorporated herein by reference. Background Technology

[0003] Light Detection and Ranging (LIDAR) systems are used for object detection and ranging, for example, in vehicles such as cars, trucks, and ships. LIDAR systems are also used in mobile applications (e.g., facial recognition), home entertainment (e.g., capturing gestures for video game input), and augmented reality. A LIDAR system measures the distance to an object by illuminating a landscape with pulses from a laser and then measuring the time it takes for photons to travel to the object and return after reflection, as measured by the LIDAR system's receiver. The detected signal is analyzed to detect the presence of reflected signal pulses in the background light. The distance to the object can be determined based on the time of flight from the transmission of the pulse to the reception of the corresponding reflected pulse.

[0004] To be precise, given the economic cost of LIDAR systems, it may be difficult to provide robust distance accuracy down to a few centimeters under all conditions. Promising new detector technologies such as single-photon avalanche diodes (SPADs) are attractive, but they suffer from significant drawbacks when used to measure time of flight and other signal characteristics due to their limited dynamic range, particularly across a wide range of environmental conditions and target distances. Furthermore, SPADs are highly susceptible to ambient noise levels due to their sensitivity to even small numbers of photons. Summary of the Invention

[0005] In some embodiments, the optical measurement system may include a housing of the optical measurement system and a light source configured to transmit one or more pulse trains in one or more time intervals as part of the optical measurement, wherein each of the one or more first time intervals may include one of the one or more pulse trains. The system may also include a light sensor configured to detect photons from one or more pulse trains reflected from the housing of the optical measurement system, and to detect photons from one or more pulse trains reflected from objects in the environment surrounding the optical measurement system. The system may additionally include a plurality of registers configured to accumulate photon counts received from the light sensor during the one or more time intervals. Each of the one or more time intervals may be subdivided into a plurality of time intervals. Each of the plurality of registers may be configured to accumulate photon counts received in a corresponding period of the plurality of time intervals in each of the one or more time intervals to represent a histogram of photon counts received during the one or more time intervals. The system may also include circuitry configured to identify an initial peak in the histogram of photon counts. The initial peak may represent photons reflected from the housing of the optical measurement system.

[0006] In any embodiment, any and all of the following features may be included in any combination, but without limitation. The circuit may be configured to identify an initial peak by identifying a predetermined number of registers among a plurality of registers that first appear in a plurality of registers. The circuit may be configured to identify an initial peak by identifying one or more registers among a plurality of registers storing the highest number of photon counts. The circuit may be configured to identify an initial peak by identifying registers among a plurality of registers having a time interval corresponding to the distance between the light source and the housing of the optical measurement system. The circuit may also be configured to identify a subset of the plurality of registers representing the initial peak. A subset of the plurality of registers may be identified by selecting a predetermined number of registers near the register storing the maximum value of the initial peak. A subset of the plurality of registers may be identified by selecting registers near the register storing the maximum value of the initial peak that store values ​​within a predetermined percentage of the maximum value. The circuit may also be configured to estimate the distance between the light source and the housing of the optical measurement system based on the position of the initial peak in the plurality of registers. The circuit may also be configured to calibrate distance measurements using the estimated distance between the light source and the housing. The system may also include a processor configured to receive additional peaks from a histogram stored in a plurality of registers to calculate distances to objects in the surrounding environment corresponding to the additional peaks, wherein initial peaks may be excluded from the additional peaks received by the processor. The processor may be implemented in an integrated circuit that is separate from and distinct from the integrated circuit in which the plurality of registers are implemented.

[0007] In some embodiments, a method for detecting peaks reflected from a housing in an optical measurement system may include transmitting one or more pulse trains as part of an optical measurement within one or more time intervals. Each of the one or more first time intervals may include one of the one or more pulse trains. The method may also include detecting photons from the one or more pulse trains reflected from the housing of the optical measurement system, and detecting photons from the one or more pulse trains reflected from an object in the environment surrounding the optical measurement system. The method may additionally include accumulating counts of photons received during the one or more time intervals into a plurality of registers. Each of the one or more time intervals may be subdivided into a plurality of time intervals, and each of the plurality of registers accumulates the photon counts received in a corresponding period in the plurality of time intervals within each of the one or more time intervals to represent a histogram of photon counts received during the one or more time intervals. The method may further include identifying an initial peak in the histogram of photon counts. The initial peak may represent photons reflected from the housing of the optical measurement system.

[0008] In any embodiment, any and all of the following features may be included in any combination, but are not limited to. The method may also include, as part of a second optical measurement, identifying a second initial peak; and comparing the second initial peak with an initial peak. The method may also include characterizing a change in the transparency of a window in the housing of the optical measurement system based on a comparison of the second initial peak with an initial peak. The method may further include identifying multiple initial peaks detected by multiple different light sensors in the optical measurement system; and determining, based on the multiple initial peaks, the level of transparency of a corresponding segment of the window in the housing of the optical measurement system in front of each of the multiple light sensors. The method may further include comparing a maximum value of the initial peaks with a threshold; and determining, based on the comparison of the maximum value of the initial peaks with the threshold, whether a blockage is located outside the optical measurement system. The method may further include identifying multiple initial peaks among multiple measurements; and storing a baseline initial peak based on a combination of the multiple initial peaks among the multiple measurements for comparison with further optical measurements. The method may further include subtracting the baseline initial peak from multiple registers. The second peak may at least partially overlap with the initial peak, and subtracting the baseline initial peak may cause the second peak to be detectable by a peak detection circuit. The second peak can correspond to an object within two feet of the optical measurement system in the environment surrounding the optical measurement system.

[0009] In some embodiments, the optical measurement system may include a light source configured to transmit one or more pulse trains during one or more time intervals as part of an optical measurement. Each of the one or more time intervals may include one of the one or more pulse trains. The system may also include a photosensor configured to detect photons from one or more pulse trains reflected from an object in the environment surrounding the optical measurement system. The system may additionally include a plurality of registers configured to accumulate photon counts received from the photosensor during the one or more time intervals to represent an unfiltered histogram of photon counts received during the one or more time intervals. The system may also include filtering circuitry configured to provide a filtered histogram of photon counts from the plurality of registers. The system may also include peak detection circuitry configured to detect the position of a peak in the filtered histogram and use the position of the peak in the filtered histogram to identify the position in a plurality of registers storing the unfiltered representation of the peak.

[0010] In any embodiment, any and all of the following features may be included in any combination, but without limitation. The system may also include a processor configured to: receive an unfiltered representation of a peak and use the unfiltered representation of the peak to calculate the distance to an object in the environment surrounding the optical measurement system. A filtering circuit may be configured to provide a filtered histogram by applying a matched filter corresponding to one or more pulse trains. One of the pulse trains may contain multiple square pulses. The filtering circuit may be configured to low-pass filter the unfiltered histogram. The system may also include a second plurality of registers storing the filtered histogram. The filtered histogram may be generated in a single pass through the plurality of registers. Peaks may be detected in the plurality of registers during a single pass, such that not all of the filtered histogram is stored. A peak detection circuit may be configured to detect the location of a peak by detecting an increase in value following a decrease in value in the plurality of registers. The processor may be implemented on an integrated circuit (IC) that is separate from and distinct from the IC on which the plurality of registers are implemented. A light source and a light sensor may form pixels among a plurality of pixels in the optical measurement system.

[0011] In some embodiments, a method for analyzing filtered and unfiltered data in an optical measurement system may include transmitting one or more pulse trains as part of an optical measurement within one or more first time intervals. Each of the one or more first time intervals may include one of the one or more pulse trains. The method may also include detecting photons from one or more pulse trains reflected from an object in the environment surrounding the optical measurement system; using the photons to populate a plurality of registers to represent an unfiltered histogram of photon counts received during the one or more first time intervals; filtering the unfiltered histogram in the plurality of registers to provide a filtered histogram of photons from the plurality of registers; detecting the position of a peak in the filtered histogram; using the position of the peak in the filtered histogram to identify the position in a plurality of registers storing an unfiltered representation of the peak; and sending the unfiltered representation of the peak to a processor to calculate the distance to an object in the environment surrounding the optical measurement system using the unfiltered representation of the peak.

[0012] In any embodiment, any and all of the following features may be included in any combination, but without limitation: Sending the unfiltered representation of the peak may include information about the time interval of an identification histogram represented in a plurality of registers storing the unfiltered representation of the peak. Filtering the unfiltered histogram in the plurality of registers may include applying a convolution of the unfiltered histogram with at least one square filter having a plurality of identical values. The plurality of identical values ​​may include a series of binary "1" values ​​and / or a series of "-1" values. Filtering the unfiltered histogram in the plurality of registers may include convolving the unfiltered histogram with at least one sequence of non-zero values ​​following a plurality of zero values. The at least one sequence may include a single binary "1" value or a single binary "-1" value following a plurality of "0" values. The method may also include sending a filtered representation of the peak to a processor, in addition to sending the unfiltered representation of the peak. The position of the peak in the filtered histogram may be detected as a single peak in the filtered histogram; and the unfiltered representation of the peak may include at least two peaks in the unfiltered histogram. One of the at least two peaks in the unfiltered histogram may represent a peak caused by reflection from the housing or window of the optical measurement system by one or more pulse trains. Photons can be detected using multiple photodetectors in an optical sensor.

[0013] In some embodiments, the optical measurement system may include a plurality of light sources configured to emit one or more pulse trains at one or more time intervals as part of the optical measurement. The system may also include a plurality of optical sensors configured to detect reflected photons from one or more pulse trains emitted from corresponding light sources among the plurality of light sources. The plurality of optical sensors may include a first optical sensor and one or more other optical sensors spatially adjacent to the first optical sensor. The system may additionally include a plurality of memory blocks configured to accumulate photon counts of photons received during one or more time intervals using corresponding optical sensors among the plurality of optical sensors to represent a plurality of histograms of photon counts. The plurality of histograms may include a first histogram corresponding to the first optical sensor and one or more histograms corresponding to one or more other optical sensors. The system may also include circuitry configured to combine information from the first histogram with information from one or more other histograms to generate a distance measurement for the first optical sensor.

[0014] In any embodiment, any and all of the following features may be included in any combination, but without limitation: One or more optical sensors may be physically adjacent to a first optical sensor in an optical sensor array. The optical sensor array may comprise a solid-state array of optical sensors. One or more optical sensors may comprise eight optical sensors orthogonally or diagonally adjacent to the first optical sensor. One or more optical sensors do not need to be physically adjacent to the first optical sensor in the optical sensor array, but one or more optical sensors may be positioned to receive photons from a physical region adjacent to the physical region from which photons are received by the first optical sensor. A plurality of optical sensors may be arranged as an optical sensor array rotating about the central axis of the optical measurement system. Information from a first histogram may include a first distance measurement calculated based on the first histogram; information from one or more histograms may include one or more other distance measurements calculated based on one or more other histograms; and the distance measurement may include a combination of the first distance measurement and one or more other distance measurements. The first distance measurement may be below the detection limit of the optical measurement system before being combined with the plurality of other distance measurements. After combining a first distance measurement with multiple other distance measurements, the distance measurement can exceed the detection limit of the optical measurement system. The detection limit can represent the minimum number of photons received by the corresponding optical sensor. The circuitry for combining information from the first histogram with information from one or more histograms may include a processor implemented on an integrated circuit, which is different from an integrated circuit on which multiple memory blocks are implemented. The circuitry and the multiple memory blocks may be implemented on the same integrated circuit.

[0015] In some embodiments, a method using spatially adjacent pixel information in an optical measurement system may include transmitting one or more pulse trains as part of an optical measurement within one or more first time intervals; and using a plurality of optical sensors to detect reflected photons from the one or more pulse trains. The plurality of optical sensors may include a first optical sensor and one or more optical sensors spatially adjacent to the first optical sensor. The method may also include accumulating a plurality of histograms representing photon counts by means of the plurality of optical sensors during the one or more time intervals. The plurality of histograms may include a first histogram corresponding to the first optical sensor and one or more histograms corresponding to the one or more optical sensors. The method may further include combining information from the first histogram with information from the one or more histograms to generate a distance measurement for the first optical sensor.

[0016] In any embodiment, any and all of the following features may be included in any combination, but without limitation: The reflected photons received by the first optical sensor and by one or more optical sensors can be reflected from the same object in the surrounding environment. Information from the first histogram may include photon counts in the first histogram; information from one or more histograms may include photon counts in one or more histograms; and a distance measurement may be calculated based on the aggregation of photon counts in the first histogram and photon counts in one or more histograms. Information from the first histogram may include one or more first peaks in the first histogram; information from one or more histograms may include one or more second peaks in one or more histograms; and a distance measurement may be calculated based on a combination of the first or more peaks and the second or more peaks. A distance measurement may be calculated based on the sum of the first or more peaks and the second or more peaks. A distance measurement may be calculated based on a Gaussian combination of the first or more peaks and the second or more peaks. A distance measurement may be calculated based on the convolution of the first or more peaks and the second or more peaks. A distance measurement may be calculated based on a weighted combination of the first or more peaks and the second or more peaks. Attached Figure Description

[0017] The remainder of the specification and the drawings provide a further understanding of the nature and advantages of the various embodiments, wherein the same reference numerals are used in several drawings to refer to similar components. In some cases, a sublabel is associated with a reference numeral to identify one of a plurality of similar components. When a reference numeral is referenced without specifying a sublabel, the reference numeral refers to all such plurality of similar components.

[0018] Figure 1A and 1B An automotive optical ranging device, also referred to herein as a LIDAR system, is shown according to some embodiments.

[0019] Figure 2 A block diagram of an exemplary LIDAR device for implementing various embodiments is shown.

[0020] Figure 3 The operation of a typical LIDAR system can be improved through examples.

[0021] Figure 4 Illustrative examples of optical transmission and detection processes for an optical ranging system are shown according to some embodiments.

[0022] Figure 5 The various stages of the sensor array and associated electronics according to an embodiment of the present invention are shown.

[0023] Figure 6 A histogram is shown according to an embodiment of the present invention.

[0024] Figure 7 The diagram illustrates the accumulation of histograms over multiple pulse trains for a selected pixel according to an embodiment of the present invention.

[0025] Figure 8 A circuit according to some embodiments is shown for receiving photons and generating a set of signals stored in a memory representing a histogram.

[0026] Figure 9 The timing associated with different shots in the measurement is shown according to some embodiments.

[0027] Figure 10 This describes the representation of a histogram memory containing peak values ​​caused by reflections from the housing of the optical measurement system.

[0028] Figure 11 This is a flowchart illustrating a method for using an optical measurement system to detect peaks caused by earlier reflections from the system housing.

[0029] Figure 12 This describes the contents of a histogram memory, according to some embodiments, that can be used to calibrate distance measurements for optical measurement systems.

[0030] Figure 13A This describes a portion of the histogram memory that receives the initial peak value corresponding to the reflection from the system enclosure, according to some embodiments.

[0031] Figure 13B This describes the variation in the magnitude of the peak reflected from the casing according to some embodiments.

[0032] Figure 14 This describes a portion of a light sensor array that can be partially obscured by window contamination, according to some embodiments.

[0033] Figure 15A This describes the initial peak value detected by the optical measurement system relative to the blocking threshold, according to some embodiments.

[0034] Figure 15B This describes the initial peak value in the later stages of the lifespan of an optical measurement system when a blockage is present, according to some embodiments.

[0035] Figure 16A This describes a histogram memory according to some embodiments, having an initial peak caused by reflection from the outer casing and a second peak caused by reflection from objects in the surrounding environment.

[0036] Figure 16B This describes a short-range peak detection problem that does not compensate for reflections exiting the housing, according to some embodiments.

[0037] Figure 17A illustrate Figure 16B Histogram memory.

[0038] Figure 17B This indicates the value after subtracting the baseline peak from the register in the histogram memory. Figure 16B Histogram memory.

[0039] Figure 18 This describes a circuit, according to some embodiments, for removing the effects of reflections from the system housing during short-range measurements.

[0040] Figure 19 This describes the contents of a histogram memory after a single pulse is transmitted and received by an optical measurement system, according to some embodiments.

[0041] Figure 20 Description of usage according to some embodiments Figure 19 The filtered version of the histogram data of the filter.

[0042] Figure 21A This illustrates an example of two adjacent peaks that are relatively close in time.

[0043] Figure 21B The low-pass filter according to some embodiments can be used for... Figure 21A The impact of unfiltered histogram data.

[0044] Figures 22A to 22B This illustrates how filtered data, according to some embodiments, can be used to identify peaks in unfiltered data.

[0045] Figure 23 This illustration shows a circuit according to some embodiments for using filtered data to transmit unfiltered data to a processor for distance calculation.

[0046] Figure 24 This describes a portion of a histogram memory following the reception of reflected photons from a multi-pulse code, according to some embodiments.

[0047] Figure 25 This describes a filtered version of the peak received from a multi-pulse code according to some embodiments.

[0048] Figure 26 This describes a filter that uses only a single binary indicator according to some embodiments.

[0049] Figure 27 A flowchart illustrating a method for analyzing filtered and unfiltered data in an optical measurement system according to some embodiments.

[0050] Figure 28 Examples of objects that, according to some embodiments, can provide reflected photons to a nearby optical sensor are described.

[0051] Figure 29 Examples illustrating how the spatially proximate viewpoint of a light sensor, according to some embodiments, can be used to calculate a distance measurement value for one of the corresponding light sensors.

[0052] Figure 30 This illustrates how histograms, according to some embodiments, can be combined for use with proximity optical sensors.

[0053] Figure 31 Examples of rectangular light sensor layouts according to some embodiments are described.

[0054] Figure 32 This describes the configuration of a rotating optical measurement system according to some embodiments.

[0055] Figure 33 This describes a circuit according to some embodiments for combining information from spatially adjacent histograms.

[0056] Figure 34 This describes alternative circuitry for combining information from a histogram, according to some embodiments.

[0057] Figure 35 This describes another circuit, according to some embodiments, for combining information from a histogram.

[0058] Figure 36 A flowchart illustrating a method for using spatially neighboring pixel information in an optical measurement system.

[0059] the term the term" Distance measurement"Especially when used in the context of methods and apparatuses for measuring the environment or assisting in the operation of vehicles, it can refer to determining the distance or distance vector from one location or position to another." Optical ranging "Optical ranging can refer to a class of ranging methods that utilize electromagnetic waves to perform ranging methods or functions. Therefore, 'optical ranging device' can refer to a device used to perform optical ranging methods or functions." Lidar "or" LIDAR "This can refer to a class of optical ranging methods that measure the distance to a target by illuminating it with a pulsed laser and then measuring the reflected pulse with a sensor. Therefore, " lidar device "or" lidar system "This can refer to a class of optical ranging devices used to perform lidar methods or functions." Optical ranging system "Can refer to a system that includes at least one optical ranging device (e.g., a lidar device). The system may also include one or more other devices or components arranged in various ways."

[0060] “ pulse train "Can refer to one or more pulses transmitted together. The transmission and detection of a pulse train can be called 'pulse train'." Excitement hair "Stimulation can be achieved in " Detection time interval "(or" Detection interval This occurred in 》.

[0061] “ Measurement "It can contain N pulse trains emitted and detected in N excitations, with each excitation lasting for a detection time interval. The entire measurement can be performed within the measurement time interval (or just...") Measurement interval In ), it can be equal to N detection intervals or longer, for example, when a pause occurs between detection intervals.

[0062] “ Light sensor "or" Photosensitive element "It can convert light into electrical signals. A light sensor can contain multiple..." Light detection instrument For example, a single-photon avalanche diode (SPAD). Optical sensors can correspond to specific resolution pixels in ranging measurements.

[0063] “ Histogram"A histogram can refer to any data structure representing a series of values ​​over time, such as values ​​discrete over time intervals. A histogram can have values ​​assigned to each time interval. For example, a histogram can store a counter of the number of photodetectors activated during a specific time interval in each of one or more detection intervals. As another example, a histogram can correspond to the digitization of an analog signal at different times. A histogram can contain both signal (e.g., pulses) and noise. Therefore, a histogram can be viewed as a combination of signal and noise as a photon time series or photon flux. The raw / digitized histogram (or accumulated photon time series) can contain the signal and noise digitized in memory without filtering." filter Wave histogram "Can refer to the output after the original histogram has passed through the filter."

[0064] The transmitted signal / pulse can refer to a distortion-free "nominal," "ideal," or "template" pulse or pulse train. The reflected signal / pulse can refer to a reflected laser pulse from an object and may be distorted. The digitized signal / pulse (or raw signal) can refer to the digitized result of detection from one or more pulse trains, such as those stored in memory, and is therefore equivalent to a portion of a histogram. The detected signal / pulse can refer to the location of the detected signal in memory. The detected pulse train can refer to the actual pulse train found by the matched filter. The expected signal profile can refer to the shape of the digitized signal caused by a specific transmitted signal with specific distortion in the reflected signal. Detailed Implementation

[0065] This disclosure generally relates to the field of object detection and ranging, and more specifically, to the use of time-of-flight optical receiver systems for applications such as real-time 3D mapping and object detection, tracking, and / or classification. Various improvements can be made with the aid of various embodiments of the invention.

[0066] The following sections introduce an illustrative automotive LIDAR system, followed by a description of example techniques for detecting signals using an optical ranging system, and then describe different embodiments in more detail.

[0067] I. Illustrative Automotive LIDAR System Figure 1A and 1BAn automotive optical ranging device, also referred to herein as a LIDAR system, is illustrated according to some embodiments. The automotive application of the LIDAR system is chosen herein for illustrative purposes only, and the sensors described herein can be used in other types of transportation, such as ships, airplanes, trains, etc., and in a variety of other applications where 3D depth images are useful, such as medical imaging, mobile phones, augmented reality, geodesy, geospatial informatics, archaeology, topography, geology, geomorphology, seismology, forestry, atmospheric physics, laser guidance, airborne laser strip mapping (ALSM), and laser altimetry. According to some embodiments, the LIDAR system, such as scanning LIDAR system 101 and / or solid-state LIDAR system 103, can be mounted on the roof of vehicle 105, such as... Figure 1A and 1B As shown in the image.

[0068] Figure 1A The scanning LIDAR system 101 shown may employ a scanning architecture in which the orientation of the LIDAR light source 107 and / or detector circuitry 109 can be scanned near one or more fields of view 110 in an external field or scene outside the vehicle 105. In the case of the scanning architecture, the emitted light 111 may scan the surrounding environment as shown. For example, the output beam of one or more light sources (such as infrared or near-infrared pulsed IR lasers, not shown) in the LIDAR system 101 may be scanned (e.g., rotated) to illuminate the scene near the vehicle. In some embodiments, the scan, indicated by the rotating arrow 115, may be implemented by mechanical means, such as by mounting the light emitter to a rotating column or platform. In some embodiments, the scan may be implemented by other mechanical means, such as by using a galvanometer. Chip-based steering techniques may also be employed, such as by using a microchip employing one or more MEMS-based reflectors, such as digital micromirror devices (DMD devices), digital light processing (DLP) devices, and the like. In some embodiments, the scan may be achieved by non-mechanical means, such as by using electronic signals to steer one or more optical phased arrays.

[0069] For fixed architectures, such as Figure 1B The solid-state LiDAR system 103 shown herein may be mounted to vehicle 105, and one or more solid-state LiDAR subsystems (e.g., 103a and 103b) may be mounted thereto. Each solid-state LiDAR unit may face different orientations (with potentially partially overlapping and / or non-overlapping fields of view between units) in order to capture a larger field of view than each unit itself can capture.

[0070] In a scanning or fixed architecture, an object within the scene may reflect portions of a light pulse emitted from a LIDAR source. One or more reflective portions then travel back to the LIDAR system and can be detected by detector circuitry. For example, reflective portion 117 can be detected by detector circuitry 109. The detector circuitry can be housed in the same enclosure as the transmitter. The scanning and fixed system aspects are not mutually exclusive and can therefore be used in combination. For example, Figure 1B Individual LIDAR subsystems 103a and 103b may employ steerable transmitters, such as optical phased arrays, or the entire composite unit may be mechanically rotated to scan the entire scene in front of the LIDAR system, for example, from field of view 119 to field of view 121.

[0071] Figure 2 A more detailed block diagram of a rotating LiDAR system 200 according to some embodiments is provided. More specifically, Figure 2 Optionally, a rotary LIDAR system employing a rotary actuator on a rotating circuit board is described, which can receive power and data (as well as transmit power and data) from a fixed circuit board.

[0072] LIDAR system 200 can interact with one or more examples of user interface 215. Different examples of user interface 215 can vary and may include, for example, a computer system with a monitor, keyboard, mouse, CPU, and memory; a touchscreen in a car; a handheld device with a touchscreen; or any other suitable user interface. User interface 215 may be local to the object on which LIDAR system 200 is installed, but it can also be a remotely operated system. For example, commands and data to / from LIDAR system 200 can be routed via cellular networks (LTE, etc.), personal area networks (Bluetooth, Zigbee, etc.), local area networks (WiFi, IR, etc.), or wide area networks such as the Internet.

[0073] The hardware and software user interface 215 can present LIDAR data from the device to a user, but can also allow the user to control the LIDAR system 200 with one or more commands. Example commands may include commands to activate or deactivate the LIDAR system, specify light detector exposure levels, bias, sampling duration, and other operating parameters (e.g., emission pulse mode and signal processing), and specify light emitter parameters (e.g., brightness). Furthermore, commands may allow the user to select a method for displaying the results. The user interface can display LIDAR system results, which may include, for example, a single-frame snapshot image, a constantly updated video image, and / or other light measurements of some or all pixels. In some embodiments, the user interface 215 may track the distance (proximity) of an object to a vehicle and potentially provide alerts to the driver or provide such tracking information for analysis of driver performance.

[0074] In some embodiments, the LIDAR system can communicate with the vehicle control unit 217 and can modify one or more parameters associated with vehicle control based on received LIDAR data. For example, in a fully autonomous vehicle, the LIDAR system can provide real-time 3D images of the vehicle's surroundings to assist navigation. In other cases, the LIDAR system can be used as part of an advanced driver assistance system (ADAS) or a safety system, which can, for example, provide 3D image data to any number of different systems, such as adaptive cruise control, automatic parking, driver drowsiness monitoring, blind spot monitoring, collision avoidance systems, etc. When the vehicle control unit 217 is communicatively coupled to the optical ranging device 210, it can provide the driver with warnings or tracking of the proximity of trackable objects.

[0075] Figure 2 The LIDAR system 200 shown includes an optical ranging device 210. The optical ranging device 210 includes a ranging system controller 250, an optical transmission (Tx) module 240, and an optical sensing (Rx) module 230. Ranging data can be generated by the optical ranging device by transmitting one or more light pulses 249 from the optical transmission module 240 to objects in the field of view surrounding the optical ranging device. The reflected portion 239 of the transmitted light is then detected by the optical sensing module 230 after a delay. Based on the delay time, the distance to the reflecting surface can be determined. Other ranging methods, such as continuous wave, Doppler, and similar methods, can also be used.

[0076] Tx module 240 includes an emitter array 242, which may be a one-dimensional or two-dimensional emitter array, and a Tx optical system 244, which, when combined, form a micro-optical emitter channel array. The emitter array 242 or individual emitters are instances of laser sources. Tx module 240 also includes a processor 245 and a memory 246. In some embodiments, pulse coding techniques, such as Barker codes, may be used. In such cases, memory 246 may store pulse codes indicating when light should be transmitted. In one embodiment, the pulse codes are stored as integer sequences in memory.

[0077] Rx module 230 may include sensor array 236, which may be, for example, a one-dimensional or two-dimensional light sensor array. Each light sensor or photosensitive element (also referred to as a sensor) may include a series of photodetectors, such as APDs or the like, or the sensor may be a single photon detector (e.g., a SPAD). Similar to Tx module 240, Rx module 230 includes Rx optical system 237. The combined Rx optical system 237 and sensor array 236 may form a micro-optical receiver channel array. Each micro-optical receiver channel measures light corresponding to an image pixel in a dissimilar field of view of the surrounding volume. For example, due to the geometric configuration of light sensing module 230 and light transmission module 240, each sensor (e.g., a series of SPADs) of sensor array 236 may correspond to a specific emitter of emitter array 242.

[0078] In one embodiment, the sensor array 236 of the Rx module 230 is fabricated as part of a monolithic device on a single substrate (using, for example, CMOS technology), comprising an array of photon detectors and an ASIC 231 for signal processing of the raw histograms from the individual photon detectors (or groups of detectors) in the array. As an example of signal processing, for each photon detector or group of photon detectors, the memory 234 of the ASIC 231 (e.g., SRAM) can accumulate counts of photons detected over consecutive time intervals, and these time intervals can be combined to recreate a time series of reflected light pulses (i.e., photon counts versus time). This time series of aggregated photon counts is referred to herein as an intensity histogram (or histogram only). The ASIC 231 can implement matched filtering and peak detection processing to timely identify the returned signal. Furthermore, the ASIC 231 can implement certain signal processing techniques (e.g., via processor 238), such as multi-curve matched filtering, to help recover photon time series less susceptible to pulse shape distortion due to SPAD saturation and quenching. In some embodiments, all or part of such filtering can be performed by a processor 258 that can be implemented in an FPGA.

[0079] In some embodiments, the Rx optical system 237 may also be part of a monolithic structure similar to an ASIC, having a separate substrate layer for each receiver channel layer. For example, the aperture layer, collimating lens layer, filter layer, and photodetector layer may be stacked and bonded at the wafer level prior to dicing. The aperture layer may be formed by placing an opaque substrate on top of a transparent substrate or by coating a transparent substrate with an opaque film. In yet another embodiment, one or more components of the Rx module 230 may be external to the monolithic structure. For example, the aperture layer may be implemented as a separate metal sheet with pinholes.

[0080] In some embodiments, the photon time series output from the ASIC is sent to the ranging system controller 250 for further processing. For example, the data may be encoded by one or more encoders of the ranging system controller 250 and then sent as data packets to the user interface 215. The ranging system controller 250 can be implemented in several ways, including, for example, by using a programmable logic device such as an FPGA as part of the ASIC, using a processor 258 with memory 254, and a combination thereof. The ranging system controller 250 may cooperate with or operate independently of a fixed base controller (via pre-programmed instructions) to control the light sensing module 230 by sending commands including starting and stopping light detection and adjusting photodetector parameters. Similarly, the ranging system controller 250 may control the optical transmission module 240 by sending commands or relaying commands from the base controller, including starting and stopping emission control and adjusting other light emitter parameters (e.g., pulse codes). In some embodiments, the ranging system controller 250 has one or more wired interfaces or connectors for exchanging data with the light sensing module 230 and the optical transmission module 240. In other embodiments, the ranging system controller 250 communicates with the light sensing module 230 and the light transmission module 240 via wireless interconnection, such as an optical communication link.

[0081] The electric motor 260 can be an optional component required when system components, such as the Tx module 240 and / or Rx module 230, need to rotate. The system controller 250 controls the electric motor 260 and can start, stop, and change its speed.

[0082] II. Detection of reflected pulses Optical sensors can be arranged in various ways to detect reflected pulses. For example, optical sensors can be arranged in an array, and each optical sensor can contain an array of photodetectors (e.g., SPADs). Different modes of pulses (pulse trains) transmitted during the detection interval are also described below.

[0083] A. Time-of-flight measurement and detector Figure 3 This describes the operation of a typical LIDAR system that can be improved by some embodiments. The laser generates short-duration optical pulses 310. The horizontal axis represents time, and the vertical axis represents power. The duration of the example laser pulse, characterized by full width at half maximum (FWHM), is a few nanoseconds, with a peak power of approximately a few watts for a single emitter. Embodiments using side-emitter lasers or fiber lasers can have much higher peak power, while embodiments with small-diameter VCSELs can have peak power ranging from tens to hundreds of milliwatts.

[0084] The start time 315 of pulse transmission does not need to coincide with the leading edge of the pulse. As shown, the leading edge of the optical pulse 310 may be after the start time 315. It may be desirable for the leading edge to differ in cases where pulses of different modes are transmitted at different times, for example, for coded pulses.

[0085] The optical receiver system can begin detecting received light simultaneously with the start of the laser, i.e., at the start time. In other embodiments, the optical receiver system can begin at a later time, a known time after the start time of the pulse. The optical receiver system initially detects background light 330 and then detects laser pulse reflection 320 after some time. The optical receiver system can compare the detected light intensity with a threshold to identify laser pulse reflection 320. The threshold distinguishes between background light 330 and light corresponding to laser pulse reflection 320.

[0086] The time of flight 340 is the time difference between the transmitted pulse and the received pulse. This time difference can be measured by subtracting the pulse's propagation time (e.g., as measured relative to the start time) from the reception time of the laser pulse reflection 320 (e.g., also measured relative to the start time). The distance to the target can be determined as half the product of the time of flight and the speed of light. Pulses from the laser device are reflected from objects in the scene at different times, and the pixel array detects the reflected pulses.

[0087] B. Object detection using arrayed lasers and optical sensor arrays Figure 4 Illustrative examples of optical transmission and detection processes for an optical ranging system are shown according to some embodiments. Figure 4 An optical ranging system (e.g., solid-state and / or scanning) is shown that collects three-dimensional distance data of the volume or scene around the system. Figure 4 This is an idealized diagram highlighting the relationship between the transmitter and the sensor, and therefore other components are not shown.

[0088] The optical ranging system 400 includes an array of optical emitters 402 and an array of optical sensors 404. The optical emitter array 402 includes an array of optical emitters, such as a VCSEL array and the like, such as emitters 403 and 409. The optical sensor array 404 includes an array of optical sensors, such as sensors 413 and 415. The optical sensors can be pixelated optical sensors, employing a set of discrete photodetectors for each pixel, such as single-photon avalanche diodes (SPADs) and the like. However, various embodiments can deploy any type of photonic sensor.

[0089] Each emitter may be slightly offset from its neighbor and may be configured to transmit light pulses into a different field of view than its neighboring emitter, thereby illuminating only the corresponding field of view associated with that emitter. For example, emitter 403 emits an illumination beam 405 (formed by one or more light pulses) into a circular field of view 407 (magnified for clarity). Similarly, emitter 409 emits an illumination beam 406 (also referred to as the emitter channel) into a circular field of view 410. Although Figure 4 Not shown to avoid complexity, but each emitter emits a corresponding illumination beam into its corresponding field of view, thus causing a 2D array of irradiated fields of view (in this example, 21 distinct fields of view).

[0090] Each field of view illuminated by the transmitter can be considered as a pixel or spot in the corresponding 3D image that generates self-range data. Each transmitter channel can be distinct from each transmitter and non-overlapping with other transmitter channels; that is, there is a one-to-one mapping between the set of transmitters and the set of non-overlapping fields or viewpoints. Therefore, in Figure 4 In this example, the system can sample 21 dissimilar points in 3D space. Denser point sampling can be achieved by having a denser array of transmitters or by scanning the angular position of the transmitter beam over time so that a single transmitter can sample several points in space. As described above, scanning can be achieved by rotating the entire transmitter / sensor assembly.

[0091] Each sensor can be slightly offset from its neighbor, and similar to the transmitter described above, each sensor can see a different field of view of the scene in front of it. Furthermore, the field of view of each sensor is substantially consistent with the field of view of the corresponding transmitter channel, for example, overlapping with it and being the same size.

[0092] exist Figure 4 In this system, the distance between the corresponding transmitter-sensor channels is magnified relative to the distance to the object in the field of view. In reality, the distance to the object in the field of view is much greater than the distance between the corresponding transmitter-sensor channels, and therefore the path of light from the transmitter to the object is roughly parallel to the path of reflected light returning from the object to the sensor (i.e., it is almost "reflected back"). Therefore, there exists a range of distances in front of the system 400 where the fields of view of the individual sensors and transmitters overlap.

[0093] Because the field of view of the transmitter overlaps with the field of view of its corresponding sensor, each sensor channel ideally detects reflected illumination beams from its corresponding transmitter channel that ideally do not have crosstalk, i.e., it does not detect reflected light from other illumination beams. Therefore, each light sensor can correspond to a corresponding light source. For example, transmitter 403 emits illumination beam 405 into a circular field of view 407, and some of the illumination beam is reflected from object 408. Ideally, the reflected beam 411 is detected only by sensor 413. Therefore, transmitter 403 and sensor 413 share the same field of view, e.g., field of view 407, and form a transmitter-sensor pair. Similarly, transmitter 409 and sensor 415 form a transmitter-sensor pair, sharing field of view 410. Although the transmitter-sensor pair in... Figure 4 The transmitter is shown in the diagram as the same relative position in its corresponding array, but depending on the design of the optics used in the system, any transmitter can be paired with any sensor.

[0094] During ranging measurements, reflected light from different fields of view around a volume distributed around the LIDAR system is collected and processed by various sensors to obtain distance information for any object in each corresponding field of view. As described above, time-of-flight techniques can be used, in which a light emitter emits precisely timed pulses, and the reflection of the pulses is detected by the corresponding sensors after a certain elapsed time. The elapsed time between emission and detection, along with the known speed of light, is then used to calculate the distance to the reflecting surface. In some embodiments, additional information can be obtained from the sensors to determine properties of the reflecting surface other than distance. For example, the Doppler shift of the pulse can be measured by the sensors and used to calculate the relative velocity between the sensors and the reflecting surface. Pulse intensity can be used to estimate target reflectivity, and pulse shape can be used to determine whether the target is a hard or diffuse material.

[0095] In some embodiments, the LIDAR system may consist of a relatively large 2D array of transmitter and sensor channels and operate as a solid-state LIDAR, i.e., it can acquire frames of range data without requiring scanning the orientation of the transmitter and / or sensors. In other embodiments, the transmitter and sensors may scan, for example, rotate about an axis, to ensure that the field of view of the set of transmitters and sensors samples the complete 360-degree area of ​​the surrounding volume (or a useful portion of the 360-degree area). For example, range data collected from the scanning system within a predefined time period may then be post-processed into one or more data frames, which may then be further processed into one or more depth images or 3D point clouds. The depth images and / or 3D point clouds may be further processed into map tiles for use in 3D mapping and navigation applications.

[0096] C. Multiple photodetectors in each optical sensor Figure 5Various levels of a sensor array and associated electronics according to an embodiment of the invention are shown. Array 510 shows light sensors 515, each corresponding to a different pixel. Array 510 can be an interleaved array. In this particular example, array 510 is an 18x4 light sensor. Array 510 can be used to achieve high resolution (e.g., 72x1024) because the implementation is suitable for scanning.

[0097] Array 520 shows an enlarged view of a portion of array 510. As can be seen, each light sensor 515 is composed of multiple light detectors 525. Signals from the light detectors of the pixels collectively contribute to the measurement of the pixel.

[0098] In some embodiments, each pixel has a large number of single-photon avalanche diode (SPAD) units, which increases the dynamic range of the pixel itself. Each SPAD may have analog front-end circuitry for biasing, quenching, and recharging. SPADs are typically biased at a bias voltage higher than their breakdown voltage. Appropriate circuitry senses the leading edge of the avalanche current, generates a standard output pulse synchronized with avalanche accumulation, quenches the avalanche by reducing the bias down to below the breakdown voltage, and restores the photodiode to its operating level.

[0099] SPADs can be positioned to maximize the fill factor in their local areas, or microlens arrays can be used, allowing for high optical fill factors at the pixel level. Therefore, imager pixels can contain SPAD arrays to increase the efficiency of the pixel detector. Diffusers can be used to diffuse rays passing through the aperture and collimated by microlenses. Can diffusers are used to diffuse collimated rays in a manner that all SPADs belonging to the same pixel receive some radiation.

[0100] Figure 5 A specific photodetector 530 (e.g., a SPAD) is also shown for detecting photon 532. In response to detection, photodetector 530 generates an avalanche current 534 of charge carriers (electrons or holes). Threshold circuitry 540 regulates the avalanche current 534 by comparing it to a threshold. When a photon is detected and photodetector 530 is operating normally, the avalanche current 534 rises above the comparator threshold, and threshold circuitry 540 generates a time-accurate binary signal 545 indicating the precise timing of the SPAD current avalanche, which is also an accurate measurement of photon arrival. The correlation between current avalanche and photon arrival can occur with nanosecond resolution, thus providing high timing resolution. The rising edge of binary signal 545 can be latched by pixel counter 550.

[0101] Binary signal 545, avalanche current 534, and pixel counter 550 are examples of data values ​​that can be provided by a light sensor including one or more SPADs. The data values ​​can be determined from corresponding signals from each of a plurality of light detectors. Each of the corresponding signals can be compared with a threshold to determine whether the corresponding light detector is triggered. Avalanche current 534 is an example of an analog signal, and therefore the corresponding signal can be an analog signal.

[0102] Pixel counter 550 can use binary signal 545 to count the number of photodetectors for a given pixel that have been triggered by one or more photons during a specific time interval (e.g., a time window of 1, 2, 3 nanoseconds, etc.) controlled by periodic signal 560. Pixel counter 550 can store counters for each of a plurality of time intervals for a given measurement. The value of the counter for each time interval can start at zero and increment based on binary signal 545 indicating that a photon has been detected. The counter can increment when any photodetector of the pixel provides such a signal.

[0103] The periodic signal 560 can be generated by a phase-locked loop (PLL), a delay-locked loop (DLL), or any other method of generating a clock signal. The coordination of the periodic signal 560 and the pixel counter 550 can act as a time-to-digital converter (TDC), a device for identifying events and providing a digital representation of when they occur. For example, the TDC can output the arrival time of each detected photon or optical pulse. The measured time can be the elapsed time between two events (e.g., the start time and the detected photon or optical pulse) rather than an absolute time. The periodic signal 560 can be a relatively fast clock that switches between a set of memories including the pixel counter 550. Each register in the memory can correspond to a histogram interval, and the clock can switch between them at sampling intervals. Thus, when the corresponding signal is greater than a threshold, a binary value indicating triggering can be sent to the histogram circuitry. The histogram circuitry can aggregate binary values ​​across multiple photodetectors to determine the number of photodetectors triggered during a specific time interval.

[0104] The time interval can be measured relative to the start signal, for example, in Figure 3The start time is 315. Therefore, the counter in the time interval immediately following the start signal can have a low value corresponding to a background signal, such as background light 330. The last time interval can correspond to the end of the detection time interval (also called excitation) of a given pulse train, which is described further in the next section. The number of cycles of the periodic signal 560 since the start time can serve as a timestamp when the rising edge of the avalanche current 534 indicates the detected photon. The timestamp corresponds to the time interval for a specific counter in the pixel counter 550. This type of operation differs from a simple analog-to-digital converter (ADC) that follows a photodiode (e.g., for an avalanche diode (APD)). Each of the counters in the time interval can correspond to a histogram, which is described in more detail below. Thus, while the APD is a linear amplifier with finite gain for the input optical signal, the SPAD is a trigger device that provides a yes / no binary output for a triggering event occurring within a time window.

[0105] D. Pulse train Ranging can also be achieved using pulse trains, which are defined as containing one or more pulses. Within a pulse train, the number of pulses, the pulse width, and the duration between pulses (collectively referred to as the pulse pattern) can be selected based on several factors, some of which include: 1- Maximum Laser Duty Cycle - The duty cycle is the fraction of time the laser is on. For pulsed lasers, this can be determined by the FWHM as explained above and the number of pulses emitted during a given period.

[0106] 2- Eye safety limits - These are determined by the maximum amount of radiation the device can emit without harming the eyes of a bystander who happens to be looking in the direction of the LIDAR system.

[0107] 3- Power Consumption - This is the power consumed by the transmitter to illuminate the scene.

[0108] For example, the interval between pulses in a pulse train can be about a single digit or tens of nanoseconds.

[0109] Multiple pulse trains can be emitted during a measurement time span. Each pulse train can correspond to a different time interval, for example, a subsequent pulse train may not be emitted until the time limit for detecting the reflected pulse of a previous pulse train expires.

[0110] For a given transmitter or laser device, the time between the emission of a pulse train determines the maximum detectable range. For example, if pulse train A is in time... The pulse train B is emitted at point B, and the pulse train B is in time. If launched from that location, then it must be... Subsequent detected reflected pulse trains are assigned to pulse train A, as they are more likely reflections from pulse train B. Therefore, the time between pulse trains and the speed of light define the maximum bound on the system's range, as given in the following equation:

[0111] The time between excitation (emission and detection of the pulse train) can be about 1 µs to allow the entire pulse train enough time to travel to a distant object approximately 150 meters away and then return.

[0112] III. Histogram signal from the photodetector One operating mode of a LIDAR system is Time-Correlated Single-Photon Counting (TCSPC), which is based on counting individual photons in a periodic signal. This technique is well-suited for low levels of periodic radiation, making it appropriate for LIDAR systems. This time-correlated counting can be affected by... Figure 5 The periodic signal 560 is controlled and can be used within a time interval, such as for... Figure 5 The discussion.

[0113] The frequency of a periodic signal specifies a time resolution within which data values ​​of the signal are measured. For example, a measurement can be obtained for each photodetector in each cycle of the periodic signal. In some embodiments, the measurement can be the number of photodetectors triggered during the cycle. The time period of the periodic signal corresponds to a time interval, where each cycle is a different time interval.

[0114] Figure 6 A histogram 600 is shown according to an embodiment of the invention. The horizontal axis corresponds to the time interval as measured relative to a start time 615. As described above, the start time 615 may correspond to the start time of the pulse train. Any offset between the rising edge of the first pulse of the pulse train and the start time of any one or both of the pulse train and the detection time interval may be considered, where the reception time will be used for time-of-flight measurement. The vertical axis corresponds to the number of SPADs triggered. In some embodiments, the vertical axis may correspond to the output of an ADC following an APD. For example, the APD may exhibit conventional saturation effects, such as a constant maximum signal rather than the dead-time-based effect of the SPAD. Some effects may occur for both SPADs and APDs; for example, pulse tailing on a highly tilted surface may occur for both SPADs and APDs.

[0115] The counters used for each time interval correspond to different bars in histogram 600. The counters in the earlier time intervals are relatively low and correspond to background noise 630. At some points, reflected pulses 620 are detected. These correspond to much larger counters and can exceed the threshold for distinguishing between the background and the detected pulse. The reflected pulses 620 (after digitization) are shown as corresponding to four time intervals, which may be caused by laser pulses of similar width, for example, 4 ns pulses when each time interval is 1 ns. However, as described in more detail below, the number of time intervals can vary, for example, based on the properties of a specific object in the incident angle of the laser pulse.

[0116] The time position corresponding to the time interval of the reflected pulse 620 can be used, for example, to determine the reception time relative to the start time 615. As described in more detail below, a matched filter can be used to identify the pulse pattern, thereby effectively increasing the signal-to-noise ratio and determining the reception time more accurately. In some embodiments, the accuracy of determining the reception time may be less than the time resolution of a single time interval. For example, for a 1 ns time interval, the resolution would correspond to approximately 15 cm. However, an accuracy of only a few centimeters may be required.

[0117] Therefore, the detected photons can cause a specific time interval of the histogram to increment based on its arrival time relative to, for example, a start signal indicated by start time 615. The start signal can be periodic, causing multiple pulse trains to be transmitted during the measurement. Each start signal can be synchronized to a laser pulse train, where multiple start signals cause multiple pulse trains to be transmitted within multiple detection intervals. Thus, a time interval (e.g., from 200 to 201 ns after the start signal) will occur for each detection interval. The histogram can be cumulatively counted, where the count for a specific time interval corresponds to the sum of the measured data values ​​that occurred throughout all of the multiple excitations in that specific time interval. When the detected photons are histogramted based on this technique, it results in a signal-to-noise ratio of the returned signal that is greater than the square root of the number of excitations made compared to a single pulse train.

[0118] Figure 7 The diagram illustrates the accumulation of histograms over multiple pulse trains for a selected pixel according to an embodiment of the present invention. Figure 7 Three detected pulse trains 710, 720, and 730 are shown. Each detected pulse train corresponds to a transmitted pulse train with the same pattern of two pulses separated by the same amount of time. Therefore, each detected pulse train has the same pulse pattern, as shown by two time intervals with distinct values. Counters for other time intervals are not shown for ease of illustration, but these other time intervals may have relatively low non-zero values.

[0119] In the first detected pulse train 710, the counters used for time intervals 712 and 714 are the same. This may be caused by the same number of photodetectors detecting photons during the two time intervals. Alternatively, in other embodiments, approximately the same number of photons are detected during the two time intervals. In other embodiments, more than one consecutive time interval may have consecutive non-zero values; however, for ease of illustration, individual non-zero time intervals have been shown.

[0120] Time intervals 712 and 714 occur 458 ns and 478 ns after the start time 715, respectively. The counters displayed for other detected bursts occur within the same time interval relative to their respective start times. In this example, the start time 715 is identified as occurring at time 0, but the actual time is arbitrary. The first detection interval for the first detected burst can be 1 µs. Therefore, the number of time intervals measured from the start time 715 can be 1,000. This first detection interval then ends, allowing for the transmission and detection of new bursts. The start and end of different time intervals can be controlled by a clock signal, which can be part of the circuitry acting as a time-to-digital converter (TDC), for example in… Figure 5 As described in the text.

[0121] For the second detected pulse train 720, the start time 725 is 1 µs, for example, at which time the second pulse train can be emitted. Such a separate detection interval can occur so that any pulse transmitted at the beginning of the first detection interval will have already been detected, and therefore will not cause confusion between pulses detected in the second time interval. For example, if there is no additional time between excitations, the circuit might confuse a retroreflective aperture sign at 200 m with an object at 50 m that reflects far less (assuming an excitation period of approximately 1 µs). The two detection time intervals for pulse trains 710 and 720 can be of the same length and have the same relationship to their respective start times. Time intervals 722 and 724 occur at the same relative times of 458 ns and 478 ns as time intervals 712 and 714. Therefore, when the accumulation step occurs, a corresponding counter can be added. For example, the counter values ​​at time intervals 712 and 722 can be added together.

[0122] For the third detected pulse train 730, the start time 735 is 2 µs, for example, during which the third pulse train can be emitted. Time intervals 732 and 734 also occur at 458 ns and 478 ns relative to their corresponding start times 735. Even if the emitted pulses have the same power, the counters in different time intervals may have different values, for example, due to the random nature of the scattering process of the light pulse leaving the object.

[0123] Histogram 740 shows the accumulation of counters from three detected bursts at time intervals 742 and 744, which also correspond to 458 ns and 478 ns. Histogram 740 may have a smaller number of time intervals measured during the respective detection intervals, for example, because time intervals at the beginning or end are discarded, or have values ​​below a threshold. In some embodiments, depending on the pattern of the bursts, approximately 10 to 30 time intervals may have significant values.

[0124] For example, the number of pulse trains emitted during a measurement to create a single histogram can be from about 1 to 40 (e.g., 24), but can be much higher, such as 50, 100, or 500. Once the measurement is complete, the counter used for the histogram can be reset, and a new set of pulse trains can be emitted to perform a new measurement. In various embodiments and depending on the number of detection intervals in the corresponding duration, measurements can be performed every 25, 50, 100, or 500 µs. In some embodiments, the measurement intervals can overlap, for example, so a given histogram corresponds to a specific sliding window of the pulse train. In such instances, memory can exist to store multiple histograms, each corresponding to a different time window. Any weights applied to the detected pulses can be the same for each histogram, or such weights can be controlled independently.

[0125] IV. Histogram Data Path Figure 8 A circuit according to some embodiments is shown for receiving photons and generating a set of signals stored in a memory representing a histogram. As relative to... Figure 5 As described above, the optical sensor array can be used to receive reflected pulses and background photons from ambient light in the optical measurement system. A single optical sensor 802 may contain multiple photodetectors. Each photodetector may be implemented using a SPAD or other photosensitive sensor, and the photodetectors may be arranged in a grid pattern for the optical sensor 802, such as... Figure 8 As explained in the documentation, filters can be used in each of the optical sensors to block light outside the range of the light source centered around the LIDAR system from being received by the optical sensor. However, even with this filter, some ambient light at or near the emitted wavelength of the light source can pass through the filter. This can cause photons from the ambient light, as well as photons emitted from the LIDAR light source, to be received by the optical sensor.

[0126] Each photodetector in the optical sensor 802 may include analog front-end circuitry for generating an output signal indicating when a photon is received by the photodetector. For example, see reference [link to reference]. Figure 5 The avalanche current 534 from the SPAD can trigger the threshold circuit 540 to generate an output binary signal 545. Returning to... Figure 8Each photodetector in the optical sensor 802 can generate its own signal corresponding to the received photon. Therefore, the optical sensor 802 can generate a set of signals 816 corresponding to the number of photodetectors in the optical sensor 802. The optical sensor 802 can also be referred to as a "pixel" or "pixel sensor" because it can correspond to a single pixel of information when displayed or analyzed in later stages of the optical measurement system. When a signal is generated in response to a received photon (e.g., a transition from logic "0" to logic "1"), this can be called a "positive" signal.

[0127] The Arithmetic Logic Unit (ALU) 804 can be used to implement... Figure 5 The functionality of the pixel counter 550. Specifically, the ALU 804 can receive a set of signals 816 from individual photodetectors of the light sensor 802 and aggregate the number of these signals, each indicating the detection of photons. The ALU 804 may include combined digital electronic circuitry that performs arithmetic and / or other bitwise operations on the set of signals 816. For example, the ALU 804 can receive each of the signals 816 as a binary signal (i.e., "0" or "1") as an input or operand of the ALU 804. By aggregating or adding the inputs together, the ALU 804 can count the number of positive signals in the set of signals 816, which indicate that photons have been received within a specific time interval. For example, by adding each of the signals indicating a "1" signal level, the output of the ALU 804 can indicate the number of signals in the set of signals 816 associated with photodetectors that have received photons during the time interval.

[0128] The ALU 804 is specifically designed to receive at least the number of inputs corresponding to the number of photodetectors in the optical sensor 802. Figure 8 In this example, the ALU 804 can be configured to receive 32 parallel inputs of a single bit width. Internally, the ALU 804 can be implemented using digital logic gates to form ripple carry adders, carry-lookahead adders, carry-hold adders, and / or any other type of adder that can aggregate a relatively large number of inputs with low propagation time. The output of the ALU 804 can be referred to as the "total signal count" and can be represented as an n-bit binary number output from the ALU 804 or from a stage of the ALU 804.

[0129] As described above, the output of ALU 804 characterizes the total number of photons received by optical sensor 802 during a specific time interval. Each time ALU 804 completes an aggregation operation, the total signal count can be added to the corresponding memory location in memory 806, representing histogram 818. In some embodiments, memory 806 may be implemented using SRAM. Thus, during a process of multiple excitations (where each excitation comprises a burst of pulses), the total signal count from ALU 804 can be aggregated with existing values ​​in the corresponding memory locations in memory 806. A single measurement may include multiple excitations that fill memory 806 to generate a histogram 818 that can be used to detect values ​​in a time interval for reflected signals, background noise, peak values, and / or other signals of interest.

[0130] ALU 804 can also perform a second aggregation operation, adding the total signal count to an existing value in the memory location of memory 806. (See also: Review) Figure 7 With each excitation, a new total signal count can be added to the existing value in the corresponding time interval of memory 806. In this way, histogram 818 can be progressively constructed in memory 806 over several excitations. When the total signal count is generated by ALU 814, current value 820 for the corresponding memory location of the time interval can be retrieved from memory 806. Current value 820 can be provided as an operand to ALU 804, which can be combined with the total signal count from the set of signals 816. In some embodiments, ALU 804 can consist of a first stage and a second stage, wherein the first stage calculates the total signal count from optical sensor 802, and the second stage combines the total signal count with the current value 820 from the memory location of the time interval in memory 806. In some embodiments, the aggregation of the set of signals 816 and the aggregation of the total signal count and current value 820 can be performed as a single operation. Therefore, even though these two operations can be functionally described as separate “aggregations,” they can actually be performed together using a combination of parallel and sequential circuitry in ALU 804.

[0131] As mentioned above, in contrast to Figure 5 As described, ALU 804 can receive a periodic signal 560 that triggers an aggregation operation. The periodic signal 560 can be generated using any of the techniques described above. The periodic signal 560 can define the length of each time interval. In some embodiments, the periodic signal 560 and the corresponding time interval can be relative to, for example... Figure 3The start signal described herein is used for measurement. Each cycle of the periodic signal 560 causes an aggregation operation to be performed in ALU 804 and causes the memory address of memory 860 to increment into the next time interval. For example, the rising edge of the periodic signal 560 causes ALU 804 to generate a result that aggregates the total signal count and the current value 820. A corresponding periodic signal 808 can also be sent to the memory interface circuit, which increments the address to the memory location of the current time interval, causing each cycle to also move into the next time interval in memory 806.

[0132] Clock circuit 810 can be used to generate periodic signal 560 based on an input that defines the excitation and measurement of the optical measurement system. For example, excitation input 814 may correspond to... Figure 3 The start signal is described in the diagram. Excitation input 814 can reset the address of memory 806 to the start memory location corresponding to a first time interval of histogram 818. Excitation input 814 can also cause clock circuit 810 to start generating periodic signal 560 for ALU 804 and / or periodic signal 808 to increment the address of memory 806. Additionally, clock circuit 810 can receive measurement input 812 defining the start / end of a measurement. Measurements may include multiple excitations that incrementally construct histogram 818. Measurement signal 812 can be used to reset the values ​​in memory 806, allowing the histogram to restart for each new measurement.

[0133] Memory 806 may contain multiple registers for accumulating photon counts from the photodetector. By accumulating photon counts in corresponding registers corresponding to time intervals, the registers in memory 806 can store photon counts based on the arrival time of the photons. For example, photons arriving in a first time interval may be stored in a first register in memory 806, photons arriving in a second time interval may be stored in a second register in memory 806, and so on. Each “excitation” may involve one traversal of each of the registers in memory 806 corresponding to the time interval used for the photodetector. Excitation signal 814 may be referred to as an “enable” signal acting on the multiple registers in memory 806, because excitation signal 814 enables the registers in memory 806 to store results from ALU 804 during the current excitation.

[0134] A periodic signal 560 may be generated such that it is configured to capture the set of signals 816 when the set of signals 816 is asynchronously provided from the optical sensor 802. For example, a threshold circuit 540 may be configured to keep the output signal high for a predetermined time interval. The periodic signal 560 may be timed such that it has a period less than or equal to the hold time of the threshold circuit 540. Alternatively, the period of the periodic signal 560 may be a percentage of the hold time of the threshold circuit 540, such as 90%, 80%, 75%, 70%, 50%, 110%, 120%, 125%, 150%, 200%, etc. Some embodiments may use... Figure 5 The rising edge detection circuit described herein converts the asynchronous signal from the photodetector into a single clock gating, which uses the same clock running the ALU 804. This ensures that photons are not counted more than once. Other embodiments may alternatively oversample the asynchronous pulse from the photodetector or use strong rising edge detection.

[0135] Figure 9 The diagram illustrates the timing associated with different excitations in a measurement according to some embodiments. The vertical axis represents the total number of photodetector signals measured for a single optical sensor. For example, the vertical axis may represent the total photon count. The horizontal axis represents time. When a new optical measurement begins as indicated by measurement signal 812, the first excitation may begin by receiving excitation signal 814-1 as a start signal. Periodic signal 506 illustrates how each timing of ALU 804 corresponds to a single time interval and a corresponding memory location in memory 806.

[0136] Using each subsequent excitation, the histogram can be constructed in memory 806, such as Figure 9 As explained in [the document]. Whenever a trigger input 814 is received, the addressing of memory 806 can be reset so that a new total signal count can be added to the existing signal count. Figure 9 In some examples, there is a non-zero time interval separating each excitation. For instance, an excitation beginning with excitation signal 814-1 ends, and a non-zero time interval elapses before the start of an excitation defined by excitation signal 814-2. Alternatively, some embodiments may not have a delay between subsequent excitations such that the periodic signal 506 continuously times throughout the measurement. Subsequent excitation signals 814-2, 814-3 may then define both the end of a previous excitation and the start of a subsequent excitation.

[0137] In some embodiments, the timing of the measurement signal 812, the excitation signal 814, and the periodic signal 506 of the timing ALU 804 can all be coordinated and generated relative to each other. Therefore, the timing of the ALU can be triggered by the start signal of each excitation and depends on the start signal of each excitation. Additionally, the period of the periodic signal 506 can define the length of each time interval associated with each memory location in the histogram.

[0138] Figure 8 The data path described herein is primarily configured to construct a histogram 818 over several excitations. However, in some embodiments, the histogram may be filled only during excitation. Any photons received between excitations will not be aggregated by the ALU 804 in the time interval and will not be stored in the memory 806. Furthermore, the contents of the memory 806 can be reset after each measurement is completed and before the start of a subsequent measurement. Therefore, photons received before and / or after the current measurement can be readily obtained without being saved. Moreover, the total count of all photons received in the histogram is not readily available in the histogram data path, and when the histogram is disabled, the total count of all photons received during the measurement period is unavailable if there is a non-zero interval between excitations. In order to record received photons in a continuous manner dependent on excitation / measurement timing, in addition to Figure 8 A second parallel data path may be used in addition to the histogram data path described herein.

[0139] V. Identify earlier reflections from the system casing. The examples above illustrate a single pulse or multiple pulses as part of a pulse train transmitted from a light source, reflected from an object in the surrounding environment, and subsequently detected by a light sensor. After several repeated excitations during the measurement, these reflections are recorded in a histogram memory and form "peaks" in the histogram. The positions of these peaks can then be used to determine the distance between the optical measurement system and an object in the surrounding environment. However, these examples are not intended to be limiting. In addition to the expected peaks caused by the pulse train reflected from the object of interest in the surrounding environment, the histogram memory may also contain other peaks caused by more immediate, unexpected reflections of the pulse train, as well as peaks that may not necessarily be generated by the pulse train at all. For example, additional peaks may correspond to external light sources, sunlight, reflections, or other ambient light phenomena in the surrounding environment. In these cases, instead of detecting only a single peak, the optical measurement system can detect multiple peaks that may therefore exist in the histogram memory. In addition to extraneous peaks caused by external light sources, initial peaks may be generated by the pulse train reflected from the internal casing of the optical measurement system.

[0140] Figure 10This diagram illustrates a histogram memory containing a peak value 1002 caused by reflections from the housing of the optical measurement system. When a light source generates one or more pulses for an emitted pulse train, these pulses can pass through a window or other transmissive material within the housing of the optical measurement system. However, some of the light emitted by the light source can be reflected back to the photosensor from the window of the housing instead of passing completely through the window. Additionally, some of the light emitted by the light source can be reflected from the inner surface of the housing itself. For example, some stray light can be reflected from portions of the housing near the transmissive window or from other internal materials of the optical measurement system. Just as reflected pulses from an object of interest in the surrounding environment are detected and stored, a portion of the light reflected from the housing of the optical measurement system can be received by the photosensor and stored in the histogram memory.

[0141] exist Figure 10 In this process, the peak value 1002 caused by the early reflection of the emitted pulse train from the housing of the optical measurement system can be stored in the initial set of registers in the histogram memory. Because the reflection occurs shortly after the pulse train is emitted from the light source of the optical measurement system, the photon count from these early reflections can be stored in the first register of the histogram memory for measurement. For example, Figure 10 The peak 1002 corresponding to an earlier reflection from the system housing is stored in the first nine registers of the histogram memory. However, this example is not intended to be limiting. The number and location of the time intervals representing reflections from the system housing can depend on the intensity of the light pulse, the distance between the housing and the light source, the distance between the housing and the photodetector, and / or other characteristics of the physical design of the optical measurement system. The method for determining the location of the peak 1002 caused by reflections from the system housing is discussed in more detail below.

[0142] While receiving the remaining photon counts for optical measurements, additional peaks caused by reflections of photons from the object of interest in the surrounding environment can also be detected. For example, after reflection from an external object within the range of the optical measurement system, Figure 10Peak 1004 may be received later in the measurement. Typically, these objects of interest in the surrounding environment will be far enough away from the optical measurement system that the resulting peak 1004 can be easily distinguished from peak 1002 reflected from the system housing. Some embodiments may ignore or remove peak 1002 from the histogram memory and / or from any calculations performed on the histogram memory to calculate the distance to the external object corresponding to peak 1004. For example, some embodiments may mask the initial set of histogram registers that may contain large peaks 1002 reflected from the housing. Some embodiments may insert a delay between when the pulse train is emitted by the light source and when the light sensor begins to accumulate photon counts in the histogram memory to avoid recording peak 1002 reflected from the housing. Because peak 1002 occurs so early in the measurement compared to when peak 1004 is received, peak 1002 can generally be ignored when using peak 1004 to perform distance calculations.

[0143] However, some embodiments may receive the peak 1002 caused by housing reflection and store it in a histogram memory, and then continue to use the peak to improve the short-range performance of the optical measurement system. These improvements may include calibrating the optical measurement system, characterizing the opacity of the optical measurement system's window, detecting blockages in the optical measurement system, improving the short-range performance of the optical measurement system, and so on. Each of these various improvements that can be achieved by first identifying the peak 1002 caused by housing reflection is described in more detail below.

[0144] Figure 11 A flowchart illustrating a method for detecting peaks caused by earlier reflections from a system housing using an optical measurement system. This method can be used to transmit / receive pulses and identify specific locations in a histogram memory corresponding to peaks reflected from the housing. The following sections of this disclosure describe different ways in which the identified peaks can be used to improve the performance of the optical measurement system. The following method describes the process for a single optical sensor in the optical measurement system. However, as described above, the optical measurement system may contain a number of optical sensors, and this method can be performed for each optical sensor in the optical measurement system.

[0145] At step 1102, the method may include transmitting one or more pulse trains from a light source within one or more first time intervals as part of an optical measurement. Each of the one or more first time intervals may represent an "excitation" repeated multiple times in the measurement. Each of the first time intervals may include one or more pulse trains encoded and transmitted by the light source such that the pulse trains can be considered as being reflected from objects in the surrounding environment. Each time interval may be subdivided into multiple time intervals such that each time interval represents an interval in a histogram of photon counts received during the optical measurement. (The above is in contrast to...) Figure 9Describe instances of how a single measurement can contain multiple excitations subdivided into time intervals for clustered photon counting.

[0146] At step 1104, the method may further include using a light sensor to detect photons from one or more pulse trains. As described in detail above, the light sensor may include multiple photodetectors, such as multiple SPADs. The light sensor may receive reflected light as well as ambient background noise received from the surrounding environment. The reflected light received by the light sensor may include light reflected from objects of interest in the surrounding environment. For example, these objects of interest may be at least 30 cm away from the light sensor and may represent surrounding vehicles, buildings, pedestrians, and / or any other objects that may be encountered near the optical measurement system during use. These objects of interest may be distinguished from objects that are part of the optical measurement system itself, such as a housing. The reflected light received by the light sensor may also include light reflected from the housing or other parts of the optical measurement system. These reflections may include reflections primarily directly from windows or the housing of the optical measurement system, as well as secondary reflections reflected near the interior of the optical measurement system. The light sensor may also be coupled to threshold detection circuitry and / or to arithmetic logic circuitry for accumulating photon counts. This combination may be referred to above as a “pixel.” Figure 5 This section describes an example of how photon counts can be received from a light sensor and how a threshold circuit and a pixel counter (e.g., an arithmetic logic circuit) can be used to count the photon counts.

[0147] At step 1106, the method may further include accumulating photon counts from the optical sensor into multiple registers to represent a histogram of photon counts received during the current measurement. These photon counts may be accumulated in multiple registers within a memory block corresponding to the optical sensor. The multiple registers may be implemented using registers in the SRAM of the histogram data path, as described above. Figures 8 to 9 As described herein. Each time interval can be subdivided into multiple first time intervals for the histogram. The corresponding time interval in each of one or more time intervals can be accumulated in a single register among multiple registers. Each time interval can represent an excitation, and one or more time intervals together can represent a measurement for the optical measurement system. Each time interval can be defined by a start signal or excitation signal that resets the memory representing the histogram back to the first register in the histogram. Received photons reflected from the housing of the optical measurement system can be stored in the histogram memory in the same manner as other received photons reflected from surrounding objects outside the optical measurement system.

[0148] At step 1108, the method may additionally include identifying a peak in a histogram representing photons reflected from the system housing. Depending on the specific embodiment, detecting this peak may be performed using several different techniques. In some embodiments, the peak may be identified by iterating through a histogram memory to identify the first peak to occur in time. The system may sequentially access registers in a plurality of registers, starting at the beginning of the optical measurement, to identify the first peak to occur in time. The center of the peak may be identified by identifying values ​​in a histogram that have smaller values ​​in the time intervals on either side. Figure 10 In some examples, registers representing the fourth and fifth time intervals can be identified as peaks because the time interval values ​​in registers flanking the fourth and fifth registers have significantly smaller values ​​stored therein. In some embodiments, the initial peak can be identified by identifying registers among a plurality of registers during the highest number of photon counts. These embodiments may assume that the initial peak may have a higher intensity than other peaks. Some embodiments may also identify the initial peak by identifying registers among a plurality of registers corresponding to the distance between the light source and the housing of the optical measurement system. For example, when this distance is known, each distance can be used to identify the register capable of receiving the peak reflected from the housing. Some embodiments may identify the initial peak by identifying a predetermined number of registers among a plurality of registers that appear first. For example, based on initial calibration and a known distance between the light sensor and the housing, some embodiments may identify the first 15 time intervals as storing the initial peak.

[0149] Identifying peaks in a histogram may involve locating a time interval window that occurs near the center of the peak. This process may involve expanding outwards from the peak location to determine the range of registers in histogram memory that encompass the entire peak. For example... Figure 10 As explained, photons caused by reflections from the system enclosure can be received in multiple time intervals rather than isolated to a single time interval. Therefore, some embodiments may identify surrounding time intervals, which should also be specified to represent the peak value of reflections from the system enclosure. For example, some embodiments may identify a predetermined number of time intervals near the maximum value of the peak. For example, a predetermined number of time intervals, such as 3 time intervals, 5 time intervals, 9 time intervals, 15 time intervals, 17 time intervals, etc., may identify the maximum value and center around the maximum value to represent the entire peak. Some embodiments may identify surrounding time intervals with values ​​within a percentage of the maximum value in the peak register. This can result in a variable number of time intervals that can be used to represent the peak depending on the width of the peak. For example, the time intervals around the maximum value may be included in the peak when their values ​​are within 25% of the maximum value.

[0150] Instead of detecting the variable peak position for each measurement, some embodiments may assume a known position of the optical measurement housing and specify a particular window of a register, which may contain values ​​from earlier reflections exiting the system housing. Figure 10 In this example, the system can designate registers corresponding to the second through sixth time intervals as registers to store the peak values ​​of reflections from the system housing. Physical measurements can be performed to determine the distance between the light source and the housing, and the distance between the housing and the light sensor. This distance can then be combined with the speed of light to determine where reflections from the system housing are likely to appear in the histogram memory.

[0151] It should be understood that, according to various embodiments, Figure 11 The specific steps described herein provide a specific method for identifying peaks representing reflections emanating from the system enclosure. Other sequences of these steps may also be performed according to alternative embodiments. For example, alternative embodiments of the invention may perform the steps outlined above in a different order. Furthermore, Figure 11 The individual steps described may contain multiple sub-steps that can be performed in various sequences suitable for the individual steps. Furthermore, additional steps may be added or removed depending on the specific application. Those skilled in the art should recognize the many variations, modifications, and alternatives.

[0152] VI. Use housing reflection to improve measurement After identifying the location of the pulse representing the reflection emanating from the system housing, the location and / or peak of the reflection can be used to improve or characterize the optical measurement system in several different ways. This may include known distances in calibrating additional distance measurements performed by the optical measurement system, characterizing the transparency of windows on the optical measurement system, detecting obstructions or other obstacles that block the field of view of the optical measurement system, and / or other similar improvements.

[0153] A. Calibration using shell reflection Figure 12The contents of a histogram memory, according to some embodiments, can be used to calibrate distance measurements for optical measurement systems. As described above, the histogram may contain a peak 1002 corresponding to reflections from the housing of the optical measurement system. The histogram may also contain a peak 1004 caused by photon reflections from an object of interest in the surrounding environment. Typically, the center position of peak 1004 can be identified and used to calculate the distance between the optical measurement system and the object from which it reflects peak 1004. This distance calculation can use the speed of light and a known time interval for each of the time intervals represented by the histogram register. For example, for a system operating at 500 MHz, the resulting time interval may represent approximately 2 ns of the received photon count. The distance of peak 1004 can be calculated by multiplying the number of time intervals (e.g., 32) by the time of each time interval (e.g., 2 ns) by the speed of light.

[0154] However, distance calculations may be inaccurate if any of the assumed values ​​above are imprecise or vary between different optical measurement systems. For example, the actual time represented by each time interval may be less accurate than that predicted based on an assumed clock period. Therefore, some systems are suitable for performing a self-calibration process that can be used to accurately determine the different constants used in distance calculations.

[0155] Some embodiments may use the position of the peak 1002, representing the reflection from the housing, to calibrate other distance calculations of the optical measurement system. The distance between the light source and the housing of the optical measurement system can be precisely known based on the manufacturing process of the optical measurement system. Similarly, the distance between the system's housing and the light sensor can also be precisely known. These distances can be used in conjunction with the peak 1002 to determine precise values ​​of constants used in the distance calculations, such as the time represented by each time interval and / or any offset in the system. These constants may be part of a transformation function that receives the peak and provides accurate distance measurements. Such a transformation may be a constant (e.g., uniformly delayed) linear transformation (e.g., a later time interval offset that is more / less than the earlier time interval by a amount proportional to the number of time intervals) or a nonlinear transformation.

[0156] exist Figure 12In this example, any of the techniques described above can be used to determine the location of peak 1002. The location of peak 1002 can be used to determine the number of time intervals 1202 between the start of the measurement and the center of the peak from the housing. The known distance between the light source, housing, and light sensor described above can be divided by the speed of light to determine the total time between when light is emitted from the light source and when peak 1002 appears in the histogram memory. This value can then be divided by the number of time intervals 1202 to calculate the precise amount of time represented by each time interval. It should be noted that this procedure assumes that the histogram memory begins accumulating photon counts in the register when light is emitted from the light source. If this is not the case, then any delay between the emission of light from the light source and the start of the accumulation period through the histogram memory can be subtracted from the time estimate.

[0157] The precise value of the constant, such as the time represented by each time interval, can then be used to calculate the distance to other peaks in the measurement. Figure 12 In the example, the distance between the optical measurement system and the object represented by peak 1004 can be calculated as described above, but instead of using the time value assumed from the clock cycle, the system can actually use the calibration value calculated using peak 1002 to calculate the distance to the object.

[0158] Using calibration values ​​derived from peak values ​​reflected from the housing of the optical measurement system allows for the generation of more accurate distance measurements. These techniques overcome problems caused by process variations over time and sensor drift. These techniques can also be used to detect movement of the system housing relative to the light source and / or the optical sensor. For example, if the position of peak 1002 changes during the lifespan of the optical measurement system, this can indicate movement of the housing relative to the rest of the system.

[0159] B. Characterizing window transparency Besides calibrating values ​​to be used for distance calculations, identifying the location of reflections from the system housing can also be used to characterize various aspects of the optical measurement system during operation. Generally, the peak value reflected from the system housing can be larger than the peak value received from the object of interest in the surrounding environment. The magnitude of this initial peak value may be due to the proximity of the housing to the optical sensor. Because the optical sensor is so close to the reflection, the intensity of the reflected light can be relatively large. However, assuming the magnitude of the reflection peak value is within the saturation limit of the histogram memory, certain measurements can be performed based on the magnitude of the peak value reflected from the housing to characterize a portion of the housing itself.

[0160] Figure 13AThis describes a portion of the histogram memory corresponding to an initial peak value 1302 corresponding to reflections from the system housing, according to some embodiments. Peak value 1302 may have an initial magnitude 1308. This magnitude characterizes the transparency of a window on the housing of the optical measurement system. As described above, the housing may include a window through which light can be transmitted from a light source, and reflected light can be received by a light sensor through the window. Assuming the rest of the housing is substantially opaque, the magnitude 1308 of peak value 1302 can be used to characterize the transparency of the window itself. For example, baseline transparency can be characterized during the initial portion of the optical measurement system's lifespan by recording the average magnitude 1308 of housing reflections, a value that can be stored over time and used to detect changes in window transparency.

[0161] Many practical applications of optical measurement systems can be found in open environments, where the system may be exposed to contamination, rain, snow, and / or other environmental elements that could damage, obscure, and / or otherwise affect the transparency of the windows on the system's housing. For example, when an optical measurement system is installed in an automotive application, it may be positioned on the roof of the vehicle. The system may also be positioned in an area near the vehicle's perimeter, such as on the bumper or side. In these locations, the optical measurement system may be subjected to weather conditions, rocks and debris, exhaust fumes or fog, and / or other effects that could affect the window's transparency. For example, rocks on the road may strike the optical measurement system, scratching or otherwise damaging the window. Humid environments can cause moisture to temporarily accumulate on the window. Rain or snow can accumulate on the exterior of the window. Mud, contamination, or grime from the driveway can accumulate on the exterior of the window. Many other adverse conditions in the automotive environment can also affect the windows of the optical measurement system.

[0162] Each of these different environmental influences can cause a change in the window's transparency. For example, when dirt or scratches accumulate on the window, less light can transmit through it. Therefore, more light can be reflected back from the window to the light sensor. Thus, considering that the window is part of the housing of the optical measurement system, the technique described above regarding monitoring the magnitude of the peak reflected from the housing of the optical measurement system over time can also be used to track differences in the window's transparency over time. It can be assumed that the transparency of the rest of the housing remains constant (e.g., the housing is completely opaque). Therefore, any change in the magnitude of the peak reflected from the housing can be attributed to a change in the window's own transparency.

[0163] Figure 13B This describes the variation in the magnitude of peak value 1304 reflected from the casing according to some embodiments. Peak value 1304 can be obtained from recording... Figure 13AThe peak value 1302 is recorded using the same optical measurement system, but peak value 1304 can be recorded later in the lifespan of the optical measurement system. For example, peak value 1304 might be recorded several months after use in an automotive application. When the magnitude 1310 of peak value 1304 is compared with the magnitude 1308 of peak value 1302, Figure 13B This explains how the peak value of 1304 increases over time. This increase can be attributed to changes in the transmissive properties of the window on the housing of the optical measurement system.

[0164] As described above, when the window of the housing is clean, the magnitude 1308 of the initial peak 1302 can be recorded as a baseline measurement. This initial value can be stored over time and used as a baseline for comparison with future peak magnitudes to track the window's transparency over time. For example, some embodiments may track the difference 1306 between the initial magnitude 1308 and the current magnitude 1310 of the peaks 1302, 1304 over time. As the difference 1306 increases, various outputs can be provided by an optical measurement system. For example, some embodiments may provide outputs that trigger warnings or messages to the user or the vehicle system indicating that the window is dirty (“…”). Please clean the windows on your LiDAR system. Some embodiments can automatically trigger systems on the vehicle to clean the optical measurement system, such as systems that spray cleaning agents onto the optical measurement system.

[0165] Figure 14 This describes a portion of an array of light sensors 1402 that can be partially obscured by contamination of a window, according to some embodiments. As described above, these methods for detecting reflected pulses from the housing of an optical measurement system can be performed for each light sensor in the array of light sensors 1402. Thus, each individual light sensor can generate a baseline magnitude of the reflected peak, and a current magnitude of the reflected peak. Each individual light sensor can then report the percentage or degree to which the corresponding portion of the window is obscured. The transparency of each light sensor can be characterized relative to one of the light sensors or relative to a baseline level of transparency. In this example, a portion 1404 of the window may be covered with contaminants, such as mud. Individual light sensors transmitting / receiving light through portion 1404 of the window can report an increase in the magnitude of the reflected pulse, while light sensors outside portion 1404 of the window can continuously report a steady magnitude of the reflected pulse. This can be used to generate a map or characteristic of the window. For example, Figure 14 Images can be generated and provided to the user interface to describe the current state of the window. Messages can be generated that characterize the window as dirty, partially obscured, or completely blocked.

[0166] In some embodiments, the optical measurement system may shut off the photosensors behind a dirty or clogged portion 1404 of the window, or, for a rotating system, shut them off when the photosensors are behind portion 1404. Due to the intensity of the reflected pulses, the histogram data path of these photosensors may saturate, causing the magnitude of the pulses reflected from the window to exceed the bit limit of the histogram data path. Reflections from this portion 1404 of the window may also be scattered by obstructions and interfere with photons received by other nearby photosensors. Because measurements can be skewed by saturation, scattering, or other effects, the system may perform corrective measurements, such as preventing these photosensors from accumulating photon counts.

[0167] C. Detection window blocking Besides detecting dirty or partially obstructed windows, the initial peak detected can also be used to determine if the optical measurement system is blocked by a foreign object. As described above, many operating environments, such as automotive environments, provide situations where the optical measurement system can be subjected to adverse conditions. Some of these environments can cause the optical measurement system to become completely blocked by its surroundings. For example, a plastic bag can be blown onto the optical measurement system. A person can place their hand in front of the optical measurement system. The optical measurement system can be intentionally disabled by tape or other objects placed in front of it. In any of these environments, it may be useful to detect when such blockage occurs and where it may be located.

[0168] Figure 15A This describes an initial peak 1502 detected by an optical measurement system relative to a blocking threshold 1506, according to some embodiments. The initial peak 1502 may be a result of reflection from the housing of the optical measurement system as described above. Based on this baseline measurement of the magnitude of peak 1502, a blocking threshold 1506 higher than the magnitude of peak 1502 can be established. The blocking threshold 1506 may represent the magnitude of peak 1502 that indicates a blockage in front of the optical measurement system. The blocking threshold 1506 may be set to a predetermined distance and / or percentage higher than the magnitude of the initial peak 1502, such that the predetermined distance and / or percentage is only exceeded when an object is in front of the optical measurement system.

[0169] Figure 15BThis describes an initial peak value 1504 in the later stages of the lifespan of an optical measurement system when a blockage is present, according to some embodiments. When an object is placed in front of the optical measurement system, most of the pulses can subsequently be reflected from the blocking object. When the blocking object is very close to the optical measurement system, the resulting peak value caused by reflections from the blocking object can be combined with other reflections from the housing of the optical measurement system as described above. In this example, the blocking object is close enough to the optical measurement system to receive reflections from both the housing and the blocking object, resulting in a peak value 1504 that is substantially higher than the peak value 1502 caused solely by reflections from the housing.

[0170] The system determines that blocking has occurred when an obstructing object is close enough to prevent the majority of light emitted by the optical measurement system from being reflected by other objects in the surrounding environment. This determination can be made when the magnitude of the initial peak value 1504 meets or exceeds the blocking threshold 1506. Figure 15B As described, reflections from the housing and the obstructing object can be combined to increase the magnitude of peak 1504 above threshold 1506. When this occurs, any of the corrective actions described above can be triggered. For example, some embodiments may enable the generation of warnings for users and / or automotive systems. Some embodiments may shut down any optical sensors in an array that detect obstruction. Some embodiments may activate other emergency or safety systems, such as indicating that the optical measurement system no longer provides an accurate description of objects in the surrounding environment.

[0171] VII. Compensation for reflections from the outer casing The use of peaks generated by reflections from the housing of the optical measurement system described above generally improves the performance of the optical measurement system through calibration systems and compensation for external influences. Some embodiments can alternatively improve the performance of the optical measurement system by removing peaks caused by housing reflections from histogram memory signals analyzed by peak detection circuitry or by an auxiliary processor. These embodiments can remove initial peaks from housing reflections to improve the short-range accuracy of the optical measurement system.

[0172] A. Near-range detection Figure 16AA histogram memory, according to some embodiments, is described, having an initial peak 1602 caused by reflection from the casing and a second peak 1604 caused by reflection from objects in the surrounding environment. As described above, when the second peak 1604 is sufficiently removed from the initial peak 1602, the distance calculation for the second peak 1604 is unaffected by the presence of the initial peak 1602. For example, when the object is 10 feet away from the optical measurement system, the peak caused by photons reflected from the object may appear at a point sufficiently far from the initial peak 1602 in the histogram memory such that the shape of the second peak 1604 is unaffected. Generally, the mid-range and long-range accuracy of the optical measurement system is unaffected by the presence of the initial peak 1602 from the casing reflection.

[0173] However, as the object moves closer to the optical measurement system, the residual effect of the initial peak 1602 can begin to influence the shape of the second peak 1604. For example, while photons begin to be received by reflections from the object, photons scattered from the inner casing of the optical measurement system can still be received by the photosensor. Because the additional photon count from the reflected photons can continue to be stored in the register, representing the time interval of the second peak 1604, the shape of the second peak 1604 can be skewed. When the shape changes, the peak detection circuit can identify the inaccurate position of the peak. For example, if the initial portion of the second peak 1604 increases due to photons reflected from the casing, then the center of the second peak 1604 can shift to the left when detected by the peak detection circuit.

[0174] Figure 16B This describes a short-range peak detection problem according to some embodiments, where reflections from the housing are not compensated. In this example, an object in the surrounding environment has moved close enough to the optical measurement system that a second peak 1604 caused by reflections from the object begins to merge with the initial peak 1604 caused by reflections from the housing. Figure 16B As explained, this has the effect of altering the magnitude and / or shape of the initial peak 1602 and changing or even concealing the presence of the second peak 1604. Because the magnitude of the second peak 1604 will be substantially smaller than the magnitude of the initial peak 1602, the shape and / or position of the second peak 1604 can be substantially reclassified by the initial peak 1602 to a point that can no longer be correctly identified as a peak by the optical measurement system.

[0175] The combined effect of this initial peak 1602 and the second peak 1604 will significantly reduce the short-range accuracy of the optical measurement system. Reflections from objects a few feet away from the optical measurement system can be affected by these shell reflections, leading to points adjacent to the optical measurement system that can create a "dead zone".

[0176] B. Remove shell reflections To compensate for the effects of reflections from the housing, the optical measurement system can use baseline measurements for the initial peak as described above. Specifically, the optical measurement system can record the baseline shape and / or position of the initial peak as observed over time during the measurement. In addition to using the variation of the initial peak to characterize the window of the optical measurement system, the shape and / or position of the baseline measurement of the initial peak can be used to compensate for reflections from the housing for short-range measurements.

[0177] Figure 17A illustrate Figure 16B Histogram memory. Additionally... Figure 17A The values ​​in the registers representing the baseline peak 1702 established from the average of previous measurements are also superimposed. Essentially, the baseline peak 1702 represents the quantity of photon counts attributable to reflections from the enclosure rather than from nearby objects of interest in the surrounding environment. Because the values ​​of each of these registers in the baseline peak 1702 are known, their values ​​can be removed from the histogram register of the current measurement.

[0178] Figure 17B This indicates the value after subtracting the baseline peak value of 1702 from the register in the histogram memory. Figure 16B A histogram memory is used. For example, the register value from the baseline peak 1702 can be subtracted from each register in the histogram memory of the current measurement. The resulting contents of the histogram memory show a second peak 1604 caused by a near-range object in the surrounding environment. It should be noted that the shape of the second peak 1604 can be preserved by removing the influence of the initial peak 1602. This allows for a significant improvement in the near-range performance and accuracy of the optical measurement system. Instead of creating a "dead zone" near the optical measurement system, near-range objects can be detected accurately even when they are extremely close to the optical measurement system.

[0179] Figure 18 This describes a circuit, according to some embodiments, for removing the effects of reflections from the system housing during short-range measurements. Figure 18 Similar to Figure 8 This includes an arithmetic logic circuit 804, a histogram memory 806, and various periodic signals that control the timing of the arithmetic logic circuit 804 and the histogram memory 806. This basic circuit can be used to generate a histogram in the histogram memory 806 from multiple excitations defined by the excitation or start signal 814 and the measurement signal 812 as described above.

[0180] Figure 18The histogram data path described also includes a baseline peak 1804, which stores the value of a histogram register representing the initial peak caused by reflections from the shell as described above. This baseline peak 1804 can be the average peak recorded in multiple previous measurements. The value at each register location can be averaged over a sliding window of previous measurements to eliminate any outliers, congestion, or other effects that may only temporarily affect the accuracy of the baseline peak 1804. Other mathematical operations can be used to combine multiple initial peaks recorded in previous measurements to generate the baseline initial peak, including maintaining a running average, identifying the minimum or maximum peak, using a step average finder, and other similar techniques.

[0181] Baseline peak 1804 can be provided to subtraction circuit 1802 so that the baseline peak 1804 can be subtracted from the corresponding register in histogram memory 806. For example, the memory interface can provide the first nine register values ​​from histogram memory 806 to subtraction circuit 1802 and subtract the corresponding value from the baseline peak 1804 of those register values. The register values ​​can then be returned to histogram memory 806 so that histogram memory 806 has the effect of reflections from the casing removed from the histogram. Subtraction circuit 1802 can be implemented using an arithmetic logic unit (ALU) or digital logic gates that implement subtraction functionality.

[0182] In some embodiments, the baseline peak 1804 may additionally or alternatively be provided to the processor 1806. Instead of using the subtraction circuit 1802 to compensate for reflections from the housing, the baseline peak 1804 may be transmitted to the processor 1806 along with raw and / or filtered data in the histogram memory 806, which may then remove the baseline peak 1804 from the current measurement. The processor 1806 may be located on the same integrated circuit as the rest of the optical measurement system, which includes the histogram memory 806 and arithmetic logic circuitry 804. Alternatively, the processor 1806 may be located on a separate integrated circuit, different from the integrated circuit implementing the optical measurement system. The processor 1806 may receive filtered and / or raw data from the histogram memory 1806 and may then use an instruction set to perform operations to remove the effects of reflections from the system housing as described above.

[0183] VIII. Peak detection using a matched filter As described in detail above, an optical measurement system generates pulse trains emitted from a light source and directed into the surrounding environment. Light from these pulse trains can be reflected from objects in the surrounding environment and received by corresponding photosensors in the optical measurement system. When photon counts are received by the photosensors, they can be accumulated in a register of a histogram memory (e.g., SRAM) to form a histogram representation of the photons received during the optical measurement. The accumulated photon counts in the histogram memory represent the raw measurement data, which may include reflected photons from the light source and photons from background noise in the surrounding environment. Depending on the noise level of the background environment, this raw data in the histogram memory may have a relatively low signal-to-noise ratio (SNR). This noise level in the histogram can increase the difficulty associated with performing on-chip peak detection. To improve the confidence level in the on-chip peak detection circuitry, the optical measurement system can apply one or more filters to the raw data in the histogram memory, after which a peak detection algorithm using the filtered data can be executed.

[0184] While filtering the data does improve the SNR of the data in the histogram memory, it can also obscure the properties of the histogram of reflected photons, which can be used for calculating distance measurements, performing statistical analysis of the histogram data, detecting edge cases, detecting nearby objects, distinguishing adjacent reflection peaks, curve fitting, and so on. The embodiments described below utilize the benefits of filtering the data in the histogram memory and preserving additional information provided by the raw data received from the optical sensor. The filtered data can be used to perform on-chip peak detection to subsequently identify time windows in the unfiltered histogram memory that may contain the detected peaks. After identifying the time windows using the filtered data, the integrated circuit in which the histogram memory is implemented can transfer the time interval containing the unfiltered histogram data during those time windows to an external processor for distance calculations and other analyses of the raw histogram data.

[0185] The following sections first describe how the filtering process can use matched filters for single-pulse scenarios to detect time windows in unfiltered data to be transmitted off-chip. Then, more complex instances are presented using pulse trains containing multiple pulse codes and more sophisticated matched filters. Some filters can be designed to compress the data of these longer pulse codes without sacrificing the shape information of the original data.

[0186] A. Remote peak detection Figure 19This describes the contents of a histogram memory after a single pulse has been transmitted and received by an optical measurement system, according to some embodiments. As described above, the initial time interval in the histogram memory may contain photon counts caused by the initial pulse reflected from the housing and / or window of the optical measurement system. When the light source of the optical measurement system emits photons into the surrounding environment, a portion of these photons may reflect back to the corresponding photosensor without leaving the housing of the optical measurement system. This early reflection of photons may cause a relatively large peak 1902 in the initial time interval stored in the histogram memory. Some embodiments may process the contents of the histogram memory such that the initial peak 1902 is distinguishable from subsequent peaks reflected from objects in the surrounding environment. For example, some embodiments may ignore peaks above a predetermined intensity threshold, which may only be exceeded by the initial peak 1902. Some embodiments may ignore peaks occurring in the initial time interval (e.g., during the first 20 time intervals). Some embodiments may average multiple signals to estimate the background level of noise including the initial peak 1902, and this average signal may be subtracted from the measurement signal to effectively remove the initial peak 1902. The initial peak 1902 can therefore be identified and distinguished from subsequent peaks representing reflections from the object of interest in the surrounding environment.

[0187] Following the initial peak 1902, the histogram memory may later contain one or more peaks caused by the reflection of photons from objects in the surrounding environment. For example, peak 1904 may appear after the initial peak 1902 and may represent photons reflected from objects in the surrounding environment. It should be noted that both peaks 1902 and 1904 can be generated by photons emitted as part of the same pulse train; however, peak 1902 may be reflected from the housing / window of the optical measurement system, while peak 1904 may be reflected from objects in the surrounding environment. The shapes of peaks 1902 and 1904 may generally correspond to the shape of the pulse emitted from the optical measurement system. For example, some embodiments may emit pulses with a square shape. Figure 19 As explained, the photon count received by the histogram memory can have an approximate square shape corresponding to the shape of the emitted pulse. Background noise, reflection patterns, and other environmental influences can cause the shapes of the received peaks 1902 and 1904 to deviate slightly from the ideal square shape of the emitted pulse.

[0188] As described above, Figure 19The photon counts in the histogram described herein can contain significant background noise and other effects that can make identifying peaks in the histogram memory difficult. Some embodiments may include on-chip peak detection circuitry that identifies peaks in the histogram memory. Because the total length and amount of data stored in the histogram memory can be relatively large, this peak detection circuitry can identify time windows in the histogram memory that contain the identified peaks. The integrated circuit in which the histogram memory is implemented can then transmit the histogram data from these time windows to an off-chip processor for distance calculation. Transmitting windows of peak data, rather than the entire contents of the histogram memory, greatly reduces the bandwidth of data that needs to be transferred between the integrated circuit and the processor. However, these on-chip peak detection algorithms / circuits are best performed when the raw data in the histogram memory is first filtered to increase the SNR.

[0189] Some embodiments may use a matched filter to increase the SNR of the raw data in the histogram memory. The shape of the matched filter may correspond to the shape of the initial pulse emitted by the optical measurement system. For example, matched filter 1906 may be applied to a system with, for example, a matched filter such as... Figure 19 The original data is stored in a generally square-shaped histogram memory as described herein. Filter 1906 may include a series of constant values, such as "1" or other values, which may be convolved with the original unfiltered histogram data to produce a filtered version of the histogram data. Other filtering shapes may be used, such as patterns with one or more zeros between sets of one or more non-zero values ​​(e.g., 1 and -1). Such patterns may occur when different excitations are assigned different weights, as described in U.S. Patent Publication 2018 / 0259645, which is incorporated herein by reference in its entirety.

[0190] Figure 20 Description of usage according to some embodiments Figure 19 The filter 1906 is a filtered version of the histogram data. Generally, the filter 1906 can act as a low-pass filter to reduce the effect of any background noise in the signal, thereby increasing the SNR. The low-pass filter also has the effect of increasing the width of any reflection peaks in the histogram data. In some embodiments, the width of the matched filter 1906 can be approximately the same as the width of the pulse transmitted from the light source of the optical measurement system. For example, for a pulse approximately four time intervals wide (e.g., 8 ns), the corresponding matched filter 1906 can also be approximately four time intervals wide. This can have the effect of approximately doubling the width of the peaks in the filtered histogram data (e.g., eight intervals wide).

[0191] exist Figure 20 In the diagram, the first peak 2002 can be represented by reflections from the housing / window of the optical measurement system. Figure 19The filtered version of the initial peak 1902. Similarly, the second peak 2004 can represent the reflection caused by the object of interest from the surrounding environment. Figure 19 The filtered version of the second peak 1902 in the image. It should be noted that the low-pass filter operation of matched filter 1906 has altered the shape of peaks 2002 and 2004 by increasing the width of peaks 2002 and 2004 and smoothing the overall shape of peaks 2002 and 2004. This can be used to more clearly distinguish peaks 2002 and 2004 from any noise incidentally stored in the histogram memory.

[0192] While matched filter 1906 makes it easier to identify peaks 2002 and 2004 in the filtered histogram data, this filtering operation also distorts the shape of the histogram data. While this results in a higher SNR and a higher confidence level for peak detection, it also removes important information that could be represented in the unfiltered histogram data. This information is useful to processors performing distance calculations, statistical analyses, and other calculations that can be best performed using the details available in the unfiltered data. While some embodiments may send peaks 2002 and / or 2004 from the filtered histogram data to the processor for distance calculation, some embodiments may use the positions of peaks 2002 and 2004 identified in the filtered histogram data to identify the corresponding peak positions in the unfiltered histogram data, and then send the unfiltered peaks from the unfiltered histogram data to the processor. This process will be described in more detail below.

[0193] B. Proximity Peak Detection In addition to losing signal information through the filtering process described above, some embodiments may also have difficulty distinguishing adjacent peaks without using unfiltered histogram data.

[0194] Figure 21A This illustrates an example of two adjacent peaks that converge in time. The first peak 2102 may correspond to the initial pulse reflection from the housing / window of the optical measurement system as described above. The second peak 2104 may correspond to an object near the optical measurement system. Figure 21A This represents the unfiltered histogram data in the histogram memory after photon counts from these two different reflections. It should be noted that in the unfiltered histogram data, the two peaks 2102 and 2104 are distinguishable from each other. Although they begin to merge, the first peak 2102 can be distinguished from the second peak 2104 using a peak detection algorithm. For example, a peak detection algorithm can scan the entire histogram memory register and identify time interval patterns of increasing and then decreasing threshold amounts. Although peaks 2102 and 2104 are temporally converging and at least partially overlapping, the second peak 2104 can still be distinguished using this type of peak detection algorithm.

[0195] Figure 21B The low-pass filter according to some embodiments can be used for... Figure 21A The impact of unfiltered histogram data. A low-pass filter can be a matched filter, such as... Figure 19 The matched filter 1906 increases the total width of the first pulse 2102 and smooths any sudden changes in the histogram data. In doing so, the first peak 2102 can be mixed with the time interval of the second peak 2104. Figure 21B This explains how the two peaks 2102 and 2104 can be encapsulated into a single peak 2106 so that the two peaks 2102 and 2104 are no longer distinguishable in the filtered histogram data. When the filtered histogram data is analyzed and used to calculate distance measurements, near-range objects in the surrounding environment represented by peak 2104 can be lost in the filtered histogram data. Therefore, the near-range accuracy of the optical measurement system can be affected when filtered histogram data is used without unfiltered histogram data.

[0196] Although Figures 21A to 21B The examples above illustrate the problems associated with near-field object detection, but the same principle can be applied to objects that are close to each other at any range. For instance, peaks 2102 and 2104 could represent objects in the surrounding environment that are both quite far from the optical measurement system (e.g., 15 feet, 20 feet, etc.). However, if these objects are extremely close to each other (e.g., 1 foot, 2 feet, etc.), the resulting peaks in the unfiltered histogram data can be relatively clustered together. When the histogram data is filtered, these two peaks can merge into a single peak, which can cause the optical measurement system to fail to distinguish between the two objects in the surrounding environment.

[0197] IX. Use filtered data to provide a window in the unfiltered data. To address these and other technical issues, some embodiments may filter the histogram data, but the unfiltered data can be sent to a processor for distance calculation, statistical analysis, curve interpolation, peak fitting, etc. Filtered data can be used to identify the locations of peaks in the histogram; however, instead of simply sending a time interval window containing only filtered peak data, these embodiments may alternatively or additionally use the locations(s) identified in the filtered histogram data to identify corresponding locations(s) in the unfiltered histogram data. The time interval window can then be sent to the processor, and the processor can use the unfiltered histogram data to analyze the peaks for distance calculation.

[0198] A. Locating the window in the unfiltered data Figures 22A to 22B This illustrates how filtered data, according to some embodiments, can be used to identify peaks in unfiltered data. Figure 22A The curve in the middle represents Figure 20 The filtered histogram data. Similarly, Figure 22B Curve description Figure 19 Unfiltered histogram data. Filters can be applied to... Figure 22B Unfiltered histogram data in the data to obtain Figure 22A The peak detection circuit / algorithm can be executed on the filtered histogram data to identify any peaks in the filtered histogram data. As described above, the peak detection algorithm can be executed on the filtered histogram data because filtered histogram data can have a higher SNR, a clearer curve, fewer transient spikes, and / or better overall results compared to using unfiltered histogram data.

[0199] The peak detection circuit / algorithm can initially identify a first peak 2002. However, as described above, the peak detection circuit can be configured to exclude the first peak 2002 detected in the unfiltered histogram data under the assumption that the first peak 2002 corresponds to the initial reflection of a light pulse exiting the housing / window of the optical measurement system. The peak detection algorithm can continue scanning the unfiltered histogram data until a second peak 2004 is detected. Generally, the peak detection circuit / algorithm can identify the location of the maximum value of peak 2004. For example, the peak detection circuit / algorithm can identify a time interval with local maxima around which peak 2004 is centered.

[0200] After detecting the center of peak 2004, the peak detection circuit / algorithm can identify surrounding time intervals that can be considered a portion of peak 2004. For example, some embodiments can identify a predetermined number of time intervals (e.g., 17 time intervals) centered around the peak location. Some embodiments can identify time intervals within a threshold amount of the peak's maximum value (e.g., within 50% of the maximum value). Some embodiments can select the number of time intervals centered around the maximum value based on the width of the light pulse emitted by the optical measurement system and the width of the filter. For example, if the emitted light pulse is four time intervals wide and the corresponding matched filter is also four time intervals wide, then the resulting peak in the filtered histogram data can be expected to be at least eight time intervals wide. Therefore, the peak detection circuit / algorithm can identify at least eight time intervals (e.g., 10 time intervals, 12 time intervals, etc.) centered around the peak's maximum value as representing peaks in the unfiltered histogram data.

[0201] Some embodiments may transmit one or more time interval windows to the processor off-chip, containing filtered peak data as described above. For example, an initial peak and three additional peaks corresponding to objects in the surrounding environment may be identified by the algorithm described above. Time interval windows centered around these peaks may be identified, and filtered data within those time intervals may be sent to the processor for processing. Using unfiltered data is acceptable in situations where the filtered data can adequately capture distance information about the surrounding environment.

[0202] Alternatively or additionally, some embodiments may use the location of peaks identified in the filtered data to identify time interval windows in the unfiltered data, and these time interval windows in the unfiltered data may be transmitted to a processor for processing. These windows of the unfiltered data may be transmitted in addition to or instead of the filtered data windows described above.

[0203] To identify a window of unfiltered data in an unfiltered histogram, the optical measurement system can use the location of the maximum value of the peak identified in the filtered histogram data. For example, the location of the maximum value of peak 2004 in the filtered histogram data can be used as the location of the maximum value of peak 1904 in the unfiltered histogram data. After identifying this location in the unfiltered histogram data, a time interval window of surrounding maximum values ​​can be identified using the techniques described above regarding the corresponding location in the unfiltered data (e.g., a predetermined number of time intervals, time intervals within a percentage range of the maximum value, etc.). As described above, the number of time intervals identified in the time interval window of the filtered histogram data can be based on the width of the emitted light pulse and the width of the applied filter. This can result in a time interval window that is at least twice the width of the emitted light pulse. However, when identifying a time interval window in the unfiltered histogram data, a smaller number of time intervals can be used. Instead of considering the way in which the low-pass filter increases the width of peak 2004, the width of the unfiltered peak 1904 can be smaller. For example, some embodiments may use a number of time intervals slightly larger than the width of the emitted light pulse (e.g., more than 2 time intervals, 4 time intervals, 6 time intervals, 8 time intervals, etc. of the emitted light pulse).

[0204] This approach, using filtered data to identify time windows within unfiltered data, retains the benefits of using unfiltered data while also preserving the benefits of using filtered data. Higher SNR and smoother peak profiles allow on-chip peak detection circuitry to accurately identify peaks in the histogram. This allows chips with histogram memory implementations to send only a small subset of the information stored in the histogram memory to the processor for processing. However, by sending unfiltered data within these time interval windows to the processor, the processor retains all the benefits of using the entire unfiltered data for processing.

[0205] Briefly return to Figures 21A to 21B , Figure 21B The peak value 2106 in the signal can be easily identified by the peak detection circuit on the chip, and Figure 21A The two peaks 2102 and 2104 can be transmitted to the processor within the time window using the method described above. The processor can then use various techniques to analyze the data in the unfiltered time window to identify whether the time window contains two peaks 2102 and 2104 rather than just a single peak. When calculating distance, the processor can also use the unfiltered data to more accurately identify the center of these peaks 2102 and 2104.

[0206] B. Circuit used for transmitting unfiltered data outside the chip Figure 23 This illustration shows a circuit according to some embodiments for using filtered data to transmit unfiltered data to a processor for distance calculation. Figure 23 Similar to the description above Figure 8 It includes an arithmetic logic circuit 804 that receives signals 816 from a photodetector in a light sensor and aggregates those signals into a photon count, the photon count being aggregated in a histogram memory 806 within multiple excitations in the optical measurement. The values ​​stored in the histogram memory 806 can represent the unfiltered histogram data described above.

[0207] Figure 23It also includes a filter 2302 that can be applied to photon counting in histogram memory 806. Filter 2302 may include a matched filter with a shape corresponding to the shape of the emitted light pulse from the optical measurement system. Filter 2302 may also operate as a low-pass filter to smooth the overall shape of the histogram data. In some embodiments, filter 2302 may be applied incrementally to time intervals in histogram memory 806. For example, filter 2302 may include a sequence of values ​​(e.g., 1 1 1 1) that can be convolved with time intervals in histogram memory 806. In some embodiments, the filtered value may be passed to peak detection circuit 2308 as it is generated by filter 2302. Peak detection circuit 2308 may then evaluate the filtered value upon receipt to detect a peak when it appears in histogram memory 806. As described above, peak detection circuit 2308 operates by identifying peaks in the filtered data characterized by increasing values ​​followed by decreasing values ​​(and vice versa). Peak resolution circuit 2308 can record the location of local maxima identified when filter 2302 passes through unfiltered histogram memory 806. This method allows the filtered histogram data to be computed during a single pass without requiring all the filtered histogram data to be stored in a separate memory. When the filtered histogram data is transferred to processor 2314 for processing, peak detection circuit 2308 can store the filtered values ​​near the detected peaks as a time interval window as described above.

[0208] In some embodiments, the output of filter 2302 may be stored in a separate buffer 2306. As described above, in embodiments where peak detection circuit 2308 operates on the direct output of filter 2302, separate buffer 2306 may not be necessary. However, embodiments that store the output of filter 2302 in separate buffer 2306 may then simultaneously store both unfiltered histogram data in histogram memory 806 and filtered histogram data in buffer 2306. Peak detection circuit 2308 may then pass through buffer 2306 to detect peaks in the filtered histogram data. Storing the filtered histogram data in buffer 2306 allows peak detection circuit 2308 to pass through the buffer multiple times and utilize iterative techniques to identify peaks in the filtered histogram. For example, each pass through buffer 2306 may detect a maximum peak value that is different from the maximum value detected during previous iterations of the peak detection algorithm.

[0209] In some embodiments, Figure 23The histogram data path depicted can operate as a multi-stage pipeline. For example, after the current measurement is completed, the first filter 2302 can use the value in the histogram memory 806 to populate the buffer 2306. During subsequent measurements, the peak detection circuit 2308 can operate on the value stored in the buffer 2306 from the previous measurement. At the same time, the histogram memory 806 can be reset and can begin receiving photon counts from the current measurement. In other embodiments, the peak detection circuit 2308 can identify peaks and send values ​​from the histogram memory 806 before receiving new values ​​from the next measurement.

[0210] After the maximum value in the filtered data is identified by the peak detection circuit 2308, one or more time interval windows 2310 can be identified in the unfiltered data in the histogram memory 806. These windows 2310 may include multiple time intervals surrounding the maximum value of the peak identified by the peak detection circuit 2308. Window 2310 may be filled with data from the histogram memory 806 as unfiltered values ​​2312. These unfiltered values ​​2312 may then be transmitted to the processor 2314 for processing, such as distance calculation, curve fitting, etc. In some embodiments, the interval identifier may be transmitted to the processor 2314 together with or in place of the photon count in the interval itself.

[0211] Processor 2314 may be implemented on an integrated circuit that is separate from and distinct from the integrated circuit on which histogram memory 806 is implemented. In some embodiments, a first integrated circuit, such as an application-specific integrated circuit (ASIC), may be fabricated including arithmetic logic circuitry 804 and histogram memory 806. The ASIC may also include peak detection circuitry 2308 and optional buffer 2306 (when it is part of the design). Thus, when referring to operations performed "on-chip," these operations may be performed on the first integrated circuit.

[0212] Some embodiments may also include a second integrated circuit including processor 2314. Processor 2314 may include a microcontroller, microprocessor, field-programmable gate array (FPGA) implementing a processor core, ASIC implementing a processor core, and / or the like. Unfiltered value 2312 can be transmitted between the first integrated circuit and the second integrated circuit including processor 2314 via printed leads on a circuit board.

[0213] exist Figure 23In this example, the unfiltered value 2312 from the histogram memory 806 is only shown as being transmitted to the processor 2314. However, other embodiments may also transmit filtered data to the processor 2314. In some embodiments, filtered data need not be transmitted to the processor 2314 because the processor 2314 can actually re-execute the filter 2302 on the unfiltered data to achieve a similar result. In these embodiments, reducing the bandwidth of information transmitted between the first integrated circuit and the second integrated circuit may be more important than ensuring that the number of processing operations performed by the processor 2314 is minimized.

[0214] X. Filtering of multipulse codes The above embodiments use a single pulse code as filtered data as a simplified example of how it can be used to identify unfiltered data and transmit it to a processor. However, the techniques described above are equally applicable to more complex multi-pulse codes. Multi-pulse coding can consist of a burst of pulses or multiple pulses transmitted as part of a single excitation during measurement. These pulse codes can be replicated during each excitation phase of the measurement. Using multi-pulse codes provides a more distinct pattern for detection purposes, and this minimizes false positives and other spurious peaks that can be caused by ambient noise rather than photons reflected from the object of interest in the surrounding environment. The following examples illustrate how these techniques can be used with multi-pulse codes to transmit unfiltered data to a processor.

[0215] A. Low-pass summing filter Figure 24 This describes a portion of the histogram memory after receiving reflected photons from a multi-pulse code, according to some embodiments. The multi-pulse code can be significantly larger than a single pulse code, and therefore filtering operations may be even more important for correctly identifying peaks in the dataset that characterize reflection patterns from objects in the surrounding environment. Figure 24 This describes a multipulse code with the following format: 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 -1 -1 -1 -1 0 0 0 0 1 1 1 1 0 0 0 0 This specific pulse code can consist of multiple excitations from a light source of the optical measurement system during each excitation period in the measurement. For example, the first excitation may include... Figure 24 The pulses with positive "1" peaks in the histogram. When those peaks are received by the optical sensor, they can be given positive weights and added to the histogram memory time interval. The second excitation may contain pulses with... Figure 24 The pulse with the negative "-1" peak described herein. When this peak is received by the optical sensor, it can be given a negative weight and added to the histogram memory time interval. The combination of these two excitation types can be used to generate pulses such as... Figure 24 The positive and negative peak values ​​in the histogram memory are illustrated in the diagram. Because of the overall pulse code length, the time interval window 2402 in the histogram memory, which contains the entire pulse code, can be significantly larger than the time interval window described above. For example, a multi-pulse code can occupy approximately 200 time intervals in the histogram memory.

[0216] When multipulse coding is received as reflected photons in a histogram memory, this can be represented as positive / negative peaks in the photon count. Figure 24 In this example, multipulse coding can cause peaks 2404, 2406, 2408, and 2409 in the histogram memory. The histogram memory can then be processed using a matched filter 2416 based on the shape of the multipulse code transmitted from the light source of the optical measurement system.

[0217] Figure 25 This describes a filtered version of the peak received from multi-pulse coding according to some embodiments. When the matched filter 2416 is convolved with peaks 2404, 2406, 2408, and 2409 in the histogram memory, the resulting filtered version of the histogram data may contain a large peak 2504 rather than multiple individual peaks. A time interval window 2502 near the filtered peak in the filtered data can be specified, and this peak 2504 can be identified by the peak detection circuitry described above.

[0218] Using the technique described above, the location of the peak 2504 in the filtered data can be used for identification. Figure 24 The unfiltered data is processed within a time interval window 2402. Unfiltered peaks 2404, 2406, 2408, and 2409 can then be sent to the processor for processing within time interval window 2402. As described above with respect to other embodiments, in addition to the unfiltered data, Figure 23 The filtered data can also be transmitted to the processor.

[0219] B. Compression Filter Figure 26 This describes a filter that uses only a single binary indicator according to some embodiments. Figure 24 The matched filter 2416 uses the same binary indicator for every value in the filter. For example, the sequence "1 1 1 1" is used in filter 2416 to correspond to the first peak in multi-pulse coding. However, using this uniform sequence of values ​​in filter 2416 produces a low-pass effect on the filtered data. As described above, this alters the shape of the filtered data, causing many characteristics of the unfiltered data to be lost in the filtered data.

[0220] exist Figure 26In some examples, the reflection peaks 2604, 2606, 2608, and 2609 in the unfiltered histogram data may not have the same precise square shape as the multi-pulse code transmitted from the light source of the optical measurement system. For instance, the photodetector may exhibit a "stacking" effect, where photons received at the beginning of the peak reflection can create an avalanche effect, after which some time can be spent for the photodetector to reset. This makes the peaks have... Figure 26 The shapes described herein result in peaks 2604, 2606, 2608, and 2609 having a more pointed or slanted shape than the ideal square shape described in the other examples above.

[0221] To preserve this shape in the filtered data, some embodiments may use a single binary indicator that still sums the peaks 2604, 2606, 2608, and 2609 into a single peak without performing a low-pass filter operation. Filter 2616 may use only a single value or indicator at the beginning of each pulse instead of the uniform set of values ​​for the entire duration of each pulse. Figure 26 In this example, filter 2616 may have the following values: 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 -1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 This filter can generate something similar to Figure 25 The peaks described herein are peaks that, in general, will resemble the peaks 2604, 2606, 2608, and 2609 in the unfiltered data. Therefore, the filtered data can retain the peaks more accurately. Figure 26 The shape of the peaks is described in the figure. In some embodiments, the filtered data, which preserves the shape of the peaks, may then be passed to a processor for processing.

[0222] In some embodiments, Figure 24 and Figure 26 The two filters described herein can operate in series simultaneously. For example, unfiltered data can be processed using filters 2416 and 2616 to generate two filtered versions of the data. Since the first part of each filter can use the same arithmetic operations, the circuitry can be shared between the two filters due to their parallel operation. For example, the first stage of the summation step used for each of these filters can be shared. Some embodiments may then use a low-pass filtered version of the histogram data to identify peaks, followed by a compressed version of the histogram data that retains its shape due to the low-pass filter, and then send it to the processor.

[0223] It should be noted that the example filters 2416 and 2616 described above with respect to multi-pulse codes use a single bit as a designator in the filter taps. However, this is only done by way of examples in this disclosure. The actual value of each filter may contain multiple bit values, such as 10-bit values ​​instead of a single bit. As described above, these values ​​may be convolved and computed as a sliding window to detect resulting peaks in the filtered data.

[0224] XI. Methods for providing unfiltered data Figure 27 This document describes a flowchart illustrating a method for analyzing filtered and unfiltered data in an optical measurement system, according to some embodiments. The following method describes a process for a single light sensor in an optical measurement system. However, as described above, an optical measurement system may contain many light sensors or "pixels," and this method can be performed for each light sensor in the optical measurement system.

[0225] At step 2702, the method may include transmitting one or more pulse trains from a light source within one or more first time intervals as part of an optical measurement. Each of the one or more first time intervals may represent an "excitation" repeated multiple times in the measurement. Each of the first time intervals may include one or more pulse trains encoded and transmitted by the light source such that the pulse trains can be considered as being reflected from objects in the surrounding environment. Each time interval may be subdivided into multiple time intervals such that each time interval represents an interval in a histogram of photon counts received during the optical measurement. (The above is in contrast to...) Figure 9 Describe instances of how a single measurement can contain multiple excitations subdivided into time intervals for clustered photon counting.

[0226] At step 2704, the method may further include using a light sensor to detect photons from one or more pulse trains. As described in detail above, the light sensor may include multiple photodetectors, such as multiple SPADs. The light sensor may receive reflected light as well as ambient background noise received from the surrounding environment. The reflected light received by the light sensor may include light reflected from objects of interest in the surrounding environment. For example, these objects of interest may be at least 30 cm away from the light sensor and may represent surrounding vehicles, buildings, pedestrians, and / or any other objects that may be encountered near the optical measurement system during use. These objects of interest may be distinguished from objects that are part of the optical measurement system itself, such as a housing. In some embodiments, the reflected light received by the light sensor may also include light reflected from the housing or other parts of the optical measurement system. These reflections may include reflections primarily directly from windows or the housing of the optical measurement system, as well as secondary reflections reflected near the interior of the optical measurement system. The light sensor may also be coupled to threshold detection circuitry and / or to arithmetic logic circuitry for accumulating photon counts. This combination may be referred to above as a “pixel.” Figure 5 This section describes an example of how photon counts can be received from a light sensor and how a threshold circuit and a pixel counter (e.g., an arithmetic logic circuit) can be used to count the photon counts.

[0227] At step 2706, the method may further include accumulating photon counts from the optical sensor into multiple registers to represent a histogram of photon counts received during the current measurement. These photon counts may be accumulated in multiple registers within a memory block corresponding to the optical sensor. The multiple registers may be implemented using registers in the SRAM of the histogram data path, as described above. Figures 8 to 9 As described herein. Each time interval can be subdivided into multiple first time intervals for the histogram. The corresponding time interval in each of one or more time intervals can be accumulated in a single register among multiple registers. Each time interval can represent an excitation, and one or more time intervals together can represent a measurement for the optical measurement system. Each time interval can be defined by a start signal or excitation signal that resets the memory representing the histogram back to the first register in the histogram. Received photons reflected from the housing of the optical measurement system can be stored in the histogram memory in the same manner as other received photons reflected from surrounding objects outside the optical measurement system.

[0228] At step 2708, the method may additionally include filtering histograms in multiple registers to provide filtered histograms of photon counts from the multiple registers. The filter may include a matched filter based on the shape of one or more pulse trains transmitted from the optical measurement system. The filter may be convolved with unfiltered histograms in the multiple registers and may be stored in separate buffers. The filter may include square pulses with repeating non-zero values. The filter may also include a single non-zero value followed by multiple approximately zero values. The filter may be as described above relative to... Figure 19 , Figure 24 and / or Figure 26 The operation described.

[0229] At step 2710, the method may further include detecting the location of a peak in the filtered histogram. Depending on the specific embodiment, detecting this peak may be performed using several different techniques. In some embodiments, iterations may be performed via the filtered histogram to identify the peak. The system may sequentially access values ​​in the filtered histogram, starting at the beginning of the measurement to identify the peak. The peak may be identified by identifying increasing values ​​followed by decreasing values ​​in the filtered histogram. Similarly, the center of the peak may be identified by identifying values ​​in histograms that have smaller values ​​in the time intervals on either side.

[0230] At step 2712, the method may further include identifying locations in multiple registers of the unfiltered representation of the peak. These locations can be identified using the location of the peak in the filtered histogram. Therefore, filtered histogram data can be used to identify the location of the peak in the unfiltered histogram data. Identifying the peak in the unfiltered histogram may involve locating a time interval window that appears near the center of the peak. This process may involve expanding outward from the peak location to determine the range of registers in the histogram memory that encompass the entire peak. For example, some embodiments may identify a predetermined number of time intervals near the location of the maximum value in the unfiltered histogram, which may be designated as the peak. For example, a predetermined number of time intervals, such as 3 time intervals, 5 time intervals, 9 time intervals, 15 time intervals, 17 time intervals, etc., may identify the maximum value and center around the maximum value to represent the entire peak. Some embodiments may identify time intervals surrounding values ​​that are within a percentage of the maximum value in the peak register. This can result in a variable number of time intervals that can be used to represent the peak depending on the width of the peak. For example, a time interval surrounding the maximum value may be included in the peak when its value is within 25% of the maximum value. The above text is relative to Figures 22A to 22B This describes the process of identifying the representation of peaks in an unfiltered histogram.

[0231] At step 2714, the method may further include sending an unfiltered representation of the peak to a processor to calculate the distance to the object. The distance to the object may represent the distance between the optical measurement system and an object in the surrounding environment. This calculation can be performed using the unfiltered representation of the peak sent to the processor. The processor may be physically separate from and distinct from the integrated circuit on which the histogram memory is implemented. Thus, the histogram memory may represent a first integrated circuit, and the processor may represent a second integrated circuit communicating with each other via a printed circuit board. (The above is in contrast to...) Figure 23 Describe an example of this circuit.

[0232] It should be understood that Figure 27 The specific steps described herein provide specific methods for using filtered histogram data to identify the location of peaks in unfiltered histogram data and for transmitting the unfiltered data to a separate processor, according to various embodiments. Other orders of steps may also be performed according to alternative embodiments. For example, alternative embodiments of the invention may perform the steps outlined above in a different order. Furthermore, Figure 27 The individual steps described may contain multiple sub-steps that can be performed in various sequences suitable for the individual steps. Furthermore, additional steps may be added or removed depending on the specific application. Those skilled in the art should recognize the many variations, modifications, and alternatives.

[0233] XII. Neighboring pixels in combination space In certain environmental conditions, when measuring distances to objects, optical measurement systems may struggle to provide reliable responses above a predetermined confidence level. These conditions may include weather phenomena such as rain, fog, or mist. They may also include situations where the object is so far from the optical measurement system that a reliable measurement cannot be generated based on the number of photons accurately reflected by the object in the environment and received by the optical sensor. In any of these cases, the optical measurement system can use a detection threshold to prevent false positives from being detected by the system.

[0234] To improve the confidence level of certain measurement conditions, some embodiments may use a form of spatial filtering. These embodiments may assume spatial correlation between photons received by spatially proximate optical sensors. For example, physically proximate optical sensors may receive photons reflected from the same objects in the surrounding environment. Even if the responses of these individual optical sensors fall below a detection threshold, techniques described below for combining the responses of these proximate optical sensors to improve the confidence of distance calculations result in a combined response above the detection threshold.

[0235] Figure 28 This section describes an example, according to some embodiments, of an object that can provide reflected photons to a nearby optical sensor. In this example, an object 2800 in the surrounding environment may include, for example... Figure 28The aperture markers described above. Other instance objects may include other street signs, buildings, vehicles, pedestrians, etc. Depending on the distance between object 2800 and the optical measurement system, object 2800 may be large enough that photons from multiple light sources are reflected back to the corresponding photosensors in the optical measurement system. Therefore, more than one neighboring photosensor can receive photons reflected from object 2800, and thus multiple photosensors can independently use the photon count reflected from object 2800 to calculate the distance to object 2800. For example, each of these photosensors may be associated with a non-dependent histogram data path and, as described above, relative to... Figure 8 The described histogram memory is associated.

[0236] Figure 28 This illustrates how multiple light sensors can "see" different parts of the same object 2800. In this example, nine light sensors can have viewing angles of object 2800. These viewing angles are indicated by the circles marked on object 2800 by aperture markers. These circles may be referred to herein as "light sensor viewing angles" to distinguish them from what the light sensors see and the light sensors themselves. It should be noted that, except... Figure 28 In addition to the nine light sensor viewing angles explicitly stated in the document, an additional light sensor may also have a viewing angle of 2800 for the object.

[0237] Depending on environmental conditions or distance from object 2800, some parameters within the optical sensor's field of view may fail to generate results that meet or exceed detection thresholds. Detection thresholds may be implemented differently depending on the embodiment. For example, the detection threshold may be based on a photon count of at least a threshold number of photons received from the object 2800. The detection threshold may be based on a calculated distance from the object 2800. The detection threshold may be based on a confidence level that distinguishes peak values ​​from noise in the corresponding histogram. For example, a particularly noisy environment with low SNR may fail to generate results that meet or exceed the detection threshold. The detection threshold may also be implemented as a peak detection threshold to detect peak values ​​in the histogram.

[0238] exist Figure 28In the example, light sensor perspectives 2802, 2810, and 2816 may generate results exceeding the detection threshold. These light sensor perspectives 2802, 2810, and 2816 are shaded with a darker color to distinguish them from other light sensor perspectives 2804, 2806, 2808, 2812, 2814, and 2818 of the object 2800 that generate results not exceeding the detection threshold. It should be noted that light sensors with adjacent light sensor perspectives may generate different results. This difference may be due to slightly different environmental conditions for each light sensor and / or slightly different reflective properties at different regions of the object 2800. This situation is more likely to occur at the detection limits of the optical measurement system. For example, when the object 2800 begins to be occluded or moves out of the range of the optical measurement system, the light sensors in the array may begin to "exit" the detection of the object 2800 because their results begin to fall below the detection limit.

[0239] Some embodiments may ignore any light sensor that drops below the detection limit. However, other embodiments may utilize the spatial correlation of neighboring pixels. It is generally assumed that neighboring light sensors are spatially correlated in terms of viewpoint. Figure 28 In these examples, each of the light sensor's viewpoints is spatially correlated because it is reflected from the same object 2800. Because of this spatial correlation, adjacent light sensors can be considered together when evaluating optical measurements. Instead of relying solely on isolated individual light sensors, these embodiments can effectively consider combinations of light sensor responses to characterize and / or generate distance measurements for each light sensor.

[0240] exist Figure 28 In this example, the central optical sensor viewpoint 2810 may be spatially adjacent to eight additional optical sensor viewpoints 2802, 2804, 2806, 2808, 2812, 2814, 2816, and 2818. These additional optical sensor viewpoints can be considered spatially adjacent when they are diagonally adjacent, such as optical sensor viewpoints 2802, 2806, 2814, and 2818, and orthogonally adjacent, such as optical sensor viewpoints 2804, 2808, 2812, and 2816. The characteristics and / or calculations of the distance measurements of the central optical sensor viewpoint 2810 can utilize information derived from the histograms of each of these other spatially adjacent optical sensors.

[0241] A. Combined distance measurements Figure 29 Examples illustrating how spatially proximate light sensor viewing angles 2808, 2812, according to some embodiments, can be used to calculate distance measurements for one of the corresponding light sensors 2810. Figure 29 It can be Figure 28The simplified view focuses on how a light sensor can be used with spatially adjacent viewpoints from two other light sensors for clarity. However, as described below, these techniques can be applied to any number of adjacent light sensor viewpoints.

[0242] In this example, the optical sensor array may include, for example, Figure 29 The light sensor 2906 is one or more individual light detectors described herein. The light sensor 2906 may be physically located adjacent to two orthogonally adjacent light sensors 2904, 2908 in the light sensor array 2902 of the optical measurement system. Because light sensors 2904, 2906, 2908 are physically adjacent in the light sensor array 2902, the corresponding light sensor viewing angles 2808, 2810, 2012 can also be considered spatially adjacent in the surrounding environment. However, this is not always the case. Some embodiments described below may use a rotated light sensor array, wherein the spatial proximity of the light sensor viewing angles does not necessarily depend on the physical proximity of the corresponding light sensors.

[0243] To calculate or characterize the distance measurement of the optical sensor 2906, some embodiments may rely solely on the photon count reflected from the object 2800 from the optical sensor's viewpoint 2810. Relying on the response of the optical sensor 2906 itself may be sufficient if the resulting histogram and / or distance calculation meets or exceeds a detection threshold. However, in some cases, the resulting histogram and / or distance calculation of the optical sensor 2906 may fall below the detection threshold based on various factors described above (e.g., environmental conditions, distance from the object 2800, ambient noise, etc.). In these cases, the optical measurement system may utilize responses from adjacent optical sensors 2904, 2908 to increase the confidence of the measurement obtained by the optical sensor 2906, such that the resulting peak or calculation meets or exceeds the detection threshold.

[0244] Some embodiments can be configured to combine information from the histogram associated with light sensor 2906 with information from the histograms of neighboring light sensors such as light sensors 2904, 2908. In some embodiments, this process may share data from neighboring light sensors. For example, photon counts from the histograms of light sensors 2904, 2908 can be added to the histogram of light sensor 2906. Since the photons are all reflected from the same object 2800, this can have the effect of enhancing the reflected signal in the histogram of light sensor 2906. It should be noted that if the viewing angles 2808, 2810, 2012 of the light sensors are not spatially adjacent as assumed, then the accumulated photons from light sensors 2904, 2908 may not have a negative impact on the histogram of light sensor 2906, because there will be no reflection at said distance. In practice, this can approximate a spatially matched filter across the histograms of neighboring light sensors. If the two peaks coincide in time, it can be assumed that the optical sensor response is spatially correlated, and the two signals can be combined to generate a larger peak in the histogram.

[0245] To combine information from the histogram, the output of the arithmetic logic circuitry of each of the adjacent optical sensors 2904, 2908 can be sent to the arithmetic logic circuitry of optical sensor 2906. This can accumulate photons received by any of the optical sensors 2904, 2906, 2908 into a single histogram to enhance the returned signal. In some embodiments, this accumulation calculation can be amplified to use a weighted sum, which applies weights to the responses of the various optical sensors. Using weights allows for higher accuracy in determining how many adjacent optical sensors will affect the response of another optical sensor. For example, since orthogonally adjacent adjacent optical sensors may have a viewpoint closer to the center optical sensor's viewpoint than diagonally adjacent optical sensors, these orthogonally adjacent optical sensors may have a higher applied weight than diagonally adjacent optical sensors when the photon counts of these orthogonally adjacent optical sensors are added to the histogram of the center optical sensor. Using weights also allows the optical measurement system to use responses of optical sensors that are not directly adjacent. As described above, object 2800 may contain more than Figure 28 The nine optical sensor perspectives described herein represent more than just the nine optical sensor perspectives. These additional optical sensor perspectives, which may not necessarily be adjacent to the central optical sensor, can also be considered. For example, photon counts from these non-neighboring optical sensors can be added to the histogram of the central optical sensor with weights less than those assigned to the responses of the directly neighboring optical sensors.

[0246] B. Combined Histogram / Peak Figure 30The histograms according to some embodiments can be combined for use with proximity optical sensors. In this example, the histogram of each of the at least nine proximity optical sensors in an array is illustrated using the optical sensor viewpoint on object 2800. It should be noted that the actual histograms may be stored in a memory block representing the histogram memory in the optical measurement system. They are displayed in the optical sensor viewpoint on object 2800 for illustrative purposes only.

[0247] Instead of simply generating a weighted sum of the ambient light sensor responses, some embodiments may use a more complex method that combines information from the histograms of the ambient light sensor views into the histogram of the central light sensor view. In this example, the light sensor view 2810 may be derived from, for example... Figure 30 The diagram illustrates eight orthogonally / diagonally adjacent optical sensor viewpoints. Because these optical sensor viewpoints are spatially adjacent and receive reflected photons from the same object 2800, each of the histograms associated with these viewpoints may contain a peak corresponding to the distance between the object 2800 and the optical measurement system. These peaks are... Figure 30 The peaks are graphically represented in each corresponding optical sensor viewpoint. It should be noted that some peaks are stronger than others, with the peak in the center optical sensor viewpoint 2810 being small enough that it does not pass the detection threshold.

[0248] Some embodiments may combine histogram information from surrounding light sensors to improve detection by the central light sensor. The information from the histograms may be specific peaks in a histogram memory. For example, when peaks are located at the same distance among multiple neighboring light sensors, the system may determine that the corresponding light sensors are spatially correlated and that the associated histograms are combined. Some embodiments may combine the entire histogram for each light sensor, while other embodiments may execute peak detection circuitry / algorithms and then combine only specific peak locations. Therefore, the information combined from the histograms may include the histogram values ​​themselves, portions of the histogram representing peaks, and / or additional values ​​derived from the histograms.

[0249] Instead of simply generating a weighted sum of surrounding histograms, some embodiments can use a Gaussian combination of neighboring histograms combined with the convolution process. For example, each of the histograms spatially adjacent to the central light sensor's viewpoint 2810 can have a Gaussian function applied before combination. Instead of summing the resulting histogram values ​​together, a matched filter can be convolved with them in the same way as the individual histogram convolutions. As described above, this approximates the spatial convolution between spatially adjacent light sensor responses.

[0250] XIII. Optical Sensor Array Configuration The examples described above use a rectangular light sensor array pattern with light sensors arranged in a grid layout. However, not all embodiments are limited to rectangular grids. Other embodiments may use different light sensor array patterns that can result in higher light sensor density. Other patterns may also be more suitable for rotating light sensor arrays. In any of these layout patterns, the techniques described above can be used to combine data from spatially adjacent light sensor viewpoints.

[0251] A. Solid-state array configuration Figure 31 An example of a rectangular optical sensor layout according to some embodiments is described. This rectangular optical sensor layout is similar to the rectangular grid pattern of the optical sensor described in the above examples. The optical sensor array 3100 can be implemented as a solid-state array having optical sensors 3102, 3104, etc., fixed relative to the rest of the optical measurement system. During measurement, each of the optical sensors in the optical sensor array 3100 can be scanned by a processor.

[0252] To use the spatial correlation algorithm described above with the rectangular light sensor array 3100, it can be assumed that the histograms of neighboring light sensors are spatially adjacent or spatially correlated. For example, light sensors 3102 and 3104 are physically adjacent in the light sensor array 3100. Therefore, the corresponding light sensor viewing angles 3106 and 3108 in the surrounding environment are also likely to be adjacent.

[0253] B. Rotating Array Configuration Figure 32 The following describes the configuration of a rotating optical measurement system according to some embodiments. This optical measurement system may include one or more light sensor arrays 3204 physically coupled to a rotating element 3200. When the optical measurement system is in operation, the rotating element 200 can rotate, and the light sensor arrays 3204 can continuously scan the surrounding area as they rotate about a central axis.

[0254] Compared to Figure 31In a non-rotating optical sensor array, the physical proximity of the optical sensors in optical sensor array 3204 does not necessarily imply that the viewing angles of the optical sensors are spatially proximate in the surrounding environment. For example, when the optical measurement system rotates, optical sensor 3202 may perform optical measurements at a certain location. During subsequent optical measurements, optical sensor 3202 will be at a different rotation angle than it was during the previous optical measurements, and may therefore see different objects and different areas in the surrounding environment. Therefore, as optical sensor 3202 rotates, the histogram of a previous measurement for optical sensor 3202 may be "spatially proximate" to the histogram of a subsequent measurement for the same optical sensor 3202 during subsequent time intervals. Therefore, some embodiments may buffer previous histograms on-chip to use as spatially proximate histograms for the methods described above.

[0255] Figure 32 3204 optical sensor array and Figure 31 Another difference between the optical sensor arrays is the pattern in which the optical sensors are arranged in optical sensor array 3204. Instead of arranging them in a rectangular grid, the columns of optical sensors in optical sensor array 3204 are slightly offset from each other. This allows the optical sensors in array 3204 to be positioned closer to each other, thereby increasing the optical sensor density. In addition, by offsetting the optical sensors by less than the width of the optical sensors, the spatial resolution of optical sensor array 3204 can be increased.

[0256] This alternative arrangement of the optical sensors in array 3204 can influence which optical sensors are considered spatially adjacent to other optical sensors in array 3204. Physical proximity does not necessarily imply spatial proximity from the perspective of optical sensors in the surrounding environment. When the optical measurement system rotates, optical sensors spatially adjacent to optical sensor 3202 may include optical sensors on other optical sensor arrays on other sides of the rotating element 3200 (not shown), as well as histograms from previously buffered and not overwritten measurements. Spatial proximity can accommodate any geometric arrangement of optical sensors in the array configuration. Instead of relying on physical proximity, embodiments can actually determine the viewing angles of adjacent optical sensors in the surrounding environment during multiple measurements used to perform the operations described above.

[0257] XIV. Circuit Implementation Scheme for Combining Pixels Figure 33The following describes circuitry according to some embodiments for combining information from spatially adjacent histograms. As described in detail above, each optical sensor 3202 may be coupled to arithmetic logic circuitry 3304 and histogram memory 3308. The histogram memory 3308 for each optical sensor 3302 may also be referred to as a memory block. Each memory block 3308 may store photon counts that accumulate during measurement to form a histogram 3306 for photon counting. This combination of optical sensor 3302, arithmetic logic circuitry 3304, and memory block 3308 may be referred to as a "pixel". Each pixel may operate independently to perform optical measurements such that each pixel can receive its own start signal and accumulate photon counts independently of timing signals used to control other pixels.

[0258] To combine information from the histograms as described above, the optical measurement system may include a summation / convolution circuit 3310 that uses mathematical operations to combine the histograms together. The summation / convolution circuit 3310 may be configured to scan individual time intervals for several spatially adjacent optical sensors or to receive said individual time intervals from each of memory blocks 3308. Each time interval from the multiple histograms may be combined together into a time interval in a single histogram 3312 representing a single optical sensor, such as optical sensor 3302b. This circuit may use an arithmetic logic unit to add the values ​​together. When the histograms 3306 are combined together, this circuit may also use a multiplier to apply weights to each of the histograms 3306. The final histogram 3312 may pass through a peak detection circuit 3314 as described above to locate peaks in the histogram. Because the histograms 3306 have been combined from spatially adjacent optical sensors, the resulting histogram 3312 is more likely to provide one or more peaks exceeding a detection threshold.

[0259] Any identified peaks can be sent to processor 3316 for distance calculation as described above. Processor 3316 may be implemented on integrated circuit 3320, which is physically separate from and different from integrated circuit 3322 on which arithmetic logic circuit 3304 and / or memory block 3308 are implemented.

[0260] exist Figure 33 In this embodiment, multiple elements can be combined to form a circuit configured to combine information from a first histogram with information from one or more spatially neighboring histograms to generate a distance measurement from the optical sensor. In this embodiment, the information from the histograms may include data values ​​from the histograms themselves, and the circuit may include a summing / convolution circuit 3310. In some embodiments, the circuit may also include a processor 3316 that actually generates the distance measurement.

[0261] Figure 34This describes an alternative circuit for combining information from a histogram, according to some embodiments. This circuit is similar to… Figure 33 The circuitry differs in that, in this case, peak values ​​can be detected by peak detection circuitry 3411 for each individual histogram 3306 before their combination. Instead of combining all histograms, this instance may combine only the identified peak values ​​from the histograms. Some embodiments may also determine whether the peak values ​​identified in each of histograms 3306 are spatially correlated before combining the information from the histograms. For example, if several spatially adjacent histograms indicate peak values ​​at similar locations, then it can be determined that the peak values ​​are generated by the same object in the surrounding environment and the histograms can be considered spatially correlated at their locations. This can also be used to identify outliers. For example, if eight of the nine adjacent histograms indicate peak values ​​at a certain location, then the ninth histogram can be excluded from the calculation as an outlier.

[0262] exist Figure 34 In some instances, the combination of spatially adjacent histograms can be performed for the center pixel, such as the pixel of histogram 3306b. In some embodiments, the histogram combined with histogram 3306b can be multiplied by a weighting factor before it is combined with the center pixel.

[0263] In this example, the circuitry for combining information from the histogram can be implemented in a processor within a second integrated circuit 3420, distinct from the integrated circuit 3422 in which peaks are detected. As described in detail above, peaks can be detected on-chip, and a time interval window can be transmitted to the processor for distance measurement. The processor can also execute a summation / convolution circuit 3425 to combine individual peaks to form a single peak 3426 representation for the light sensor 3302b. This single peak 3426 may exceed a detection threshold and can be used by a distance calculation algorithm 3427 on the processor.

[0264] Figure 35Another circuit, according to some embodiments, is described for combining information from a histogram. In this example, the processor can calculate the distance to each light sensor based on its individual histogram without any combination of histogram information. After calculating the distance, information from each pixel can be combined via a summation / convolution process 3504 on the processor. This operation is significantly simpler than combining information from the original histogram and can be performed relatively faster than the example described above. For example, if distances have been calculated with respect to several neighboring pixels, these distances can be compared to each other. Some embodiments may average these distances of the surrounding pixels to determine the distance to the center pixel. This improves the accuracy of each distance measurement for each pixel because it is based on measurements of several different pixels rather than just a single pixel. Some embodiments may also use surrounding pixels to calculate confidence values. If the surrounding pixels contain distance measurements similar to those of the center pixel, then the confidence value of the center pixel can be relatively high.

[0265] XV. A method for combining spatially adjacent data Figure 36 A flowchart illustrating a method for using spatially neighboring pixel information in an optical measurement system is provided. The following method describes the process for a single optical sensor combined with one or more spatially neighboring optical sensors in an optical measurement system. Specifically, information from one or more spatially neighboring optical sensors can be combined with histogram information from a single optical sensor to calculate a distance measurement. However, as described above, an optical measurement system may contain many optical sensors or "pixels," and this method can be performed for each optical sensor in the optical measurement system. Therefore, this method can be performed multiple times for each optical sensor, once when considering a single optical sensor where histogram information is combined, and multiple times depending on the spatially neighboring optical sensors of other optical sensors in the array.

[0266] At step 3602, the method may include transmitting one or more pulse trains within one or more first time intervals as part of an optical measurement. Each of the one or more time intervals may represent an "excitation" repeated multiple times in the measurement. Each of the time intervals may include one or more pulse trains encoded and transmitted by a light source, such that the pulse trains can be considered as being reflected from an object in the surrounding environment. Each of the time intervals may be subdivided into multiple time intervals, such that each time interval represents an interval in a histogram of photon counts received during the optical measurement. (The above is in contrast to...) Figure 9 Describe instances of how a single measurement can contain multiple excitations subdivided into time intervals for clustered photon counting.

[0267] At step 3604, the method may include detecting reflected photons from one or more pulse trains. These reflected photons may be detected using multiple optical sensors. The multiple optical sensors may include a first optical sensor and one or more optical sensors spatially adjacent to the first optical sensor. As described above, an optical sensor may be “spatially adjacent” to the first optical sensor when the viewing angles of two optical sensors are close in the environment surrounding the optical measurement system. In some configurations, this may include optical sensors that are physically adjacent to the optical measurement system, while other configurations (e.g., rotational configurations) do not require physical proximity. As described in detail above, each optical sensor may include multiple photodetectors, such as multiple SPADs. The optical sensors may receive reflected light as well as ambient background noise received from the surrounding environment. The reflected light received by the optical sensors may include light reflected from objects of interest in the surrounding environment. For example, these objects of interest may be at least 30 cm away from the optical sensor and may represent surrounding vehicles, buildings, pedestrians, and / or any other objects that may be encountered near the optical measurement system during use. These objects of interest may be distinguished from objects that are part of the optical measurement system itself, such as a housing. The optical sensor can also be coupled to a threshold detection circuit and / or to an arithmetic logic circuit for cumulative photon counting. Figure 5 This describes an example of how photon counts can be received from a light sensor and counted using threshold circuitry and a pixel counter (e.g., arithmetic logic circuitry).

[0268] At step 3606, the method may include accumulating a count of photons received during one or more time intervals. The photons may be accumulated using arithmetic logic circuitry and may be accumulated into one or more memory blocks of a histogram memory as described above. The memory blocks may be implemented using registers in the SRAM of the histogram data path, as described above. Figures 8 to 9 As described herein. Each of the time intervals can be subdivided into multiple first time intervals for the histogram. The corresponding time interval in each of one or more time intervals can be accumulated in a single register in a memory block. Each of the time intervals can represent an excitation, and one or more time intervals together can represent a measurement for an optical measurement system. Each of the time intervals can be defined by a start signal or excitation signal that resets the memory representing the histogram back to the first register in the histogram.

[0269] At step 3608, the method may include combining information from a first histogram with information from one or more histograms to generate a distance measurement for the optical sensor. As described above, the information from the first histogram may include raw data from the histogram itself, identified peaks in the histogram, calculations or statistics derived from the histogram, distance measurements calculated based on the histogram, and / or any other information that can be calculated or derived from photon counts in the histogram. Similarly, information from one or more histograms spatially adjacent to the optical sensor may also include any of these types of information. Combining the information may include recalling histogram information, applying numerical weights to histogram information, convolving histogram information, applying a Gaussian function to histogram information, and / or any other mathematical operations that can combine information from different histograms. Combining may result in a new histogram, new peaks in the histogram, a new distance measurement, and / or another value. The circuitry used to perform this combination and / or generate distance measurements from the first optical sensor may include on-chip circuitry for summing / convolving histogram information, as well as all or part of a separate processor configured to calculate the distance measurements.

[0270] It should be understood that Figure 36 The specific steps described herein provide a specific method for using spatially neighboring pixel information in an optical measurement system according to various embodiments. Other sequences of steps may also be performed according to alternative embodiments. For example, alternative embodiments of the invention may perform the steps outlined above in a different order. Furthermore, Figure 36 The individual steps described may contain multiple sub-steps that can be performed in various sequences suitable for the individual steps. Furthermore, additional steps may be added or removed depending on the specific application. Those skilled in the art should recognize the many variations, modifications, and alternatives.

[0271] XVI. Additional Examples While some embodiments disclosed herein focus on the application of optical ranging in 3D sensing scenarios for automotive use, the systems disclosed herein can be used in any application without departing from the scope of this disclosure. For example, the system may have a small or even miniature form factor, enabling several additional use cases, such as for solid-state optical ranging systems. For example, the system can be used in devices such as mobile phones, tablet PCs, laptop computers, desktop PCs, or other peripheral devices and / or user interface devices. For example, one or more embodiments can be employed in mobile devices to support facial recognition and face tracking capabilities, eye tracking capabilities, and / or 3D scanning of objects. Other use cases include forward-facing depth cameras for augmented and virtual reality applications in mobile devices.

[0272] Other applications involve deploying one or more systems on airborne vehicles such as airplanes, helicopters, drones, and the like. Such instances can provide 3D sensing and depth imaging to aid navigation (autonomous or otherwise) and / or generate 3D maps for later analysis, such as supporting geophysical, architectural, and / or archaeological analyses.

[0273] The system can also be installed on fixed objects and structures such as buildings, walls, poles, bridges, scaffolding, and the like. In such cases, the system can be used to monitor outdoor areas such as manufacturing facilities, assembly lines, industrial facilities, construction sites, excavation sites, roads, railways, bridges, etc. Furthermore, the system can be installed indoors to monitor the movement of people and / or objects within buildings, such as the movement of inventory in a warehouse or the movement of people, luggage, or goods within office buildings, airports, train stations, etc. As will be understood by those skilled in the art to which this disclosure pertains, many different applications of optical ranging systems are possible, and therefore, the examples provided herein are for illustrative purposes only and should not be construed as limiting the use of such systems to the explicitly disclosed examples.

[0274] XVII. Computer System Any computer system or circuit mentioned herein may utilize any suitable number of subsystems. Subsystems may be connected via system bus 75. As examples, a subsystem may include input / output (I / O) devices, system memory, storage devices, and network adapters (e.g., Ethernet, Wi-Fi, etc.) for connecting other devices in the computer system (e.g., engine control unit). System memory and / or storage devices may embody computer-readable media.

[0275] A computer system may include multiple identical components or subsystems connected together, for example, through an external interface, an internal interface, or via a removable storage device that can be connected to and removed from one component to another. In some embodiments, the computer system, subsystem, or device may communicate via a network.

[0276] Aspects of the embodiments may be implemented using hardware circuitry (e.g., application-specific integrated circuits or field-programmable gate arrays) and / or in a modular or integrated manner using computer software in the form of control logic via a generally programmable processor. As used herein, a processor may comprise a single-core processor, a multi-core processor on the same integrated chip, or multiple processing units on a single circuit board or networked, as well as dedicated hardware. Based on this disclosure and the teachings provided herein, those skilled in the art will recognize and understand other ways and / or methods for implementing embodiments of the invention using hardware and combinations of hardware and software.

[0277] Any software component or function described in this application may be implemented as processor-executable software code using any suitable computer language such as Java, C, C++, C#, Objective-C, Swift, or a scripting language such as Perl or Python, employing techniques such as conventional or object-oriented methods. The software code may be stored as a series of instructions or commands on a computer-readable medium for storage and / or transmission. Suitable non-transitory computer-readable media may include random access memory (RAM), read-only memory (ROM), magnetic media such as hard disk drives or floppy disks, or optical media such as optical discs (CDs) or digital versatile optical discs (DVDs), flash memory, and the like. The computer-readable medium may be any combination of such storage or transmission means.

[0278] Such programs can also be encoded and transmitted using carrier signals suitable for transmission via wired, optical, and / or wireless networks conforming to various protocols, including the Internet. Therefore, computer-readable media can be generated using data signals encoded with such programs. Computer-readable media encoded with program code can be packaged with compatible devices or provided separately from other devices (e.g., downloaded via the Internet). Any such computer-readable media can reside on or within a single computer product (e.g., a hard drive, CD, or an entire computer system) and can reside on or within different computer products within a system or network. The computer system may include a monitor, printer, or other suitable display for providing a user with any of the results mentioned herein.

[0279] Any method described herein can be performed wholly or partially by a computer system comprising one or more processors configured to perform the steps. Therefore, embodiments may relate to a computer system configured to perform the steps of any method described herein, the computer system possibly having different components performing the respective steps or groups of steps. Although presented as numbered steps, the steps of the methods herein may be performed simultaneously or at different times or in different orders. Furthermore, portions of these steps may be used in conjunction with portions of other steps of other methods. Moreover, all or part of the steps may be optional. Additionally, any step in any method may be performed using modules, units, circuits, or other components of a system for performing those steps.

[0280] Specific details of particular embodiments may be combined in any suitable manner without departing from the spirit and scope of the embodiments of the invention. However, other embodiments of the invention may be specific embodiments relating to each other aspect or a particular combination of these individual aspects.

[0281] The foregoing description of exemplary embodiments of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms described; many modifications and variations are possible in accordance with the teachings above.

[0282] Unless explicitly indicated otherwise, the use of “a / an” or “the” is intended to mean “one or more”. Unless explicitly indicated otherwise, the use of “or” is intended to mean “inclusive or” rather than “exclusive or”. A reference to the “first” component does not necessarily require the provision of the second component. Furthermore, unless explicitly stated otherwise, mentioning the “first” or “second” component does not limit the referenced component to a specific location. The term “based on” is intended to mean “at least partially based on”.

[0283] All patents, patent applications, publications, and descriptions mentioned herein are incorporated herein by reference in their entirety for all purposes. This is not an admission that they are prior art.

Claims

1. An optical measurement system, comprising: Multiple light sources are configured to emit one or more pulse trains at one or more time intervals as part of optical measurements; A plurality of optical sensors are configured to detect photons reflected from one or more pulse trains emitted from corresponding light sources of the plurality of light sources, wherein the plurality of optical sensors includes a first optical sensor and one or more other optical sensors spatially adjacent to the first optical sensor; Multiple memory blocks are configured to: accumulate photon counts of photons received during the one or more time intervals through corresponding optical sensors among the multiple optical sensors, to represent multiple histograms of photon counts, wherein the multiple histograms include a first histogram corresponding to the first optical sensor and one or more histograms corresponding to the one or more other optical sensors; as well as The circuit is configured to combine information from the first histogram with information from one or more other histograms to generate a distance measurement for the first optical sensor.

2. The optical measurement system of claim 1, wherein the one or more optical sensors are physically adjacent to the first optical sensor in the optical sensor array.

3. The optical measurement system according to claim 2, wherein the optical sensor array comprises a solid-state array of optical sensors.

4. The optical measurement system of claim 2, wherein the one or more optical sensors comprise eight optical sensors orthogonally adjacent to or diagonally adjacent to the first optical sensor.

5. The optical measurement system of claim 1, wherein the one or more optical sensors are not physically adjacent to the first optical sensor in the optical sensor array, but wherein the one or more optical sensors are positioned to receive photons from a physical region adjacent to a physical region from which the first optical sensor receives photons.

6. The optical measurement system of claim 5, wherein the plurality of optical sensors are arranged as an array of optical sensors rotating about the central axis of the optical measurement system.

7. The optical measurement system according to claim 1, wherein: The information from the first histogram includes a first distance measurement calculated based on the first histogram; The information from the one or more histograms includes one or more other distance measurements calculated based on the one or more other histograms; and The distance measurement value includes a combination of the first distance measurement value and one or more other distance measurements.

8. The optical measurement system of claim 7, wherein the first distance measurement value is below the detection limit of the optical measurement system before being combined with the plurality of other distance measurements.

9. The optical measurement system of claim 8, wherein after combining the first distance measurement value with the plurality of other distance measurements, the distance measurement value is higher than the detection limit of the optical measurement system.

10. The optical measurement system of claim 8, wherein the detection limit represents the minimum number of photons received by the corresponding optical sensor.

11. The optical measurement system according to claim 1, wherein, The circuitry for combining the information from the first histogram with the information from the one or more histograms includes a processor implemented on an integrated circuit, which is different from the integrated circuit on which the plurality of memory blocks are implemented.

12. The optical measurement system of claim 1, wherein the circuitry and the plurality of memory blocks are implemented on the same integrated circuit.

13. A method for using spatially neighboring pixel information in an optical measurement system, the method comprising: As part of optical measurement, one or more pulse trains are transmitted in one or more first time intervals; Multiple optical sensors are used to detect reflected photons from the one or more pulse trains, wherein the multiple optical sensors include a first optical sensor and one or more optical sensors spatially adjacent to the first optical sensor; The photon counts received during the one or more time intervals are accumulated using the plurality of optical sensors to represent a plurality of histograms of the photon counts, wherein the plurality of histograms includes a first histogram corresponding to the first optical sensor and one or more histograms corresponding to the one or more optical sensors; as well as Information from the first histogram is combined with information from one or more histograms to generate a distance measurement for the first optical sensor.

14. The method of claim 13, wherein the reflected photons received by the first optical sensor and by the one or more optical sensors are reflected from the same object in the surrounding environment.

15. The method according to claim 13, wherein: The information from the first histogram includes the photon count in the first histogram; The information from the one or more histograms includes photon counts from the one or more histograms; and The distance measurement is calculated based on the aggregation of the photon count in the first histogram and the photon count in one or more histograms.

16. The method of claim 13, wherein: The information from the first histogram includes one or more peaks in the first histogram; The information from the one or more histograms includes a second or more peaks in the one or more histograms; and The distance measurement is calculated based on a combination of the first or more peak values ​​and the second or more peak values.

17. The method of claim 16, wherein the distance measurement is calculated based on the sum of the first one or more peaks and the second one or more peaks.

18. The method of claim 16, wherein the distance measurement is calculated based on a Gaussian combination of the first or more peaks and the second or more peaks.

19. The method of claim 16, wherein the distance measurement is calculated based on the convolution of the first one or more peaks and the second one or more peaks.

20. The method of claim 16, wherein the distance measurement is calculated based on a weighted combination of the first or more peaks and the second or more peaks.

21. An optical measurement system, comprising: The light source is configured to transmit one or more pulse trains in one or more time intervals as part of optical measurement, each of the one or more time intervals containing one of the one or more pulse trains; A light sensor is configured to detect photons from one or more pulse trains reflected from an object in the environment surrounding the optical measurement system; Multiple registers are configured to accumulate photon counts received from the optical sensor during the one or more time intervals to represent an unfiltered histogram of photon counts received during the one or more time intervals; The filtering circuit is configured to: provide a filtered histogram of the photon counts from the plurality of registers; and The peak detection circuit is configured as follows: Detect the position of the peak value in the filtered histogram, and Using the position of the peak in the filtered histogram, the position in the plurality of registers storing the unfiltered representation of the peak is identified.

22. The optical measurement system of claim 21, further comprising a processor configured to: receive the unfiltered representation of the peak and use the unfiltered representation of the peak to calculate the distance to the object in the environment surrounding the optical measurement system.

23. The optical measurement system of claim 21, wherein the filtering circuit is configured to provide the filtered histogram by applying a matched filter corresponding to the one or more pulse trains.

24. The optical measurement system of claim 21, wherein the pulse train in the one or more pulse trains comprises a plurality of rectangular pulses.

25. The optical measurement system of claim 21, wherein the filtering circuit is configured to perform low-pass filtering on the unfiltered histogram.

26. The optical measurement system according to claim 21 further includes a second plurality of registers, the second plurality of registers storing the filtered histogram.

27. The optical measurement system of claim 21, wherein the filtered histogram is generated in a single pass through the plurality of registers.

28. The optical measurement system of claim 27, wherein the peak value is detected by the plurality of registers during the single pass, such that not all of the filtered histogram is stored.

29. The optical measurement system of claim 21, wherein the peak detection circuit is configured to detect the position of the peak by detecting an increase in value following a decrease in the plurality of registers.

30. The optical measurement system of claim 21, wherein the processor is implemented on an integrated circuit (IC) that is separate from and different from the plurality of registers implemented thereon.

31. The optical measurement system of claim 21, wherein the light source and the light sensor form a pixel among a plurality of pixels in the optical measurement system.

32. A method for analyzing filtered and unfiltered data in an optical measurement system, the method comprising: As part of the optical measurement, one or more pulse trains are transmitted in one or more first time intervals, each of the one or more first time intervals containing one of the one or more pulse trains; Detecting photons from one or more pulse trains reflected from an object in the environment surrounding the optical measurement system; The photons are used to populate multiple registers to represent an unfiltered histogram of the photon counts received during one or more first time intervals; The unfiltered histograms in the plurality of registers are filtered to provide a filtered histogram of the photon from the plurality of registers; Detect the position of the peak value in the filtered histogram; The location of the peak in the filtered histogram is used to identify the location in the plurality of registers storing the unfiltered representation of the peak; as well as The unfiltered representation of the peak value is sent to the processor to calculate the distance to the object in the environment surrounding the optical measurement system using the unfiltered representation of the peak value.

33. The method according to claim 32, wherein, Sending the unfiltered representation of the peak includes sending information about the identification histogram time interval represented in the plurality of registers storing the unfiltered representation of the peak.

34. The method according to claim 32, wherein, Filtering the unfiltered histogram in the plurality of registers includes: applying a convolution between the unfiltered histogram and at least one rectangular filter having a plurality of identical values.

35. The method according to claim 34, wherein, The multiple identical values ​​include a series of binary "1" values ​​and / or a series of "-1" values.

36. The method according to claim 32, wherein, Filtering the unfiltered histogram in the plurality of registers includes: convolving the unfiltered histogram with at least one sequence, the at least one sequence including non-zero values ​​followed by a plurality of zero values.

37. The method of claim 36, wherein the at least one sequence comprises a single binary "1" value or a single binary "-1", the single binary "1" value or the single binary "-1" being followed by a plurality of "0" values.

38. The method of claim 32, further comprising: In addition to sending the unfiltered representation of the peak value, a filtered representation of the peak value is sent to the processor.

39. The method of claim 32, wherein: The position of the peak in the filtered histogram is detected as a single peak in the filtered histogram; and The unfiltered representation of the peak value includes at least two peak values ​​in the unfiltered histogram.

40. The method according to claim 39, wherein, One of the at least two peaks in the unfiltered histogram represents a peak caused by reflection of the one or more pulse trains from the housing or window of the optical measurement system.

41. The method of claim 32, wherein the photon is detected using a plurality of photodetectors in a photosensor.

Images