Timing Jitter in LIDAR Systems

Abstract

A LIDAR system with a light emitter and a light detector can apply single shot jitter to produce variations in the interval between consecutive emitter pulses. The operation of the detector can be synchronized with the operation of the emitter so that consistent time-of-flight measurements correspond to consistent distances. Applying single shot jitter can reduce the effects of crosstalk from other pulsed light sources and can also reduce range aliasing effects.

Description

Background Art

[0001] The present disclosure relates generally to light imaging, detection, and ranging (LIDAR) systems, and more particularly to using temporal random dithering to reduce crosstalk and range aliasing in LIDAR systems.

[0002] LIDAR systems measure the distance to a target by illuminating the target with a pulsed laser and measuring the reflected pulses with a sensor. The time-of-flight measurement can then be used to digitally represent the target in three dimensions (3D). LIDAR systems can be used in a variety of applications that require the use of 3D depth images, including archeology, geography, geology, forestry, mapping, construction, medical imaging, security and military applications, and other applications. As just one example, autonomous vehicles can use LIDAR for obstacle detection and avoidance and vehicle navigation. As another example, a security system can use LIDAR to detect the 3D position and movement of possible intruders in a restricted area. Summary of the invention

[0003] Certain embodiments of the invention described herein relate to LIDAR systems in which per-shot jitter is applied to produce variations in the intervals between successive transmitter pulses. The operation of the detector is synchronized with the operation of the transmitter so that consistent time-of-flight measurements correspond to consistent distances regardless of the jitter. Applying per-shot jitter can reduce the effects of crosstalk from other pulsed light sources and can also reduce range aliasing effects. Reducing the effects of crosstalk and / or range aliasing can improve the accuracy of ranging operations.

[0004] In some embodiments, the LIDAR system may include an optical transmission module, an optical sensing module, and a timing controller coupled to the optical transmission module and the optical sensing module. The optical transmission module may be a transmitter to emit an optical pulse during each of multiple shots in response to a trigger pulse. The optical sensing module may have at least one optical sensor channel to receive light, and may be configured to generate a histogram of the intensity of the received light as a function of time for a sequence of time bins in response to the trigger pulse. The optical sensing module may also have a first memory circuit configured to accumulate histograms of multiple shots. The timing controller may be configured to generate a sequence of trigger pulses to synchronize the operation of the optical transmission module and the optical sensing module for each shot, and may also be configured to apply a single shot jitter to each trigger pulse. The single shot jitter may vary for different shots, and for at least some of the multiple shots, the single shot jitter may be greater than one time bin.

[0005] The single shot jitter may be selected in various ways, for example, using an equal energy sampling technique. The jitter value may be selected within a predefined range (e.g., up to 20%, 50%, 100%, 120%, or some other fraction of the total number of time bins in the histogram). In some embodiments, the second memory circuit may store a master jitter sequence comprising a sequence of jitter values ​​uniformly distributed within the jitter value range, and the timing controller may be further configured to select the single shot jitter for each trigger pulse according to the master jitter sequence. In some embodiments, the control logic may be used to randomly select a starting position in the master jitter sequence during a system startup operation.

[0006] In some embodiments, the LIDAR system may further include a processor configured to analyze the histogram to determine if crosstalk is present and adaptively modify the single shot jitter in response to determining the presence of crosstalk. In some embodiments, other operating behaviors may also be adaptively modified, such as using pulse train introduction or changing the pulse encoding scheme. In some embodiments, the modification may be selectively applied in directions associated with crosstalk.

[0007] In some embodiments, the same single shot jitter may be applied to all emitters and sensor channels of the LIDAR array. In other embodiments, different single shot jitters may be applied to different emitters and sensor channels.

[0008] In some embodiments, a method of operating a LIDAR system may include capturing multiple shots and subsequently capturing a shot, calculating ranging information based at least in part on a histogram. Capturing each shot may include determining a single shot jitter; generating a trigger pulse in a timing controller of the LIDAR system at a time determined at least in part based on the single shot jitter; operating an optical transmission module of the LIDAR system to transmit a light pulse synchronized with the trigger pulse; and operating an optical sensing module of the LIDAR system in synchronization with the trigger pulse to receive light and accumulate a histogram of received light intensity as a function of time, the histogram defining a series of time intervals. The histogram may be accumulated over all shots, and for at least some shots, the single shot jitter may be greater than one time interval.

[0009] The following detailed description will provide a better understanding of the nature and advantages of the claimed invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1A and 1B An example of an automotive application of a LIDAR system that may be used in conjunction with some embodiments described herein is shown.

[0011] Figure 2 A block diagram of an exemplary LIDAR system is shown in accordance with some embodiments.

[0012] Figure 3 A simplified side view of components of an optical ranging device is shown according to some embodiments.

[0013] Figure 4 An example of the operation of a LIDAR channel of a single "shot" LIDAR system as a function of time is shown.

[0014] Figures 5A-5D An example of a histogram accumulating photon counts per time bin in the presence of inter-system crosstalk is shown.

[0015] Figure 6 Per-shot jitter according to some embodiments of the present invention is illustrated.

[0016] Figures 7A-7D An example of a histogram accumulating photon counts per time bin with single shot jitter is shown according to some embodiments of the present invention.

[0017] Figure 8 A graphical representation of an example master dithering sequence according to some embodiments of the present invention is shown.

[0018] Fig. 9 A flow chart of a process for operating a LIDAR channel according to some embodiments of the invention is shown.

[0019] Fig.10 A flow chart showing a process that may be used in some embodiments to infer the presence of crosstalk sources from histogram data.

[0020] Fig.11 A flow chart showing a process that may be used in some embodiments to infer the presence of crosstalk sources from histogram data collected using background patterns.

[0021] Fig.12 is a flow chart of a process for detecting crosstalk in a LIDAR system and adaptively modifying the operation of the LIDAR system to reduce the effects of the crosstalk, according to some embodiments.

[0022] Fig.13 A timeline view illustrating range aliasing that may occur in the absence of jitter is shown.

[0023] Fig.14 A timeline view illustrating the removal of range aliasing by applying single-shot dithering according to some embodiments of the present invention is shown. DETAILED DESCRIPTION

[0024] 1. LIDAR System Overview

[0025] Some embodiments described herein relate to techniques for reducing crosstalk in a LIDAR system that emits light pulses at specified times and uses the time between emitting the pulses and detecting reflected light from the pulses to generate ranging data. To facilitate understanding of the disclosed techniques, an example of a LIDAR system and operation will be described. It should be understood that embodiments of the claimed invention can be implemented in many different LIDAR systems and are not limited to the specifically described example systems.

[0026] 1.1 Example LIDAR System

[0027] Figure 1A and 1B An example of an automotive application of a LIDAR system that can be used in conjunction with some embodiments described herein is shown. The automotive application is chosen here only for illustration purposes, and the sensors described herein can be employed in other types of vehicles such as ships, airplanes, trains, and in a variety of other applications where 3D depth images are useful, such as medical imaging, geodesy, mapping, archeology, geography, geology, geomorphology, seismology, forestry, atmospheric physics, security systems, laser guidance, airborne laser swath mapping (ALSM), and laser altimetry.

[0028] Figure 1A An example of an angle scanning (e.g., rotational or spin) LIDAR system 100 in an automotive application is shown. According to some embodiments, the scanning LIDAR system 100 may be mounted on the roof of a vehicle 105, as shown. In other embodiments, one or more LIDAR and / or imaging sensors may be mounted at other locations on the vehicle, including but not limited to the front or rear of the vehicle, the side of the vehicle, and / or the corner of the vehicle.

[0029] The scanning LIDAR system 100 may include an optical transmission module (also referred to as an optical Tx module) 102 for transmitting laser pulses and / or an optical reception module (also referred to as an optical Rx module) 104 that may include a sensor array to sense reflected light from the laser pulses emitted by the optical Tx module 102. In some embodiments, the optical Tx module 102 may be disposed in the same housing as the optical Rx module 104.

[0030] The LIDAR system 100 may adopt a scanning architecture, in which the direction of the optical Tx module 102 and the optical Rx module 104 may be scanned around one or more fields of view 110 (e.g., 360 degree fields in some embodiments) within an external field or scene outside the vehicle 105. In the case of a scanning architecture, the emitted light 112 may be pulsed as the LIDAR system 100 scans the surrounding environment. For example, the optical Tx module 102 may emit one or more pulsed output beams, and the scanning LIDAR system 100 may be scanned and rotated to illuminate different parts of the scene around the vehicle at different times. In some embodiments, the scanning represented by the rotating arrow 115 may be implemented by mechanical means, for example, by mounting the optical emitter and sensor on a rotating column or platform. In some embodiments, scanning may be implemented by other mechanical means (e.g., by using a galvanometer). Chip-based steering technology may also be used, such as by using a microchip that uses one or more MEMS-based reflectors, such as a digital micromirror (DMD) device, a digital light processing (DLP) device, etc. In some embodiments, scanning may be performed by non-mechanical means, for example, by using electronic signals to control one or more optical phased arrays.

[0031] In operation, when the scanning LIDAR system 100 is in a particular direction, the light Tx module 102 can be operated to generate a pulsed light beam, such as light beam 112, at a known time. Objects within the scene (e.g., object 110) can reflect portions of the light pulse emitted from the light Tx module 102. One or more reflected portions then return to the scanning LIDAR system 100 and can be detected by the light Tx module 104. For example, the light pulse 112 can be reflected by the object 110, producing a reflected portion 114. The reflected portion 114 can be detected by the light Rx module 104. In addition, extraneous light 116 from other sources can also be detected by the light Rx module 104. As described below, the extraneous light 116 may affect the ranging measurements made using the scanning LIDAR system 100. In some embodiments, the LIDAR system 100 can be operated in a manner intended to reduce the impact of the extraneous light 116 on the ranging measurements.

[0032] Figure 1BAn implementation 150 is illustrated in which static LIDAR systems 152a-b are implemented on top of a road vehicle 155. Each static LIDAR system 152a-b can face a different direction (possibly with partially overlapping and / or non-overlapping fields of view between units) in order to capture a composite field of view that is larger than each unit could capture on its own. For example, LIDAR system 152a can capture field of view 157a, and LIDAR system 152b can capture field of view 157b. In some embodiments, the number of static LIDAR systems, the placement of the static LIDAR systems, and the field of view of each static LIDAR system can be selected to obtain most, if not all, of a 360 degree field of view of the vehicle's surroundings.

[0033] As an example, referring to the static LIDAR system 152a, each static LIDAR system may include an optical transmission module (also referred to as an optical Tx module) 154 and an optical receiving module (also referred to as an optical Rx module) 156. The optical Tx module 154 and the optical Rx module 156 may be connected to Figure 1A The optical Tx module 102 and the optical Rx module 104 of the scanning LIDAR system 100 are similar or identical. For example, the optical Tx module 154 can be operated to generate a pulsed light beam, such as the light beam 162, at a known time. Objects within the scene (e.g., object 160) can reflect portions of the light pulse emitted from the optical Tx module 154. One or more reflected portions then return to the static LIDAR system 152a and can be detected by the optical Tx module 156. For example, the light beam 162 can be reflected by the object 160, generating a reflected portion 164. The reflected portion 164 can be detected by the optical Rx module 156. In addition, extraneous light 166 from other sources can also be detected by the optical Rx module 156. As described below, the extraneous light 156 can affect the ranging measurements made using the static LIDAR system 152a.

[0034] In some embodiments, each of the static LIDAR systems 152a, 152b can image its entire field of view (shown as areas 157a and 157b, respectively) at one time. In other embodiments, the static LIDAR systems 152a, 152b can electronically scan the scene to capture an image of the scene. As used herein, "electronic scanning" refers to collecting data for different portions of the scene at different times without physical movement (e.g., reorientation) of the sensor array; thus, electronic scanning is different from the rotational operation described above. For example, electronic scanning can be implemented by activating different portions of the LIDAR emitter array in the optical Tx module 154 and corresponding subsets of the array of LIDAR sensor channels in the optical Rx module 156 at different times, or by other means (such as chip-based beam steering technology), for example, by using a microchip that uses one or more MEMS-based reflectors (such as digital micromirrors (DMDs), digital light processing (DLPs), etc.) to control the light from the Tx module 154 so that the light is reflected at different portions of the sensor array in the optical Rx module 156 at different times. Thus, static LIDAR system 152a may electronically scan between points 160 and 170 to capture objects in the scene at area 157a, and similarly for LIDAR system 152b and area 157b.

[0035] Figure 1A and 1B The system shown is illustrative and can be modified as desired. For example, although Figure 1A and 1B A LIDAR system is shown mounted on the roof of the vehicle, but those skilled in the art will appreciate that a static or scanning LIDAR system may be mounted on other parts of the vehicle in addition to or in lieu of the roof. For example, a static or scanning LIDAR system may be mounted on an exterior area of ​​the vehicle, such as near the front and rear fenders. A vehicle may carry a combination of static and scanning LIDAR systems, such as a scanning LIDAR system on the roof and a static LIDAR system near the front and rear fenders.

[0036] Further, an automotive embodiment for a LIDAR system is selected herein for illustration purposes only, and the system described herein can be used in other types of vehicles (e.g., ships, airplanes, trains, etc.) and in a variety of applications requiring the use of 3D depth images, such as any of the applications mentioned above. It should also be understood that static and scanning LIDAR systems can be used together, and some LIDAR systems can be configured to selectively operate in static or scanning modes.

[0037] Figure 2A block diagram of an exemplary LIDAR system 200 is shown according to some embodiments. The LIDAR system 200 may implement such Figure 1A The LIDAR system 100 is a scanning LIDAR system, or a system such as Figure 1B The LIDAR system 152a or 152b is a static LIDAR system.

[0038] The LIDAR system 200 may include an optical ranging device 202 and a user interface 250. The optical ranging device 202 may include a ranging system controller 204, an optical transmission module 206, and an optical sensing module 208. The optical ranging device 202 may generate ranging data by transmitting one or more light pulses from the optical transmission module 206 to a field of view outside the optical ranging / imaging device 202. The field of view may include one or more light reflecting objects at variable distances from the optical transmission module 206 and the optical sensing module 208. The reflected portion of the transmitted light is detected by the optical sensing module 208 after some delay time. Based on the delay time, the distance to the reflecting surface can be determined. In some embodiments, the delay time and the distance to the reflecting surface can be determined using a histogram technique as described below. Other ranging methods may also be used, such as continuous wave, optical demodulation, Doppler, etc.

[0039] Optical transmission (Tx) module 206 (which may correspond to, for example, Figure 1A The optical Tx module 102 or Figure 1B The optical Tx module 152a or 152b of the embodiment of the present invention includes an emitter array 214, which can be a one-dimensional or two-dimensional array of light emitters (e.g., infrared lasers), and a Tx optical system 216, which when combined with the emitter array 214 can form a light transmission system 238; an example implementation of the light transmission system 238 is described below. The optical Tx module 206 can also include an optional processor 218 and memory 220, although in some embodiments, these computing resources can be incorporated into the ranging system controller 204. In some embodiments, the memory 220 (or the memory 232 of the ranging system controller 204) can store information indicating when a light pulse should be emitted and / or program code executable by the processor 218 (or the processor 230 of the ranging system controller 204) to determine when a light pulse should be transmitted. A specific example for determining when a light pulse should be transmitted is described below.

[0040] The optical sensing (Rx) module 208 (which may correspond to, for example, Figure 1A The optical Rx module 104 or Figure 1BThe optical Rx module 154a or 154b) may include a sensor array 226, which may include a light sensor capable of detecting a reflected portion of the transmitted light from the optical transmission module 206, and an Rx optical system 228 that, when together with the sensor array 226, forms a light detection system 236; an example implementation of the light detection system 236 is described below. The optical Rx module 208 may also include a processor 222 and a memory 224. (In alternative implementations, these computing resources may be incorporated into the ranging system controller 204.)

[0041] In some embodiments, the optical ranging device 202 can be operated in an electronic scanning mode, where a LIDAR image of a scene is captured by activating only a subset of emitters at a time and by reading only a corresponding subset of LIDAR sensor channels while energizing the emitters. Different subsets of emitters can be activated at different times, where corresponding subsets of LIDAR channels are read synchronously with emitter activation; ultimately all emitters can be activated, and all LIDAR channels in the sensor array can be read with a single shot. As an example, the emitter array can emit light by activating one column at a time and sequentially from left to right during each shot cycle, and the sensor array can be configured to read corresponding LIDAR channels in a corresponding sequence. Alternatively, all emitters and all sensors can be activated simultaneously.

[0042] Some embodiments of the optical ranging device 202 may include one or more components to synchronize the emission and sensing of light, including support for electronic scanning. In some embodiments, the optical detection system 236 may include a sensor controller 225 coupled to the sensor array 226 and configured to control the operation of the sensor array 226. The sensor controller 225 may be any suitable component or group of components capable of selecting one or more optical sensors to sense light, such as an ASIC, a microcontroller, an FPGA, or any other suitable processor coupled to a selection circuit (e.g., a multiplexer). Similarly, the optical emission system 238 may include an emitter controller 215 coupled to the emitter array 214 and configured to control the operation of the emitter array 214. The emitter controller 215 may also be any suitable processor mentioned above for the sensor controller 225, and may include one or more drive components for operating the emitter array 214.

[0043] In some embodiments, the sensor controller 225 and the emitter controller 215 are synchronized so that light emission in the emitter array 214 is synchronized with reading out the light sensors in the sensor array 226. As an example, both the sensor controller 225 and the emitter controller 215 can be coupled to the timing controller 217 so that both controllers can operate based on the same timing scheme. The timing controller 217 can include a clock circuit 219 that generates a clock signal that oscillates between high and low states at a certain speed for coordinating the actions of digital circuits; examples of clock circuits are well known in the art. The various time periods described below can be defined with reference to the clock signal, and the timing controller 217 can provide the clock signal to the emitter controller 215 and the sensor controller 225. In some embodiments, the timing controller 217 can also include control logic (e.g., an ASIC, a microcontroller, an FPGA, or any other suitable processor) to generate additional timing control signals to the emitter controller 215 and the sensor controller 225. For example, the timing controller 217 can be configured to generate a trigger pulse signal that indicates when the emitter controller 215 should begin activating the emitters in the emitter array 214 and a sensor pulse signal that indicates when the sensor controller 225 should begin and end collecting data from the sensor array 226. In some embodiments, the trigger pulse signal and the sensor pulse signal can be the same signal, or the trigger pulse signal and the sensor pulse signal can be offset from each other by a fixed amount of time. In addition, as described below, the timing controller 217 can introduce jitter between consecutive shots by changing the interval between generating trigger pulses. In some embodiments, the sensor controller 225, the emitter controller 215, and the timing controller 217 can all be manufactured in the same integrated circuit device, and the emitter array 214 and the sensor array 216 can also be manufactured in the same device, thereby providing a compact LIDAR array.

[0044] In some embodiments, instead of or in addition to the sensor controller 225 and the emitter controller 215, the ranging system controller 204 may be configured to synchronize the operation of the light sensing module 208 and the light transmission module 206 so that the emission of light by the emitters in the emitter array 214 is synchronized with the readout of the light sensors in the sensor array 226. For example, the ranging controller 204 may instruct the emitter array 214 of the light transmission module 206 to emit light (from one or more emitters at any given time) and correspondingly instruct the sensor array 226 in the light sensing module 208 to sense light (using one or more light detector channels). In such an embodiment, the ranging / imaging system controller 204 may have its own clock signal based on which it instructs the light sensing module 208 and the light transmission module 206.

[0045] Any of the above or other synchronization techniques may be used to implement an electronic scanning mode of operation, wherein different subsets of emitters in the emitter array 214 are triggered at different times and different subsets of photosensors in the sensor array 226 are triggered at times synchronized with the triggering of specific emitters. It should be understood that other forms of sequencing for light emission and detection are contemplated and do not require electronic scanning.

[0046] In some embodiments, the processor 222 and memory 224 of the light sensing module 208 can perform signal processing operations. As an example of signal processing, for each photosensor or grouping of photosensors, the memory 224 can accumulate the counts of photons detected in consecutive time intervals (or other light intensity measurements), and these time intervals can be combined to recreate a time series of reflected light pulses (e.g., photon counts relative to time). This time series of aggregated photon counts (or other measurements of light intensity) is referred to herein as an intensity histogram (or just histogram). In addition, the processor 222 can apply certain signal processing techniques (such as matched filtering) to help recover the photon time series, which is not susceptible to pulse shape distortion caused by saturation and quenching of the photodiode. In some embodiments, the processor 222 can apply signal processing techniques (e.g., calibration-based correction) to reduce noise and / or compensate for channel-to-channel variations in intensity measurements.

[0047] In some embodiments, the output of the processor 222 from the light sensing module 208 is sent to the ranging system controller 204 for further processing. The ranging system controller 204 can be implemented in a variety of ways, including, for example, by using a programmable logic device such as an FPGA, as an ASIC or part of an ASIC, using a processor 230 with a memory 232, or some combination of the above. The ranging system controller 204 can control the light sensing module 208 by sending commands including commands to start and stop light detection and commands to adjust light detector parameters. Similarly, the ranging / imaging system controller 204 can control the light transmission module 206 by sending commands including start and stop light emission control commands and commands that can adjust other light emitter parameters such as emitter temperature control (for wavelength tuning), emitter drive power and / or voltage. In some embodiments, one or more components of the ranging system controller 204 can also be integrated into the same ASIC as the sensor array 226, processor 222, and memory 224, thereby eliminating the need for a physically separate ranging controller module.

[0048] If the transmitter array 214 has multiple independent drive circuits, there may be multiple on / off signals that can be appropriately sequenced by the ranging system controller 204, for example. Similarly, if the transmitter array includes multiple temperature control circuits to tune different transmitters in the array differently, the transmitter parameters can include multiple temperature control signals. In some embodiments, the ranging system controller 204 has one or more wired interfaces or connectors (e.g., traces on a circuit board) for exchanging data with the light sensing module 208 and with the optical transmission module 206. In other embodiments, the ranging system controller 204 communicates with the light sensing module 208 and the optical transmission module 206 via a wireless interconnect such as an optical communication link.

[0049] The user interface 250 may include hardware and software components, such as a computer system with a monitor, keyboard, mouse, CPU, and memory; a touch screen mounted in a console (e.g., on a car dashboard); a handheld device with a touch screen; or any other suitable user interface device. The components of the user interface 250 may be local to the object on which the LIDAR system 200 is mounted, but may also be located in a different physical system that remotely operates the LIDAR system 200. For example, commands and data sent to / from the LIDAR system 200 may be routed through a cellular network (LTE, etc.), a personal area network (Bluetooth, Zigbee, etc.), a local area network (WiFi, IR, etc.), or a wide area network such as the Internet.

[0050] The user interface 250 can present LIDAR data from the optical ranging device 202 to the user and / or allow the user or the upper-level program to control the optical ranging device 202 using one or more commands. Example commands may include commands that activate or deactivate the optical ranging device 202; specify the exposure level, bias, sampling duration, and other operating parameters of the photodetector (e.g., pulse pattern and signal processing for emission); specify light emitter parameters such as brightness; and so on. In addition, the command may allow the user or the upper-level program to select a method for displaying or interpreting the results. The user interface may display LIDAR system results, which may include, for example, single-frame snapshot images, continuously updated video images, and / or displays of other measurements of some or all pixels or sensor channels. Examples of other measurement results for pixels or sensor channels include ambient noise intensity, return signal intensity, calibration target reflectivity, target classification (hard target, diffuse target, retroreflective target), range, signal-to-noise ratio, target radial velocity, return signal time pulse width, and so on. In some embodiments, the user interface 250 may provide and track information indicating the distance (proximity) of an object from the vehicle. For example, such information may be used to identify and track objects in the field of view and potentially provide alerts to the user or perform any other operations that utilize odometry data, including but not limited to autonomous vehicle control.

[0051] In some embodiments, the optical ranging device 202 can communicate with other systems or devices. For example, in some automotive applications, the optical ranging device 202 can communicate with an automatic vehicle control unit (not shown), and the automatic vehicle control unit can modify one or more parameters associated with vehicle control based on data received from the optical ranging device 202. For example, in a fully autonomous vehicle, the optical ranging device 202 can provide a real-time depth image of the vehicle's surroundings to assist navigation. In other automotive applications, the optical ranging device 202 can be used as part of an advanced driver assistance system (ADAS) or as part of a safety system, which can, for example, provide depth image data to any number of different systems (e.g., adaptive cruise control, automatic parking, driver drowsiness monitoring, blind spot monitoring, collision avoidance systems, etc.). When the vehicle control unit is communicatively connected to the optical ranging device 202, an alert can be provided to the driver, or the proximity of an object can be tracked and / or displayed.

[0052] The optical ranging system 200 may also include Figure 2 For example, in a scanning LIDAR system (e.g., Figure 1A In some embodiments of the scanning LIDAR system 100, the optical ranging system 200 may include appropriate hardware components to rotate the housing containing the emitter array 214 and the sensor array 226, such as a rotary actuator, a motor, a magnetic motor, a rotary encoder, etc., and control logic implemented in programmable and / or fixed function logic circuits. In other scanning embodiments, the optical ranging system 200 may include light guiding components such as MEMS mirrors and appropriate control logic to support electronic scanning.

[0053] In some embodiments, the optical ranging device 202 may implement a rotational ranging system that may rotate continuously (e.g., at a rate of 10-30 Hz) and may determine when to start and stop data collection based on the current rotation angle. For example, the optical ranging device 202 may track the angular position of the sensor array 226 (e.g., using a rotary encoder) and may define a set of M "measurement angles" , which corresponds to uniformly spaced angular positions. The sensor array 226 can (along with the other components of the ranging device 202) rotate continuously at a uniform angular velocity, and the sensor channels can continuously generate signals. The memory 224 can accumulate counts of detected photons over consecutive time selections, which can be used to create an intensity histogram as described below. A controller (e.g., the ranging system controller 204) can receive an indication of when the encoder position corresponds to a measured angle. One of the signals. This signal (also referred to as a "marker" signal) can mark the boundary between consecutive measurement periods of the sensor channel. In response to this signal, the histogram data collected in the memory 224 can be sent to a digital signal processor (DSP) (e.g., processor 222 or processor 230) for analysis, which can include, for example, applying a filter to the histogram data to determine the exact time when the reflected LIDAR pulse is received. In response to the same signal, the memory 224 can start accumulating data for the next histogram. In some embodiments, the memory 224 can include two (or more) libraries dedicated to storing photon counts, and the photon count data from alternative measurement periods can be stored in alternative libraries. It should also be understood that continuous rotation is not required. In some embodiments, the rotating ranging system can rotate and collect data in a step-by-step manner, for example, rotating to a first measurement angle, collecting data for a measurement period, and then rotating to the next measurement angle and repeating data collection. In addition, as described above, the LIDAR system can be a static system operating without rotating parts, and the measurement period can be started at any desired time.

[0054] It will be understood that the LIDAR system 200 is illustrative and variations and modifications are possible. For example, while specific processors, controllers, and memories are shown as being associated with different modules within the LIDAR system 200, some embodiments may provide different arrangements, for example, where one processor and memory provide operational control and support for both the light transmission module and the light sensing module. All processors may be implemented using conventional digital logic circuits, including microprocessors, microcontrollers, FPGAs, ASICs, and the like. All memories may be implemented using conventional semiconductor-based memory circuits, such as SRAM, DRAM, flash RAM, and the like. Other implementations are also possible.

[0055] Figure 3 FIG. 3 is a simplified side view of components of an optical distance measuring device 300 according to an embodiment of the present invention. The optical distance measuring device 300 may be Figure 2 The optical ranging device 300 includes an optical transmission (Tx) module 306 (eg, implementing the optical transmission module 206) and an optical sensing (Rx) module 308 (eg, implementing the optical sensing module 208).

[0056] like Figure 3 As shown, the Tx module 306 may include a Tx side micro-optics package 312 and an integral optical element 314. The Tx side micro-optics package 312 includes a plurality of optical emitters 316 (eg, Figure 2The transmitter array 214 of FIG. 214 may include a microlens layer 318 and an aperture layer 320. The transmitters 316 may be arranged in a one-dimensional or two-dimensional array of transmitter channels (e.g., channels 325 shown in the box-shaped area). Each of the transmitter channels has one or more transmitters 316 capable of emitting narrowband light (e.g., near infrared (NIR) vertical cavity semiconductor lasers (VCSELs), etc.), as well as microlenses from the lens layer 318 and apertures from the aperture layer 320.

[0057] The Rx module 308 may include an Rx-side integral imaging optics module 330 and an Rx-side sensor package 332. The Rx-side sensor package 332 includes a plurality of light sensing channels 334 (eg, Figure 2 The sensor array 226 of the Tx side may include a microlens layer 336 and an aperture layer 338. The light sensing channels 334 may be arranged in a one-dimensional or two-dimensional array, which may match the arrangement of the Tx side micro-optical package 312, with a light sensing channel 334 corresponding to each micro-optical transmitter channel 325. Each light sensing channel 334 may include collimating optics, diffusers, etc. as needed, as well as a narrow bandpass filter to selectively pass vertically incident photons having a wavelength emitted by the corresponding emitter 316. Each light sensing channel may also include a set of one or more photosensors that are arranged to receive photons that pass through the narrow bandpass filter. Each photosensor may be a photosensor capable of detecting photons using a detector active area made of, for example, one or more standard photodiodes, avalanche photodiodes (APDs), single photon avalanche diodes (SPADs), RCPs (resonant cavity photodiodes), optical nanoantennas, microbolometers, or other suitable photodetectors. It should be understood that a single light sensing channel 334 may include multiple photosensitive regions (e.g., multiple SPADs) that work together to act as a single sensor, typically with a higher dynamic range, faster response time, or other beneficial characteristics compared to a single large photon detection region. Other structures may also be provided to improve detection efficiency and reduce crosstalk with adjacent sensor channels.

[0058] In operation, the Tx module 306 can provide effective illumination of objects in the area surrounding the LIDAR system 300 by transmitting, for example, pulses of NIR light having a spectral width of, for example, 10 nm, 2 nm, 1 nm, 0.5 nm, 0.25 nm, or less, into one or more fields of view. Light emitted from one emitter 316 diverges as it approaches one of the micro-optics of the Tx-side micro-optic lens layer 318. The micro-lenses of the micro-lens layer 318 capture the diverging light and refocus it onto a focal plane that coincides with an aperture in the aperture layer 320, which can include an aperture array that corresponds in situ to the micro-lens array and the array of emitters 316. The aperture layer 320 can reduce crosstalk in the system. After leaving the aperture, the focused light diverges again in the form of a cone and then encounters the Tx-side integral imaging optical module 314. In some embodiments, the spacing between the microlens layer 318 and the Tx-side integral imaging optical module 314 is equal to the sum of their focal lengths, so that the light focused on the aperture array 320 appears as collimated light at the output of the Tx-side integral imaging optical module 314, where each collimated beam leaves the Tx-side integral imaging optical module 314 at a different chief ray angle. Therefore, the light from each emitter is directed to a different field of view just in front. In some embodiments, the Tx-side integral imaging optical device 314 is telecentric on the imaging side (i.e., the emitter side) of the lens, i.e., for each position on the image plane (i.e., the emitter plane), the chief rays on the image side of the integral imaging optical device 314 are substantially parallel to each other and perpendicular to the image plane. In this configuration, the emitter array can be used as a telecentric light source, i.e., the optical device captures substantially all of the light generated by the emitter array, even light emitted from emitters on the outer edges of the array.

[0059] Part of the light emitted by the Tx module 306 and reflected from an object in the field of view, as shown by light ray 340, enters the Rx-side integral imaging optical module 330 from multiple directions. The Rx-side integral imaging optical module 330 may include a single lens or a plurality of lens groups that focus the light ray 340 onto a plane that coincides with the Rx-side input aperture layer 338, thereby allowing the light to enter the light sensor channel 334. In some embodiments, the Rx module 308 includes a light sensing channel 334 for each emitter 316, and the field of view of each individual light sensing channel 334 matches the field of view of its corresponding emitter 316.

[0060] It should be understood that the above-described LIDAR system is illustrative and variations and modifications are possible. Components of the LIDAR system described with reference to different figures may be combined in the same LIDAR system. In some embodiments, the LIDAR system may have an optical Tx module and an optical Rx module manufactured on the same ASIC, which may also incorporate a processor and memory. However, no specific implementation is required. The LIDAR system may have any number of channels (including only one channel). Specific optical wavelengths, timing parameters, optical elements, etc. may be changed as desired. A LIDAR system incorporating features of the claimed invention may be used in any application where ranging data is useful.

[0061] 1.2 Example LIDAR Operation

[0062] A LIDAR system such as LIDAR system 300 has multiple LIDAR channels, each LIDAR channel including an emitter (e.g., emitter channel 325) and a corresponding light sensor channel (e.g., light sensor channel 334). LIDAR system 300 (or other LIDAR systems) may be operated to determine the distance to an object in the field of view based on the time between the emission of a light pulse from emitter channel 325 and the detection of reflected light by the corresponding light sensing channel 334. An example of the operation of a single LIDAR channel will now be described; it should be understood that each LIDAR channel may operate similarly.

[0063] In operation, the transmitter channel 325 may be activated from time to time to emit light pulses that propagate into the environment in a particular direction. Some light may be reflected from objects in the environment and detected by the photosensor in the light sensing channel 334. To support ranging applications, the light sensing channel 334 may measure the light intensity received in each period of a set of short time periods (referred to as time intervals). For example, as described above, the light sensing channel 334 may include multiple SPADs, and the number of SPADs that emit photon signals may be used as a representative of light intensity. Photon counting detectors or other types of detectors capable of measuring light intensity may also be used. The duration of the time interval may be selected as desired; in various embodiments, each time interval may be 1ns, 2ns, 5ns, etc.

[0064] Figure 4 A simplified example of the operation of a LIDAR channel of a single "shot" LIDAR system 300 as a function of time is shown. Line 402 represents the power to the transmitter channel 325. At time t 1 , pulse 403 is applied to emitter channel 325, and emitter channel 325 emits a light pulse in response to pulse 403. Line 404 represents a histogram of the photon counts in light sensing channel 334 at each time interval. Light sensing channel 334 in this example emits a light pulse at time t1 Start sensing. (In some embodiments, data collection can be done with t 1 Initially, noise may be detected; however, as used herein, "noise" may include photons of ambient light in the environment or stray light from adjacent channels of the LIDAR system 300, as well as electronic noise (i.e., signals not caused by photons). After a period of time, at time t R , a peak is detected in the photon count 406. In some embodiments, the indicated threshold N may be used min to detect peak 406. The flight time can be calculated as t R -t 1 , the distance to the reflecting object can be defined as c(t R -t 1 ) / 2, where c is the speed of light. The data collection cycle can have a fixed duration t s , after which the next pulse can be generated.

[0065] In some embodiments, signal-to-noise discrimination can be improved by accumulating histogram 404 across multiple shots. As long as the photon counts added to each bin during a different shot correspond to the pulse start time t relative to that shot, 1 For the same fixed time of θ (and assuming that the total accumulation time is short relative to the expected speed of motion of objects in the environment), the time intervals corresponding to the signal are expected to accumulate photons faster than the time intervals corresponding to the noise. This can allow for a higher threshold for distinguishing signal from noise, which can improve the accuracy of the ranging determination.

[0066] Other techniques for improving the discrimination between signal and noise, including filtering techniques, may also be used. For example, in each shot, the transmitter may generate a pulse train having multiple pulses (which may have different durations), and a matched filter may be applied to the raw histogram data (a single shot or after accumulating a histogram of a "cycle" consisting of multiple shots) to generate filtered histogram data. The filtered histogram data may be analyzed to detect patterns corresponding to the emission pattern of the pulse train. Examples of accumulation, pulse trains, matched filtering, and related techniques for improving signal-to-noise discrimination in LIDAR systems are described in co-owned U.S. Patent Application Publication No. 2018 / 0259645 (the disclosure of which is incorporated herein by reference). It should be understood that any of these and other techniques may be applied to a LIDAR system that implements single-shot jitter as described below.

[0067] 2. Crosstalk from external pulse sources

[0068] It should be understood that a LIDAR sensor channel (e.g., any of the sensor channels described above) may be designed to detect photons and / or otherwise measure light intensity within a specific (usually narrow) frequency band. Such a LIDAR sensor channel may not be able to distinguish between light originating from the LIDAR emitter and light originating from some other source that simultaneously produces light at the same wavelength as the LIDAR emitter. "Crosstalk" is used herein to refer to light sensed by a LIDAR sensor channel that originates from a source other than the LIDAR emitter (or LIDAR emitter array) to which the LIDAR sensor channel is operably coupled. Certain embodiments of the present invention relate to techniques for identifying, reducing and / or eliminating such crosstalk.

[0069] In this section, it is assumed that a LIDAR system such as LIDAR system 300 is operated in an "accumulation" mode, in which light intensity data (e.g., photon counts in a particular light sensing channel) is accumulated over multiple shots (e.g., as described above), and that a crosstalk source emits pulses at regular intervals that may accumulate to form a crosstalk peak in a histogram that may be misidentified as a signal. Examples of techniques that can be used to reduce the effects of this type of crosstalk will now be described. Although the examples are described with reference to LIDAR system 300 or LIDAR system 200, it should be understood that the same techniques can be applied to any LIDAR system in which light intensity data corresponding to multiple shots is accumulated (e.g., in a histogram) as described above.

[0070] 2.1 Crosstalk Peak

[0071] The emission may occur at fixed intervals. However, in some cases, LIDAR system 300 may be operated in the field of view of another LIDAR system, and light sensing channel 334 may sense light from the other LIDAR system. If the emitters in both LIDAR systems operate at the same emission frequency, this may cause crosstalk between the LIDAR systems. To illustrate, Figures 5A-5D An example of a histogram accumulating photon counts per time bin in the presence of inter-system crosstalk is shown. Figures 5A-5C Representative histograms from three different shots are shown. Figure 5D Shown for including Figures 5A-5C The cumulative histogram of a set of shots is the result. Figure 5D In the example, two different peaks 504 and 506 both exceed the threshold N. minIn this case, peak 504 is a "real" peak corresponding to reflected light from emitter 316, while peak 506 is a "crosstalk" peak corresponding to light from an external pulsed light source. However, LIDAR system 300 may not be able to distinguish between real peak 504 and crosstalk peak 506. For example, while this example shows that real peak 504 has a higher maximum photon count than crosstalk peak 506, this is not always the case.

[0072] 2.2 Single shot jitter to reduce crosstalk

[0073] Certain embodiments may reduce this type of crosstalk by introducing quasi-random variations in the timing of successive pulses of transmitter 316, referred to herein as single-shot jitter (or just jitter). Figure 6 The per-shot jitter according to an embodiment of the present invention is illustrated. Figure 6 601, 602, 603, 604. However, within each shot (i), the trigger pulse to the transmitter 316 indicated by event markers 611, 612, 613, 614 is delayed by the shot-specific jitter (time t i ). (Note that in this example, t 2 The data collection period for each shot indicated by the shaded rectangles 621, 622, 623, 624 is synchronized with the trigger pulse for that shot and is constant in duration (e.g., as described above with reference to Figure 4 Defined s ).

[0074] The jitter for each shot can be selected using various techniques; specific examples are described below. The shot-specific jitter can be longer than the histogram time interval and can be, for example, up to 20% of the total duration of the data collection phase for the shot, or more as desired. Different shots preferably have different shot-specific jitter so that the trigger pulses do not occur at regular intervals.

[0075] The light sensing channel 334 begins recording data during each shot in sync with the trigger pulse (rather than at the start of the shot) so that in the histogram accumulated over multiple shots, the photon counts due to reflected light from an object at a constant distance from the LIDAR system 300 are accumulated in the same time bin. As long as any crosstalk light source is not running with the same shot-specific jitter, different shots will have photon counts due to crosstalk accumulation in different time bins. Thus, crosstalk peaks can be "spread" across multiple bins of the histogram. Figures 7A-7D An example of a histogram accumulating photon counts per time bin with single shot jitter is shown in accordance with an embodiment of the present invention. Figures 7A-7C Representative histograms from three different shots are shown. Fig.7D Shown for including Figures 7A-7C The cumulative histogram of a set of shots is the result. Figures 7A-7C In FIG. 1 , crosstalk peaks 706a-c occur at different times relative to the start of data collection for the outgoing signal, while true peaks 704a-c occur at the same time relative to the start of data collection for the outgoing signal. As a result, in Fig.7D In the cumulative histogram shown, the crosstalk peak 706d is spread across multiple time intervals, while the true peak 704d is not similarly spread. Therefore, the appropriate threshold N min The true peak 704d and the crosstalk peak 706d can be reliably distinguished.

[0076] The single shot jitter for each shot can be selected, and the jitter can be as long as desired. To effectively spread the crosstalk peak, the jitter selected for different shots within a cycle can vary by a large enough amount to span multiple time intervals. For example, in some embodiments, the data collection period for each shot (i.e., the time between the trigger pulse and the last time interval of the histogram) is 1 μs, and the jitter time for each shot can be selected in the range of 0 to 200 ns (20%). The shot interval (i.e., the time from the start of one shot to the start of the next shot, such as Figure 6 The interval between shots (as shown) can be long enough to allow for jitter so that the next shot does not start before the data collection period for the previous shot ends. For example, if the data collection period is 1 μs and the jitter is as high as 200ns, the shot interval can be 1.2 μs (or longer).

[0077] In some embodiments, the single shot jitter t i The selection may be random; however, random selection may be "awkward" (e.g., the same value may be selected for multiple consecutive shots), which may reduce the desired propagation effect. Therefore, some embodiments use equal-energy sampling techniques or other techniques that are not purely random to avoid "awkward" sequences of single-shot jitter. For example, if the histogram is accumulated over 20 shots and the jitter has ten possible values, then each possible jitter value should be selected twice; more generally, if the histogram is accumulated over M shots and the jitter has J possible values, then each possible jitter value should be selected M / J times (rounded up if M / J is not an integer). Random or pseudo-random selection techniques subject to this constraint may be used.

[0078] In some embodiments, a "master" dithering sequence may be predefined (eg, by loading a sequence of dithering time values ​​into device firmware). Figure 81 shows a graphical representation of an example master jitter sequence according to an embodiment of the present invention. The vertical axis corresponds to the shot number and the horizontal axis corresponds to the single shot jitter t assigned to the shot number. i In some embodiments, the single shot jitter t i It can be expressed as a specific amount of time (e.g., in the range from 0 ns to 200 ns or from 0 μs to 1.2 μs). In other embodiments, the single shot jitter t i It can be expressed as a fraction or percentage of the data collection period (eg, 0 to 0.2 or 0 to 1.2), which allows the same master dither sequence to be used in systems where the length of the data collection period and / or the shot interval are adjustable parameters.

[0079] In this example, assume that the histogram is accumulated over a "cycle" of 40 shots. Within a cycle, a series of jitter values ​​for each shot is created using equal energy sampling. For example, jitter values ​​can be assigned in increments ranging from 0 to 200 ns, in increments of the width of a time interval or multiples thereof (e.g., 1 ns, 2 ns, 5 ns, etc.). Different sequences of jitter values ​​can be assigned to different cycles; for example, Figure 8 Cycle 1 and cycle 2 in have different jitter sequences. Any number C of jitter sequences can be defined (e.g., hundreds of cycles). In some embodiments, jitter sequences of different cycles can be different permutations of the same set of possible jitter values. The jitter sequence can be generated prior to operation (e.g., as part of device firmware) and stored in a memory of the LIDAR system 200, for example, as a jitter array indexed by the cycle number and the number of shots within the cycle. In some embodiments, all instances of the LIDAR system 200 have the same contents in their respective jitter arrays. At startup, the LIDAR system 200 can randomly (or pseudo-randomly) select a starting cycle number and use the corresponding jitter sequence, and then increase the cycle number to select a different jitter sequence for the next cycle. Once the end of the main jitter sequence is reached, the LIDAR system 200 can loop back to the beginning. This makes it less likely that two different instances of the LIDAR system 200 that happen to be close to each other will also happen to be at the same point in the main jitter sequence at the same time. As long as two instances of LIDAR system 200 apply different jitter values ​​at any given time, crosstalk between them is unlikely to produce false signal peaks in either system.

[0080] In some embodiments, a master dither sequence may be generated for a large number of cycles (e.g., as described above) and then truncated (e.g., by leaving one shot from the last cycle) such that the length of the master dither sequence is not an integer multiple of the number of shots per cycle. In this case, the LIDAR system 200 may randomly (or pseudo-randomly) select any individual entry in the master dither sequence as a starting point and increment through the sequence from that point, looping each time the end point is reached. This may further reduce the likelihood that two different LIDAR system 200 instances that happen to be near each other also happen to be at the same point in the master dither sequence at the same time, making crosstalk between them less likely to produce false signal peaks in either system.

[0081] To further illustrate the use of the master dither sequence, Fig. 9 is a flow chart of a process 900 for operating a LIDAR channel according to an embodiment of the present invention. Process 900 may be performed, for example, in Figure 2 The LIDAR system 200 is implemented using the timing controller 217 and / or the processor 230 .

[0082] Process 900 may begin when LIDAR system 200 is initialized to begin performing ranging operations. For example, process 900 may begin during system startup of LIDAR system 200 (e.g., in response to powering up the system) or in response to a user input command to begin ranging operations. At block 902, process 900 may obtain a seed. The seed may be obtained in a variety of ways. For example, a seed may be generated based on a system clock using techniques known in the art. Other techniques may also be used. It is assumed that a different seed may be obtained each time process 900 begins, although this is not required. At block 904, process 900 may select a master jitter sequence based on the seed (e.g., as described above with reference to FIG. 1 ). Figure 8 For example, the master dither sequence may be stored as an array in the memory 232 (or a memory of the timing controller 217), and the seed may be used to select a position in the array.

[0083] At block 906, a ranging cycle begins. As described above, a ranging cycle may include a fixed number of shots (eg, 20 shots, 40 shots, or some other number of shots), and at block 908, a shot within the ranging cycle may begin.

[0084] At block 910, the process 900 selects the next jitter value from the master jitter sequence. For the first shot, the selected jitter value may be the jitter value corresponding to the starting position selected at block 904. At block 912, the process 900 generates a trigger pulse (for at least one emitter channel) with an offset time based on the selected jitter value. At block 914, the process 900 begins data collection by the light sensing channel in synchronization with the trigger pulse to the corresponding emitter channel. It should be understood that data collection need not begin simultaneously with the trigger pulse, as long as any offset between the time of the trigger pulse and the time to begin data collection remains constant (within a time interval). Data collection for one shot may include generating or adding a photon count histogram for each time interval, etc. At block 916, the process 900 may wait for data collection for the shot to complete. The duration of the wait depends on the length of the data collection period t s , which in turn may depend on the maximum range of the particular LIDAR system.

[0085] At block 918, process 900 determines whether the cycle is complete. As described above, each ranging cycle may include a fixed number of shots, and a counter or similar structure may be used to determine whether the fixed number of shots have been completed. If not, process 900 returns to block 908 to start the next shot. For the next shot, at block 910, process 900 may step to the next position in the main dither sequence and select a corresponding dither value; upon reaching the end of the main dither sequence, process 900 may loop back to the beginning of the main dither sequence.

[0086] After the cycle of block 918 is completed, process 900 may proceed to block 920 to determine ranging information (e.g., distance to an object in the environment) using the accumulated histogram. The ranging information and / or other information, such as some or all of the raw photon counts in the histogram, information characterizing peaks (e.g., height and / or width) may be provided to a downstream device or system, such as a rendering engine, which renders a graphical image based on the ranging information and / or other information provided by the LIDAR system, a data collection and storage system, a user interface for displaying data, or an automatic control system that operates in response to the ranging information. At block 922, process 900 may initiate a new cycle by returning to block 906. Initiating a new cycle may include resetting all histogram data, but not resetting the position in the master jitter sequence. In this way, process 900 may continue indefinitely. If at any time it is determined that no more cycles are to be initiated, process 900 may end at block 930.

[0087] It should be understood that process 900 is illustrative and variations and modifications are possible. Operations described in a particular order may be performed in a different order or in parallel to the extent logically permitted; some operations may be omitted; and other operations may be modified or combined.

[0088] For example, in the above example, the jitter value for a given shot is selected according to a predefined master jitter sequence. Using a predefined master jitter sequence allows simple run-time logic to step through the sequence and select each jitter value in turn. Another implementation may allow the jitter value for a given shot to be selected in real time (or near real time), for example, by implementing an isoenergetic sampling technique that generates outputs in real time or near real time, although this may complicate the real-time control logic.

[0089] In the above example, process 900 provides a single pulse per shot. Those skilled in the art will appreciate that process 900 may be applied to a LIDAR system in which each shot includes a pulse train of one or more pulses spaced at specified intervals; jitter may be applied to the first pulse of the pulse train, and subsequent pulses in the pulse train may be generated at consistent times relative to the time of the first pulse. The use of pulse trains enables various pulse encoding schemes (e.g., Barker encoding) to facilitate distinguishing pulses from different sources; examples of pulse trains and encoding techniques are described in U.S. Patent Application Publication No. 2018 / 0259645 cited above. In some embodiments of the present invention, jittering as described above may be applied to distinguish pulses originating from sources that happen to use the same or similar pulse encoding schemes.

[0090] The specific choice of jitter value may vary. As described above, jitter may be used to spread crosstalk peaks between multiple time periods in a histogram; therefore, jitter should be greater than one time interval. In some embodiments, the maximum amount of jitter may be defined as any fraction of the sampling period, and in some embodiments, the fraction may be greater than 1. For example, some embodiments allow jitter up to ten times the data collection period (e.g., up to 10 μs of jitter for a sampling period of 1 μs). In the event that the jitter exceeds the data collection period, the throughput (or rate of successive shots) may be reduced accordingly so that the shots do not overlap. In addition, while the above example assigns a "positive" jitter offset that delays the trigger pulse relative to the nominal start of the shot period, other implementations may use a "negative" jitter offset so that the trigger pulse occurs earlier than the nominal start of the shot period, as long as data collection for one shot is completed before the first pulse of the next shot. It should be noted that jitter differs from the pulse interleaving described in U.S. Patent Application Publication No. 2018 / 0259645 cited above in at least two respects. One is that pulse interleaving introduces variations between the emitter and sensor timing, shifting the arrival time of the reflected light from the emitter; in contrast, the dithering described in this article does not change the relative emitter / sensor timing. Another is that dithering is intended to spread the crosstalk pulses across multiple histogram bins during the accumulation process, where interleaving is typically limited to a time offset of less than one bin. That said, it should be understood that both dithering and interleaving can be implemented in the same LIDAR system.

[0091] Additionally, the above description relates to a single LIDAR channel having an emitter and a corresponding light sensing channel. As described above, a LIDAR system may include an array of multiple LIDAR channels. In this case, dithering may be applied to each channel in the manner described. In some embodiments, the same dither offset is applied to all channels for a given shot. In other embodiments, a dither offset may be selected for each channel such that different channels in the same array apply different dither offsets; for example, different channels may start at different positions in the same master dither sequence. This option may reduce crosstalk between channels in the same LIDAR system, although it may require more complex control circuitry to trigger different emitters (and corresponding detectors) at different times relative to each other.

[0092] 3. Adaptive operation to reduce crosstalk

[0093] In some embodiments, a LIDAR system such as LIDAR system 200 can detect the presence of possible crosstalk and can adaptively modify its own operation in a manner that can reduce the impact of crosstalk on ranging operations. Examples of detecting crosstalk and adjusting system operation based on crosstalk detection will now be described.

[0094] 3.1 Detecting crosstalk

[0095] In some embodiments, a LIDAR system such as LIDAR system 200 may analyze histogram data (which may be collected with or without dithering) to infer the presence of crosstalk sources. Fig.10 1 is a flow chart showing a process 1000 that may be used in some embodiments to infer the presence of a crosstalk source from histogram data. The process 1000 may be implemented, for example, in the processor 230 of the LIDAR system 200. The process 1000 may use histogram data from one or more sensor channels, which may be described above with reference to Figures 5A-5D or Figures 7A-7D The process 900 may be used to collect histogram data with or without single shot dithering. For example, the process 900 may be used to collect histogram data with dithering. (To collect histogram data without dithering, a process similar to the process 900 may be used, omitting the operations associated with selecting and applying single shot dithering.)

[0096] Process 1000 begins after collecting histogram data for one cycle (or in some cases a single shot). At block 1002, process 1000 may identify a time interval corresponding to the highest peak in the histogram, e.g., a peak time interval or a set of adjacent time intervals where the number of photons exceeds a threshold. The threshold may be the same threshold used for peak detection in the context of ranging operations (e.g., as described above) or a different value. The time interval identified in block 1002 may be used for ranging operations, but is ignored in the remainder of process 1000.

[0097] At block 1004, process 1000 may calculate photon count statistics for "non-peak" time intervals, i.e., time intervals not identified as corresponding to peaks at block 1002. The statistics may include a "global" average photon count for each time interval, a standard deviation of the photon count for each time interval, and / or other desired statistics.

[0098] At block 1006, process 1000 may determine a "local" average photon count for the non-peak time interval. In some embodiments, the local average may be the photon count for a single non-peak time interval. In other embodiments, the local average may be the average photon count for a small group of consecutive time intervals (e.g., 3 intervals, 5 intervals, etc.); if desired, the local average may be a weighted average (e.g., weighting the center interval in the group most strongly).

[0099] At block 1008, process 1000 may compare the local average photon count calculated at block 1006 with the global average photon count calculated at block 1004, and at block 1010, process 1000 may determine whether the difference between the local average photon count and the global average photon count is significant. The criteria for identifying significance may be selected as desired. For example, a significant difference may be defined as a difference from the global mean greater than one standard deviation, a difference from the global mean greater than two standard deviations, or greater than some other threshold (which may be determined relative to a standard deviation or other measure of natural variability in noise). If a significant difference is found, then at block 1012, process 1000 may infer that there is a possible source of crosstalk. If not, then at block 1014, process 1000 may infer that there is no evidence of crosstalk.

[0100] In some embodiments, blocks 1006-1014 may be performed separately for some or all of the off-peak time intervals; the local averages may or may not overlap if desired. In addition, the inference of a possible crosstalk source may be based on more than one time interval having a local average photon count that is significantly different from the global average. For example, in some embodiments, a possible crosstalk source is inferred if at least a minimum number of consecutive time intervals have a local average photon count that is significantly different from the global average. When the histogram being analyzed is collected using single shot jitter, considering multiple time intervals may increase the likelihood of detecting a trailing crosstalk peak such as peak 706d in FIG. 7d.

[0101] Other processes may be used in addition to or in place of process 1000. For example, in some embodiments, a LIDAR system such as LIDAR system 200 may be operated in a "background" mode, where the light detection channels operate as usual but the emitters are not activated. Fig.11 A flow chart of a process 1100 is shown that can be used in some embodiments to infer the presence of a crosstalk source from histogram data collected using a background pattern. The process 1100 can be implemented, for example, in the processor 230 of the LIDAR system 200. At block 1102, data is collected for one cycle without activating the emitter. For example, each light sensing channel can be operated to accumulate a histogram based on a series of shots as described above while not activating the emitter to generate light pulses. Under these conditions, it can be assumed that any photons detected by the light detection channel are from an external source (i.e., not from the emitter of the LIDAR system).

[0102] At block 1104, the histogram accumulated during block 1102 may be analyzed to determine whether any peaks are detected, e.g., using the same peak detection techniques used during normal ranging operations. In some embodiments, other analyses may also be performed. For example, similar to the above reference Fig.10The technique described can be used to perform a statistical analysis of background noise and detect significant local deviations from the global mean of the background noise. (Any peaks detected during the analysis can be excluded.)

[0103] At block 1106, process 1100 may determine whether a peak is detected, or in some embodiments, whether a significant local deviation from a global average of background noise is detected. If a peak (or other significant local deviation from a global average) is detected, then at block 1108, process 1100 may conclude that there is a possible source of crosstalk. If not, then at block 1110, process 1100 may conclude that there is no evidence of crosstalk.

[0104] It should be understood that processes 1000 and 1100 are illustrative and that variations and modifications are possible. Specific techniques for detecting crosstalk may be adapted to a particular LIDAR system.

[0105] 3.2 Adaptive Response to Crosstalk Detection

[0106] In some embodiments, a LIDAR system such as LIDAR system 200 can use a process such as process 1000 or process 1100 to detect possible crosstalk and can adaptively modify its operation in response to detecting possible crosstalk. It should be understood that crosstalk can come from various types of sources. One type, sometimes referred to herein as "harmonic" crosstalk, is generated by a light source that pulses periodically with a period that is in harmony with the shot interval in the LIDAR system. The pulse period of the harmonic crosstalk source can be the same as the shot interval, a multiple of the shot interval, or a harmonic fraction of the shot interval (e.g., 1 / 2, 1 / 3, 1 / 4, etc.), so that, at least in the absence of jitter as described above, light from multiple pulses of the harmonic crosstalk source accumulates in the same histogram bin. Examples of harmonic crosstalk sources include other LIDAR systems that happen to have similar operating characteristics to the LIDAR system experiencing the crosstalk (e.g., two or more LIDAR systems manufactured according to the same design). Another type of crosstalk, sometimes referred to herein as "non-harmonic" crosstalk, can be generated by a light source that emits a single intense pulse during the period of the LIDAR system experiencing the crosstalk. Examples of non-harmonic crosstalk sources include LIDAR systems of different designs that may periodically emit pulses with a pulse period that is unrelated to the firing interval of the LIDAR system experiencing the crosstalk. Those skilled in the art with knowledge of the present disclosure will appreciate that the application of jitter specific to firing (e.g., as described above with reference to Fig.10 The described) can reduce the peaks associated with harmonic crosstalk, but may not have any effect on non-harmonic crosstalk. Therefore, other adjustments may be applied.

[0107] Fig.121 is a flow chart of a process 1200 for detecting crosstalk in a LIDAR system and adaptively modifying the operation of the LIDAR system to reduce the effects of crosstalk according to an embodiment of the present invention. The process 1200 may be performed, for example, in Figure 2 The LIDAR system 200 or other LIDAR system may be implemented in the LIDAR system 200 or other LIDAR system, including any LIDAR system described herein.

[0108] At block 1202, the LIDAR system may collect histogram data, for example, for a range finding period. In some embodiments, the LIDAR system may initially operate in a default mode without jitter and may adaptively add or modify jitter as described below. At block 1204, the LIDAR system may analyze the histogram data to infer possible crosstalk. For example, Fig.10 Course 1000 and / or Fig.11 The process 1100 may be used to infer possible crosstalk. The analysis of the histogram data may be performed, for example, by the processor 222 or the processor 230 of the LIDAR system 200.

[0109] At block 1206, a determination is made as to whether possible crosstalk is inferred. If not, the LIDAR system may continue to operate in its current mode (e.g., a default mode) at block 1208. Process 1200 may be repeated from time to time (e.g., for each ranging cycle or at regular intervals, e.g., once every 10, 20, 50, or 100 ranging cycles or some other defined interval) as conditions may change during operation.

[0110] If possible crosstalk is inferred at block 1206, then at block 1210, the LIDAR system may modify its operating mode. For example, if the LIDAR system is operating without dithering, then the modification of the operating mode at block 1210 may include adding dithering. As another example, if the LIDAR system is operating with dithering, then the modification of the operating mode at block 1210 may include further measures, such as increasing the range of dithering values, jumping to a new position in the master dithering sequence, or otherwise modifying the dithering characteristics. As yet another example, in an embodiment where the LIDAR system supports coded pulse (e.g., Barker code) modification of the operating mode at block 1210, it may include changing to a different supported pulse coding scheme. In one embodiment of the LIDAR system 200, the processor 230 may determine the modifications to be made and send appropriate commands to the timing controller 217, the sensor controller 225, and / or the transmitter controller 215.

[0111] In some embodiments, after modifying the operating mode at block 1210, process 1200 may return to block 1202 to evaluate the effect of the modification. If possible crosstalk is still inferred, the operating mode may be further modified until crosstalk is no longer inferred or until any inferred crosstalk has reached a sufficiently low level that no crosstalk peaks are expected to reach the signal detection threshold.

[0112] In some embodiments, the modification of the operating mode of the LIDAR system at block 1210 may be adaptively selected based on additional analysis of the inferred crosstalk. For example, crosstalk is typically strongest when the LIDAR system is pointed toward the source of the crosstalk pulses. In the case of a scanning LIDAR system, the crosstalk may depend on the rotation angle of the LIDAR system; in the case of a static LIDAR system, the crosstalk may be different for different parts of the array. Thus, in some embodiments, the LIDAR system 200 may use information collected from across the LIDAR array (e.g., which pixels in the field of view exhibit crosstalk signatures) to infer the direction of the crosstalk source. When pointed in the direction of the crosstalk source, the LIDAR system 200 may modify its behavior, for example, by increasing the transmitter energy (or intensity) and the threshold for detecting peaks in ranging operations, and / or by selectively using coded pulse trains when pointed in that direction, to enhance signal detection. In embodiments where the jitter is controllable per channel (or per channel group), the LIDAR system 200 may selectively apply jitter to channels pointed in the direction of the crosstalk source.

[0113] Another technique for modifying the LIDAR system operating mode may include attempting to characterize the source of the crosstalk. For example, in some embodiments, the LIDAR system may perform a background cycle (e.g., as described above with reference to Fig.11 As described). If a peak is detected during a background period, the LIDAR system 200 can analyze the peak to infer the characteristics of the source. For example, the LIDAR system 200 may have information characterizing pulses emitted from various types of light sources, such as other LIDAR systems. Such information may include, for example, pulse encoding (the number and timing of pulses), pulse shape (e.g., duration, ramp-up and / or ramp-down behavior), etc. If the pulses observed during the noise period match the pulse characteristics from a particular type of source to a sufficient degree, the LIDAR system 200 can determine that the pulse is likely to come from that type of source. In some embodiments, this information can be used to select modifications to be performed. For example, if the crosstalk source uses a specific pulse encoding, the LIDAR system 200 can select a different pulse encoding. As another example, the LIDAR system 200 can select a jitter scheme so that crosstalk from a particular source type is less likely to accumulate into a peak.

[0114] It should be appreciated that process 1200 is illustrative and that variations and modifications are possible.Different techniques for detecting crosstalk and adaptively modifying system characteristics to reduce the effects of the detected crosstalk may be implemented.

[0115] 4. Distance aliasing

[0116] In some embodiments, dithering as described above can also be used to reduce the effects of range aliasing. As used herein, "range aliasing" occurs when a light pulse from an emitter during a first shot reflects off an object far enough away that the reflected light does not return to the sensor until some time during the next shot (or any subsequent shots). If the emitter emits light regularly, and the sensor channels operate synchronously with the emitter, the late returning light can cause false peaks in the histogram, which are similar to the crosstalk peaks described above. These false peaks can cause errors in the range determination. Fig.13 A simplified illustration showing the range aliasing that can occur without jitter. Fig.13 is a timeline view. Shots start at fixed intervals (times 0, 2, 3, etc.) as indicated by event markers 1301, 1302, 1303, and 1304. Trigger pulses (event markers 1311, 1312, 1313, 1314) occur at fixed intervals relative to the start of each shot, i.e., no single shot jitter is applied. Data collection periods are represented by shaded rectangles 1321, 1322, 1323, 1324. A reflected light pulse 1331 from a light pulse generated in response to the first trigger pulse (event marker 1311) arrives during the second data collection period (rectangle 1322). Similarly, a reflected light pulse 1332 from a light pulse generated in response to the second and third trigger pulses (event markers 1312, 1313) arrives during the third and fourth data collection periods (rectangles 1323, 1324), respectively. The arrival times of the reflected pulses 1331, 1332, 1333 are consistent relative to the start of the data collection period during which they arrive, so the peaks can be accumulated in the histogram. However, if the peak outgoing pulse is detected assuming a peak response, the ranging information will be "aliased" to a shorter flight time than the actual flight time.

[0117] In some embodiments of the present invention, applying single shot dither in the manner described above may reduce range aliasing. Fig.14 A simplified illustration of range aliasing removal by applying per-shot dithering in accordance with an embodiment of the present invention is shown. Fig.14is a timeline view. Shots start at fixed intervals (times 0, 2, 3, etc.) as indicated by event markers 1401, 1402, 1403, and 1404. Trigger pulses (event markers 1411, 1412, 1413, 1414) occur at different intervals relative to the start of each shot, i.e., single shot jitter is applied. The data collection periods indicated by shaded rectangles 1421, 1422, 1423, 1424 are Figure 6 . A reflected light pulse 1431 from a light pulse generated in response to the first trigger pulse (event marker 1411) arrives during the second data collection period (rectangle 1422). A reflected light pulse 1432 from a light pulse generated in response to the second trigger pulse (event marker 1412) arrives just before the third data collection period (rectangle 1323), and a reflected light pulse 1433 from a light pulse generated in response to the third trigger pulse (event marker 1413) arrives during the fourth data collection period (rectangle 624). However, due to single shot jitter, the arrival time of the reflected pulse (relative to the start of data collection) varies from one shot to the next. Therefore, such late pulses are not expected to produce peaks in the cumulative histogram; instead, range aliasing peaks will propagate in a manner similar to crosstalk. In this case, the histogram may not have any significant peaks, which would result in determining that the nearest reflecting surface is out of range in this case (which is the desired result). Therefore, in addition to reducing the effects of crosstalk, applying single-shot dithering as described herein can also reduce range aliasing, further improving the accuracy of the ranging information.

[0118] 5. Other Embodiments

[0119] Although the present invention has been described with reference to specific embodiments, those skilled in the art will recognize, after having access to this disclosure, that many variations and modifications are possible. For example, a LIDAR array of the type described herein may be manufactured to include any number of rows and any number of columns per row. (The terms "row" and "column" are used to distinguish the two dimensions of the sensor array, particularly in the case of an array used in a scanning mode, and are not intended to imply any particular spatial orientation of the sensor array.) The specific construction of the sensor channels and emitters may vary. The emitter may emit a burst of one or more pulses during each shot, and jitter may be applied in the manner described above to vary the start time of consecutively fired bursts of pulses. Identification of reflected light may be based on an intensity pattern in a detection histogram corresponding to a burst pattern.

[0120] The LIDAR sensor channel (and / or other depth sensing channels) can provide various forms of time series data, for example, using a histogram of photon counts or signal intensity as a function of time. The LIDAR sensor channel can operate at various wavelengths, including near infrared, short wave infrared (e.g., 1600nm), mid wave infrared, and / or long wave infrared (e.g., up to 15μm).

[0121] Sensor arrays of the type described herein may be incorporated into a variety of sensing systems, including but not limited to the specific LIDAR systems described above. The LIDAR system may be implemented using a rotating and / or static platform as described above, and may be used in any application where it is necessary to collect ranging data. Those skilled in the art who understand the present disclosure will appreciate that any of the techniques described herein for detecting and / or reducing crosstalk may be applied to a variety of LIDAR systems, including but not limited to any of the example systems described herein.

[0122] The data generated by the LIDAR system may be analyzed using a variety of computer-implemented algorithms that operate on any portion of the data. In some embodiments, the data may be used to generate an image for display to a user, which may include directly rendering intensity data or distance measurements calculated from intensity histograms and / or rendering an image of a scene (or portion thereof) based on algorithmic inferences from the data. Although some of the above examples relate to vehicle navigation and / or driver assistance, the present invention is not limited to any particular data analysis or any particular application of LIDAR technology.

[0123] For the purpose of illustration and description, the above description of exemplary embodiments of the present invention has been presented. The above description is not intended to be exhaustive or limit the present invention to the precise form described, and in view of the above teachings, many modifications and variations are possible. The purpose of selecting and describing the embodiments is to explain the principles of the present invention and its practical application, so that other technical personnel in the field can utilize the present invention in various embodiments and with various modifications suitable for the specific purpose envisioned. Therefore, although the present invention has been described with reference to specific embodiments, the present invention should be understood to be limited only by the appended claims.

Claims

1. A LIDAR system, comprising: an optical transmission module having an emitter to emit a light pulse during each of a plurality of shots in response to a trigger pulse, wherein each shot is a time period including a light pulse and a subsequent data collection period; a light sensing module having at least one light sensor channel to receive light and configured to: generate a histogram of received light intensity as a function of time for a sequence of time intervals in response to the trigger pulse, the light sensing module also having a first memory circuit configured to accumulate the histogram over the plurality of shots; as well as a timing controller coupled to the light transmission module and the light sensing module and configured to generate a trigger pulse sequence to synchronize the operation of the light transmission module and the light sensing module for each emission, The timing controller is further configured to apply a single shot jitter to each trigger pulse, wherein the single shot jitter varies for different shots, and wherein for at least some of the plurality of shots, the single shot jitter is greater than a time interval; as well as A processor is configured to analyze the histogram to determine whether crosstalk is present and adaptively modify the single-shot jitter in response to determining that crosstalk is present.

2. The LIDAR system of claim 1 , wherein the single shot jitter is selected within a predefined jitter value range using an equal energy sampling technique. 3 . The LIDAR system according to claim 2 , wherein the predefined jitter value range is 0-20% of the total number of time bins in the histogram. 4 . The LIDAR system of claim 2 , wherein the predefined jitter value range is 0-120% of the total number of time bins in the histogram.

5. The LIDAR system of claim 1, further comprising: a second memory circuit configured to store a master jitter sequence, the master jitter sequence comprising a jitter value sequence uniformly distributed within a jitter value range, The timing controller is further configured to select the single shot jitter for each trigger pulse according to the main jitter sequence.

6. The LIDAR system of claim 5, further comprising: Control logic for randomly selecting a starting position in the main dithering sequence during a system startup operation.

7. The LIDAR system of claim 1 , wherein the processor is further configured such that analyzing the histogram to determine if crosstalk is present comprises: identifying the one or more time intervals as peak time intervals based on the number of photons in the one or more time intervals in the histogram exceeding a threshold; Calculating photon count statistics for time intervals other than the peak time interval, the photon count statistics comprising a global average photon count for each time interval and a standard deviation of the photon count for each time interval; determining a local average photon count for each time interval except the peak time interval; as well as comparing the local average photon count to the global average photon count to determine whether a difference between the local average photon count and the global average photon count is significant based on a threshold defined relative to a measure of natural variability in noise, The crosstalk is inferred based on whether the difference is significant.

8. The LIDAR system of claim 1, wherein the emitter emits a light pulse train consisting of one or more light pulses during each emission, and the light pulses of the light pulse train have a fixed time relationship with each other.

9. The LIDAR system of claim 1, wherein: The optical transmission module includes a plurality of transmitters; The light sensing module includes a plurality of light sensor channels; and The timing controller is configured such that the same single shot jitter is applied to each of the plurality of emitters and each of the plurality of light sensor channels.

10. A method of operating a LIDAR system, the method comprising: Capturing a plurality of shots, wherein capturing each shot of the plurality of shots comprises: Determine single shot jitter; In a timing controller of the LIDAR system, generating a trigger pulse at a time determined at least in part based on the single shot jitter; operating a light transmission module of the LIDAR system to emit a light pulse synchronized with the trigger pulse; and operating a light sensing module of the LIDAR system in synchronization with the trigger pulse to receive light and accumulate a histogram of intensity of the received light as a function of time, the histogram defining a series of time intervals, wherein the histogram is accumulated over the plurality of shots, and wherein for at least some of the plurality of shots, the single shot jitter is greater than one time interval; and After capturing the plurality of shots, calculating ranging information based at least in part on the histogram analyzing the histogram to determine if crosstalk is present; and The single-shot jitter is adaptively modified in response to determining that crosstalk is present.

11. The method of claim 10, wherein determining the single shot jitter for each shot comprises: The single shot jitter is selected within a predefined jitter value range using an equal energy sampling technique.

12. The method according to claim 11, wherein the predefined jitter value range is 0-20% of the total number of time intervals in the histogram.

13. The method of claim 11, wherein the predefined jitter value range is 0-120% of the total number of time intervals in the histogram.

14. The method of claim 10, wherein determining the single shot jitter for each shot comprises accessing consecutive positions in a master jitter sequence stored in a memory of the LIDAR system.

15. The method of claim 14, wherein determining the single shot jitter for each shot comprises randomly selecting a starting position in the master jitter sequence.

16. The method of claim 10, wherein analyzing the histogram to determine if crosstalk is present comprises: Based on the number of photons exceeding a threshold, identifying one or more time intervals in the histogram as peak time intervals; Calculating photon count statistics for time intervals other than the peak time interval, the photon count statistics comprising a global average photon count for each time interval and a standard deviation of the photon count for each time interval; determining a local average photon count for each time interval except the peak time interval; as well as comparing the local average photon count to the global average photon count to determine whether a difference between the local average photon count and the global average photon count is significant based on a threshold defined relative to a measure of natural variability in noise, The crosstalk is inferred based on whether the difference is significant.

17. The method of claim 10, wherein operating the light transmission module of the LIDAR system comprises operating a single emitter to emit a light pulse train consisting of one or more light pulses during each shot, the light pulses of the light pulse train having a fixed time relationship to each other.

Images