SYSTEM AND METHOD FOR SOLID-STATE LIDAR WITH ADAPTIVE BLOOMING CORRECTION - Patent application

Abstract

The LiDAR method includes configuring a laser array such that each laser generates a light beam having a FOV and intensity that illuminates the ROI, and configuring a detector array such that each detector receives light from the detector FOV within the ROI. The laser array is excited and light is received at the detector array from the illuminated ROI. The received light is processed with standard parameters for selected ones of the detectors to determine TOF data for selected ones of the detectors and generate an image of the object within the ROI. The TOF data of the selected detectors is analyzed to determine if a blooming criterion is met, and if so, the laser is re-excited. Light is then received at the detector array from the illuminated ROI and processed with parameters to determine TOF and intensity data that compensates for blooming.

Description

[Technical field]

[0001] The section headings used herein are for organizational purposes only and are not to be construed in any way as limiting the subject matter described in this application.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a non-provisional extension of U.S. Provisional Patent Application No. 63 / 312,356, entitled "System and Method for Solid-State LiDAR with Adaptive Blooming Correction," filed February 21, 2022. The entire contents of U.S. Provisional Patent Application No. 63 / 312,356 are incorporated herein by reference. [Background technology]

[0003] Introduction Autonomous, self-driving, and semi-autonomous cars use a combination of different sensors and technologies, such as radar, image recognition cameras, and sonar, to detect and locate surrounding objects. These sensors enable many improvements in driver safety, including collision warning, automatic emergency braking, lane departure warning, lane keeping assist, adaptive cruise control, and piloted driving. Among these sensor technologies, Light Detection and Ranging (LiDAR) systems play a key role, enabling real-time, high-resolution 3D mapping of the surrounding environment. Summary of the Invention [Means for solving the problem]

[0004] LiDAR systems need to be able to perform under a variety of conditions, including situations involving close and far objects and a variety of weather and ambient lighting condition combinations. It is important that the LiDAR can provide accurate object size information in these and other conditions. There is a need for adaptive LiDAR systems that can advantageously provide improved image and object identification characteristics as conditions change and evolve.

[0005] The present teachings, in accordance with preferred and exemplary embodiments, together with further advantages thereof, are more particularly described in the following detailed description taken in conjunction with the accompanying drawings. Those skilled in the art will appreciate that the drawings described below are for illustrative purposes only. The drawings are not necessarily to scale, emphasis instead being placed generally on illustrating the principles of the teachings. The drawings are not intended to limit the scope of the applicant's teachings in any way. [Brief description of the drawings]

[0006] [Figure 1A] FIG. 1A illustrates a schematic diagram of a known solid-state LiDAR system.

[0007] [Figure 1B] FIG. 1B illustrates a two-dimensional projection of the system field of view (FOV) of the LiDAR system of FIG. 1A.

[0008] [Figure 2A] FIG. 2A illustrates a two-dimensional projection of the laser FOV and detector FOV of a LiDAR system for one embodiment of a LiDAR system of the present teachings.

[0009] [Figure 2B] FIG. 2B shows a detector corresponding to a single laser FOV in a two-dimensional projection of the LiDAR system FOV of FIG. 2A.

[0010] [Figure 2C] FIG. 2C indicates the detector areas affected by blooming in the LiDAR system of FIG. 2A.

[0011] [Diagram 3] FIG. 3 illustrates a block diagram of one embodiment of an adaptive blooming identification and correction LiDAR system including separate transmitter and receiver systems connected to a host processor of the present teachings.

[0012] [Figure 4] FIG. 4 illustrates a process flow diagram of one embodiment of the adaptive blooming identification and correction LiDAR system of the present teachings.

[0013] [Diagram 5] FIG. 5 illustrates a process flow diagram for analyzing a region of interest to determine whether a blooming mode or a standard operating mode should be used for one embodiment of an adaptive LiDAR system of the present teachings.

[0014] [Figure 6] FIG. 6 illustrates a process flow diagram of a blooming mode of one embodiment of the adaptive LiDAR system of the present teachings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Description of Various Embodiments The present teachings will now be described in more detail with reference to exemplary embodiments thereof shown in the accompanying drawings. Although the present teachings are described in conjunction with various embodiments and examples, it is not intended that the present teachings be limited to such embodiments. On the contrary, the present teachings encompass various alternatives, modifications, and equivalents, as will be understood by those skilled in the art. Those skilled in the art with access to the teachings herein will recognize additional implementations, modifications, and embodiments within the scope of the present disclosure described herein, as well as other fields of use.

[0016] References herein to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the present teachings. Appearances of the phrase "in one embodiment" in various places in the specification do not necessarily all refer to the same embodiment.

[0017] It should be understood that the individual steps of the methods of the present teachings can be performed in any order and / or simultaneously so long as the teachings remain operable. Further, it should be understood that the apparatus and methods of the present teachings can include any number or all of the described embodiments so long as the teachings remain operable.

[0018] LiDAR systems for autonomous vehicles must be able to perform under various driving scenarios. For example, accurate range and image data, which may be in the form of a three-dimensional point cloud, should be acquired for a reflective traffic cone several meters away, as well as for the tires of a vehicle on the roadway 150 meters away. From an optical point of view, these two scenarios present substantially different characteristics, since the amplitude of the optical signal received from each object varies by several orders of magnitude. The traffic cone provides a very strong light return, while the distant tire produces a weak signal. The ratio between the maximum and minimum optical signals that can be reliably received by the LiDAR system is sometimes called the receiver dynamic range.

[0019] Direct time-of-flight (TOF) LiDAR systems that use single-photon avalanche diode (SPAD) detectors are one type of LiDAR system that presents a high dynamic range. SPAD devices are sensitive to single photons (the smallest possible optical power) but are not damaged by high optical power. Damage from high optical power can occur, for example, in conventional avalanche photodiodes (APDs). As a result of the high detector dynamic range, the SPAD continues to function whether the SPAD detector receives a single photon or the full power of the reflected laser transmitter. Generally, SPADs are formed in a two-dimensional array, and each SPAD detector in the detector's SPAD array is called a pixel.

[0020] One common configuration for LiDAR systems is to use a SPAD focal plane array composed of groups of pixels, each group having a common signal output, also referred to as a "multipixel". In this configuration, each multipixel incorporates several individual SPAD pixels grouped together. The multipixel has a single composite received signal output from all pixels in the group of multipixels, and this composite signal provides a measure of light intensity across several optical power ranges not provided by a single pixel output. SPAD arrays without a multipixel configuration would produce a binary image, since each individual SPAD pixel is essentially a digital device with two stable output states. That is, a SPAD pixel will either produce a very low quiescent current, or, when operating in saturation, produce a maximum current due to received photons triggering an avalanche. The transition between these two states is fast enough to provide an essentially digital output.

[0021] In contrast, 2D SPAD arrays using "multiple pixels" can be used to generate grayscale intensity images that provide additional information about the scene compared to binary images. With multipixels, not all SPADs saturate over several power levels, giving a graduated scale of intensity. This allows, for example, the ability to determine the reflectivity of objects and, in some conditions, to distinguish lane markings or read signs. In other words, with a multipixel configuration, it is possible to determine information about the intensity of the returns in addition to the time-of-flight of the returns.

[0022] However, at sufficiently high received optical power levels, all SPAD pixels in the multipixel become saturated simultaneously. In this limiting case, the distance accuracy from the time-of-flight determination is mostly preserved, but the imaging performance is degraded due to loss of contrast. As a result, at high input powers, the image can become more binary in nature, even with a multipixel SPAD.

[0023] Another issue beyond the loss of contrast is a phenomenon called "blooming", which is characterized by the image of an object appearing larger than the true object. Thus, the three-dimensional point cloud representation of the object is larger than the true object. As an example, road signs that use retroreflector materials present a classic case of highly reflective objects that often result in blooming for LiDAR systems. In the presence of blooming, reflective road signs may appear larger than normal or lose their shape, making it difficult to distinguish between stop signs and speed limit signs, for example.

[0024] In any imaging lens system, light from an infinitesimal point source (such as a star) forms a spot on the focal plane detector array with some finite radius and distribution of optical energy. The physical properties of light and the limitations of practical imaging systems do not allow a point source to result in an infinitesimal imaged spot on the focal plane array. When an image of a point source such as a star is brought to its sharpest focus by a lens system, the physics of diffraction and aberrations / distortions inherent in non-perfect lenses result in that optical energy being distributed over a pixel area of ​​some size.

[0025] In optics, the term encircled energy (EE) refers to a measure of the concentration of optical energy. Encircled energy is equal to the amount of energy in an imaged spot at a given radius and is often reported in microns. Typically, one number is reported for EE, corresponding to the radius where 80% of the optical energy is enclosed. For typical imaging lenses used in the LiDAR systems considered here, the EE ranges from 5 to 40 microns. The EE radius is larger than the size of a typical individual SPAD pixel in a focal plane detector array, which is about 10 microns in size for current generation technology. It is also important to note that the EE number only corresponds to 80% of the energy, and thus some optical power is present at larger radii, which can be considered as one type of stray light in LiDAR optics.

[0026] For LiDAR systems using SPAD detector arrays with sufficient sensitivity to detect single photons, the physics that results in light diffusion as characterized by encircled energy causes blooming. Accordingly, a highly reflective object that results in a large light reception power has a larger amount of stray light in a wider radius than an object of the same size with a lower reflectivity. If the stray light is high enough, a larger area of ​​pixels in the detector receives enough light to trigger an avalanche and result in TOF detection. This is undesirable because a highly reflective object makes the image appear larger than the actual object. That is, a processed received image, which may be a three-dimensional point cloud representation from a highly reflective object that causes blooming, will be larger than the true image size derived from a three-dimensional point cloud representation from an object without blooming. It should be understood that a three-dimensional point cloud representation from an object without blooming is considered to be a true three-dimensional point cloud representation of the object.

[0027] Traditional flash LiDAR systems employ an emitter that emits laser light over a wide FOV. Some flash LiDAR systems are solid-state. Flash LiDAR systems can illuminate an entire scene in a single illumination event. Flash LiDAR systems that use SPAD arrays can therefore suffer from significant blooming when the entire scene is illuminated at once, causing images of highly reflective objects to appear much larger than their actual size and, at the limit, filling the entire FOV.

[0028] Conventional flash LiDAR systems using SPAD arrays where the entire field of view (FOV) is illuminated at once can, in some cases, be completely blinded by blooming. Significant blooming, where every pixel in the array is subject to stray light, can cause the system to report an object as large as the entire FOV, when in fact the object may be a small highly reflective object such as an automobile taillight assembly or a reflective traffic cone. For autonomous vehicles utilizing LiDAR, it is highly undesirable for an object to appear larger than it actually is, as this can cause the vehicle to make larger than necessary path adjustments or brake or stop unnecessarily, depending on the object's range and reported image size derived from the object's three-dimensional point cloud representation. Either way, this can be an unsafe condition.

[0029] It is highly desirable for LiDAR systems that employ highly sensitive detector arrays to have the ability to determine when light blooming has occurred, and further, to have the ability to adaptively mitigate the effects of blooming on the reported image and TOF data.

[0030] The pulsed TOF LiDAR systems of the present teachings use collimated transmitter laser beams, and an illuminated laser FOV is created for each laser's optical beam. The laser FOV is much smaller in size compared to conventional flash LiDAR systems. In addition, the pulsed TOF LiDAR systems of the present teachings can use pulse averaging and / or pulse histogramming of multiple received laser pulses to improve the signal-to-noise ratio (SNR), thereby further improving range and performance. These LiDAR systems can also employ very high single-pulse frame rates that can well exceed 100 Hz.

[0031] FIG. 1A illustrates a schematic diagram of a known solid-state LiDAR system 100. The system illustrated in FIG. 1A does not employ a flash transmitter that illuminates the entire system field of view at once. A laser array 102 generates light beams in various patterns. When an emitter in the array 102 is activated by a control pulse, the emitter emits a light beam. One or more emitters are activated, possibly according to a specific sequence. The light beams from the lasers in the laser array 102 propagate through a common transmitter optics 104 that projects the light beam to a target 106 at a target plane 110. The target 106 in this particular example is an automobile 106, although it is understood that the target can be any object.

[0032] A portion of the light from the incident light beam is reflected by the target 106. These portions of the reflected light beam share the receiver optics 112. The detector array 114 receives the reflected light projected by the receiver optics 112. In various embodiments, the detector array 114 is solid-state with no moving parts. The detector array 114 typically has fewer individual detector elements than the transmitter array 102 has individual lasers. Each detector in the detector array 114 has a detector FOV at the target plane 110 based on its position, size, and the configuration of the receive optics 112.

[0033] The measurement resolution of the LiDAR system 100 is determined not by the size of the detector elements in the detector array 114, but by the number of lasers in the transmitter array 102 and the collimation of the individual light beams. In other words, the resolution is limited by the field of view of each light beam. A processor (not shown) in the LiDAR system 100 performs a time-of-flight (TOF) measurement that determines the distance of the target 106 from the light beams transmitted by the laser array 102 and detected at the detector array 114.

[0034] One feature of a LiDAR system according to the present teachings is that individual lasers and / or groups of lasers in the transmitter array 102 can be individually controlled. Each individual emitter in the transmitter array can fire independently, and the light beam emitted by each laser emitter corresponds to a 3D projection angle that occupies only a portion of the total system field of view. An example of such a LiDAR system is described in commonly assigned U.S. Patent Application Publication No. 2017 / 0307736. The entire contents of U.S. Patent Application Publication No. 2017 / 0307736 are incorporated herein by reference.

[0035] Another feature of the LiDAR system according to the present teachings is that the detectors and / or groups of detectors in the detector array 114 can also be individually controlled. This independent control over the individual lasers and / or groups of lasers in the transmitter array 102 and over the detectors and / or groups of detectors in the detector array 114 provides various desirable operating characteristics including control of the system's field of view, optical power levels, and scan patterns. In some embodiments, it is also possible to change the laser FOV, making it larger or smaller, or to move the relative position within the detector FOV at the target plane 110 by adjusting the relative positions of the transmission optics 104 and the laser array 104.

[0036] FIG. 1B illustrates a two-dimensional projection of the system field of view 150 of the LiDAR system described in connection with FIG. 1A. With reference to both FIG. 1A and FIG. 1B, the field of view of an individual detector in the detector array is represented by a small square 152. The illuminated measurement points associated with an individual emitter in the transmitter laser array 102 are illustrated by a circle 154. A single 3D measurement point in the total field of view of the LiDAR system of FIG. 1A is shown as a specific black circle 158 corresponding to a specific individual laser in the laser array. In FIG. 1B, it can further be seen that this measurement point is within an individual detector, and the field of view of that individual detector in the detector array 114 is shown within a square 156 with a cross-hatch pattern for identification. This diagram illustrates that the 3D resolution of some embodiments of the LiDAR system is determined by the number of lasers, since each laser corresponds to a specific angular projection angle that gives rise to the size of the circle 154 at the target range, as well as the relative size of the circle 154 and the square 152 representing the field of view of the individual detector element.

[0037] Thus, a desired field of view can be established by controlling specific individual lasers or groups of lasers in the transmitter array and / or individual detectors or groups of detectors in the receive array. Various system fields of view can be established using different relative fields of view for individual emitters or groups of emitters and / or individual detectors or groups of detectors. Fields of view can be established to yield specific and / or combinations of performance metrics. These performance metrics include, for example, improved signal-to-noise ratio, longer range or control range, eye-safe operating power levels, and lower or higher controllable resolution. Importantly, these performance metrics can be modified during operation to optimize LiDAR system performance.

[0038] The LiDAR system according to the present teachings uses an array drive control system that can provide selective control of specific laser devices in an array of laser devices to illuminate a target according to a desired pattern. The LiDAR system according to the present teachings can also use an array of detectors that generate detector signals that can be processed independently. Thus, a feature of the LiDAR system of the present teachings is the ability to provide various operational capabilities from a LiDAR system that has only electronic, non-mechanical or non-moving parts, including a fixed emitter array and a fixed detector array, with both transmission and reception light beams projected using shared transmission and reception optics. Such a LiDAR system configuration can result in a flexible system that is compact, reliable, and also relatively low cost.

[0039] The LiDAR system of the present teachings also utilizes a laser array, transmitter optics, receiver optics, and detector array as described in connection with the known system shown in FIG. 1A. However, these elements in the present teachings are selected and configured such that the two-dimensional projection of the system field of view is different. One feature of the present teachings is that the elements are configured such that the field of view of a single emitter is larger than the field of view of a single detector.

[0040] FIG. 2A illustrates a two-dimensional projection of a LiDAR system FOV 200 including a laser FOV 202 and a detector FOV 204 for one embodiment of a LiDAR system of the present teachings. The system illustrated in FIG. 2A is a projection of a LiDAR system having a laser array and transmission optics as described in connection with the LiDAR system of FIG. 1A and FIG. 1B. However, the elements are spaced and positioned to produce the LiDAR system FOV 200 as shown. Specifically, the array of lasers produces an array of beams having a circular FOV with a particular size represented by 16 circles, i.e., the laser FOV 202, as shown. Various embodiments produce laser beam FOVs 202 of various shapes depending on the emitter and projection optics.

[0041] The LiDAR system FOV 200 shown in FIG. 2A is generated by a 4×4 (16) laser array. The divergence / collimation of the lasers is selected such that there is only enough overlap such that there are no “gaps” in the field of view. That is, the laser FOVs 202 overlap to form a 4×4 array. The detector array provides an array of square detector FOVs 204 having a particular size, represented by 256 squares. The area of ​​each detector FOV 204 represented by a square may be referred to as a pixel. It can be seen that there are 16×16 (256) detectors with substantially continuous coverage across the array. It should be understood that the number of lasers and detectors, as well as the particular size and shape of the FOV of the emitter and detector elements, are selected to illustrate features of the present teachings and do not necessarily represent an actual system.

[0042] 2A , the number of detectors (256) exceeds the number of lasers (16). This embodiment represents an important use case of a LiDAR system according to the present teachings, where the FOV of the laser emitter FOV 202, represented by a circle, covers the FOV of several detector FOVs 204, represented by squares.

[0043] Various detector technologies can be used to build detector arrays for LiDAR systems according to the present teachings. For example, single photon avalanche diode detector (SPAD) arrays, avalanche photodiode (APD) arrays, silicon photomultiplier arrays (SPA) can be used. The detector size not only sets the resolution by setting the FOV of a single detector, but also relates to the speed and detection sensitivity of each device. State-of-the-art two-dimensional arrays of detectors for LiDAR are already approaching the resolution of VGA cameras and are expected to follow a trend of increasing pixel density similar to that seen in CMOS camera technology. Thus, it is expected that even smaller sizes of detector FOV 204, represented by a square, will be realized over time. For example, an APD array with 264,000 pixels (688(H) × 384(V)) was recently reported in the paper “A 250m Direct Time-of-Flight Ranging System Based on a Synthesis of Sub-Ranging Images and a Vertical Avalanche Photo-Diodes (VAPD) CMOS Image Sensor”, Sensors 2018, 18, 3642.

[0044] FIG. 2B points out the detectors corresponding to a single laser FOV in a two-dimensional projection of the LiDAR system FOV 250 of FIG. 2A. Similar to the LiDAR system FOV 200 shown in FIG. 2A, the single laser FOV 202 is represented by a circle and the single detector FOV 204 is represented by a square. The lasers of interest are excited by a controller to illuminate the FOV 252 represented by a particular circle. The controller generates bias signals to excite the desired laser or lasers at the desired times. The detector FOVs that overlap at least a portion of the laser beam FOV represented by the circle are within a shaded region 254 in the system FOV. In this particular configuration, a detector region 254 including 32 individual detector FOVs is realized for the single laser beam FOV 252. Each detector FOV in the detector region 254 has a detector FOV associated with a small square in the region 254. It should be noted that in various embodiments, the detector region 254 is not necessarily square or rectangular. The shape of region 254 depends on the detector shape and the laser beam profile, both of which can be any shape (circular, square, or other).

[0045] The controller selects a set of one or more detectors in the region 254 that falls within the laser beam FOV 252 of the selected laser. The signals from the selected set of detectors are detected simultaneously, and the detected signals are provided to the controller and then processed to generate one or more measurement signals. The LiDAR system can measure the TOF for all detectors in the selected set independently and simultaneously, or alternatively, the system can combine the signal outputs from the selected detectors to form a single measurement. Also, each detector may be either single pixel or multi-pixel, as previously described. For long-range operation, including operation at the longest specified range of the LiDAR system, the number of pixels (i.e., individual detectors) used to generate the measurement pulses may be selected to maximize the SNR at the expense of resolution. For example, the best SNR may correspond to a measurement made by summing or combining in some way the received signals from all detectors in the region 254 highlighted in FIG. 2B. That is, multiple adjacent detectors that fall within the laser FOV 252 of the selected laser may be selected. In some embodiments, only detectors that are entirely illuminated by light within the laser FOV 252 of the laser are selected. In that way, noise from detectors that are not entirely illuminated is not accumulated. Alternatively, a smaller subset of detectors may be selected. For example, in some configurations according to the present teachings, the power from the laser is not uniformly distributed across the beam profile. In these configurations, a subset of detectors that matches the beam profile can be used such that detectors that receive higher intensity incident light are selected.

[0046] Blooming can occur in the LiDAR system described by FIG. 2A and FIG. 2B. For example, in FIG. 2B, the top half and corners of the car are illuminated by the laser FOV 252. The car's headlight assemblies are highly reflective at some angles as they act as parabolic mirrors, producing very high optical power returning to the LiDAR system. In this situation, blooming can occur and all detectors corresponding to the laser FOV 252 can receive a signal high enough to trigger an avalanche response. The corresponding measurement image looks like the entire shaded area for all detectors in region 254 when all detectors are driven to near saturation. All lasers in the system that intersect with one of the two headlights can present blooming. In the worst case, the size of the car corresponds to all detectors of the four central lasers, each overlapping with one headlight assembly.

[0047] FIG. 2C points out the detector regions affected by blooming in the LiDAR system of FIG. 2A. A depiction of a worst-case blooming image is shown in FIG. 2C, with all detectors corresponding to the four central laser FOVs shown in the shaded region 256. The apparent size of the car in the image is now more than six times the area of ​​the car's actual size. An autonomous vehicle receiving such information would need to steer a wider path than would be necessary to avoid the oncoming vehicle. Also, considering that the reported object is much larger than its true size, it may not be clear that it is a vehicle, which can cause further uncertainty in path planning and response.

[0048] FIG. 2C also shows that blooming is limited to the area of ​​the detector associated with the laser that illuminates the headlight. The size of the blooming is limited by the size of the laser in this type of LiDAR system. In general, the smaller the physical size of an individual laser, the smaller the corresponding maximum blooming associated with that laser. Reduction of the maximum worst-case blooming compared to that seen in FIG. 2C can be done by increasing the number of individual lasers, as well as by reducing the size and overlap of the lasers by changing the size and / or shape of the laser FOV. For example, if the lasers were non-overlapping rectangular instead of circular, the blooming would correspond to only two lasers instead of four, as shown in the shaded area 256 in FIG. 2C. Also, reducing the area of ​​the laser FOV by a factor of two would require 64 lasers instead of 16, greatly reducing the area of ​​blooming. It can be seen that LiDAR systems using highly collimated laser beams with small individual FOVs are less susceptible to blooming than flash LiDAR systems where a large portion of the entire FOV is illuminated simultaneously. Of course, cost and complexity can increase with the number of lasers the system uses. Instead of, or in addition to, increasing the number of lasers, it may be preferable for the system to be able to adapt in some way to detect and reduce blooming.

[0049] A LiDAR system according to the present teachings has the ability to determine that blooming is potentially occurring and can then take steps to adapt system parameters to reduce the effects of blooming on the reported image and TOF data that can be based on a three-dimensional point cloud representation. The LiDAR system of the present teachings implements a set of decision criteria for determining whether blooming is potentially occurring.

[0050] 3 illustrates a block diagram of one embodiment of an adaptive blooming compensation (ABC) LiDAR system 300 including a transmitter system and a receiver system connected to a host processor in accordance with the present teachings. The LiDAR system 300 has six main components: (1) controller and interface electronics 302, (2) transmit electronics 304 including a laser driver, (3) laser array 306, (4) receive and time-of-flight and intensity calculation electronics 308, (5) detector array 310, and (6) in some embodiments, an optical monitor 312 and a splitter 311 that provides a portion of the transmitted light to the monitor 312.

[0051] The LiDAR system's controller and interface electronics 302 control the overall function of the LiDAR system and provide digital communications to a host system processor 314. The transmission electronics 304 controls the operation of the laser array 306 and, in some embodiments, sets the pattern and / or power of the laser emission of individual elements in the array 306.

[0052] The reception and time-of-flight calculation electronics 308 receive electrical detection signals from the detector array 310 and then process these electrical detection signals to calculate distance by time-of-flight calculation. Intensity information can also be processed. The reception and time-of-flight calculation electronics 308 can also control the pixels of the detector array 310 to select a subset of pixels to be used for a particular measurement. The strength of the return signal is also calculated in the electronics 308. In some embodiments, the reception and time-of-flight calculation electronics 308 determines whether the return signal from a region of interest in the target plane indicates the occurrence of blooming.

[0053] In some embodiments, the controller and interface electronics 308 determines whether the return signal from the region of interest indicates blooming. Also, in some embodiments, the host system processor 314 determines whether the return signal from the region of interest indicates blooming. In some embodiments, the transmission controller 304 controls pulse parameters such as pulse amplitude, pulse width, and / or pulse delay.

[0054] It should be understood that the block diagram of the LiDAR system 300 in FIG. 3 illustrates connections between components and is not intended to limit the physical structure in any way. The various elements of the system can be physically located in various locations depending on the embodiment. Additionally, the elements can be distributed in various physical configurations depending on the embodiment. Additionally, the block diagram of the LiDAR system 300 is not intended to limit which particular electronic components are used to perform the adaptive blooming correction, nor is it intended to limit how many electronic components are used to perform the adaptive blooming correction.

[0055] FIG. 4 illustrates a process flow chart 400 of an embodiment of an adaptive blooming identification and correction LiDAR system of the present teachings. The process flow chart 400 describes the overall process steps involved in an embodiment of an adaptive blooming correction process. The LiDAR system described in connection with the process flow chart 400 has a default set of standard operating conditions, including standard parameters used in processing TOF and intensity data for subsequent reporting to a host system processor, and operates normally under the standard operating conditions. In a first step 402, the LiDAR system acquires TOF and intensity data over a region of interest using the standard parameters. The region of interest can be, for example, a portion of the FOV, or the entire FOV, or a portion of a frame, or the entire frame.

[0056] In a second step 404, the system performs an analysis of at least a portion of the acquired TOF, intensity, and / or return pulse characteristic data to determine blooming conditions. In some embodiments, in the second step 404, the system performs an analysis of the TOF and intensity data contained in a region of interest (ROI) and creates an estimate of whether blooming is occurring. The estimate can be a probability or some other metric that correlates to the occurrence of blooming. The system then compares the estimate to some blooming criteria that can be determined at the time the system is manufactured or adaptively determined during system operation. In a third decision step 406, the system determines whether blooming is occurring based on whether the blooming criteria are met. If the blooming occurrence criteria are not met, the system returns to the first step 402 and continues to operate under standard operating conditions. If the blooming criteria are met, the system proceeds to a fourth step 408 and modifies operation to use blooming parameters.

[0057] In some embodiments, in a fourth step 408, the system changes the operating mode to account for the possibility that blooming is occurring and reports the received TOF, intensity, and / or return pulse characteristic data processed using blooming mode parameters. In the process flow chart 400, the operating mode is referred to as "blooming mode". During blooming mode operation, the operating parameters of the system are different from the standard operating parameters. For any region of interest during blooming mode operation, the system performs an analysis of the TOF and intensity data contained in the ROI and makes an estimate of whether blooming is occurring in a fifth decision step 410. If the blooming occurrence criteria are not met, the system switches to standard operating conditions and returns to the first step 402. If the blooming occurrence criteria are met, the system continues to operate under the blooming mode operating parameters by returning to the fourth step 408.

[0058] FIG. 5 illustrates a process flow chart 500 for analyzing a region of interest to determine whether a blooming mode or a standard operating mode should be used for an embodiment of an adaptive LiDAR system of the present teachings. More specifically, the process flow chart 500 illustrates method steps of an embodiment of an analysis performed on TOF, intensity, and / or return pulse characteristic data from the region of interest to determine whether the operating mode used should be blooming mode or standard mode. For example, the steps of the process flow chart 500 can be used in steps 406 and / or 410 of the process flow 400 described in connection with the process flow chart of FIG. 4. In these embodiments, the transition to steps 406, 410 initiates an analysis start 502. This start 502 transitions to a first step 504 of calculating a set of estimates of the occurrence of blooming using the methods described herein. An optional second step 506 is to adaptively calculate a criterion to which the set of estimates from step 504 is applied. In some methods according to the present teachings, the criteria for the second step 506 may need to be adaptively determined for a wide range of operating conditions that the LiDAR system operates in. For example, different criteria may be used when the ambient light is low (nighttime conditions) and when the ambient light is high (bright sunlight conditions).

[0059] The third step 508 is a decision step in which the set of estimates of blooming occurrence is compared to a criterion. If the criterion is met, the analysis algorithm reports to the system that the blooming mode should be used, at 510. If the criterion is not met, the analysis algorithm reports to the system that the standard operating mode should be used, at 512. The analysis then ends. For example, in the embodiment of process 400 of FIG. 4, reporting the use of blooming mode, at 510, triggers a transition to the fourth step 408, and reporting the use of standard mode, at 512, triggers a transition to the first step 402.

[0060] A first set of criteria can be inferred from the description associated with FIG. 2B. In this LiDAR system FOV 250, there is a known mapping from the region of detectors 254 to the laser 252. One possible set of criteria for the occurrence of blooming is that all detectors in the region 254 report similar TOFs and the strength of the received signal from the detectors is either saturated or close to saturation. Also, instead of requiring all detectors in the region to have similar TOFs and near-saturation intensities, it may be a defined subset of detectors. In some situations, it may be important to further extend the number of detectors beyond the region 254. If the criteria of similar TOFs at near-saturation intensities are met over any defined region of the image, the system assumes that blooming is probably occurring and performs some function to accommodate the blooming.

[0061] One common boundary or edge condition is the situation when the LiDAR system is looking at a "wall", so that the true image is the same TOF for all detectors looking at the wall, and the wall may be a "white" wall that is sufficiently reflective to lead to nearly saturation of all detectors. Thus, in this condition, one cannot know with 100% confidence that blooming has definitely occurred, only that there is a strong possibility. Additional measures or adaptations by the LiDAR system are required to confirm the blooming and also to determine the extent of the blooming to the extent possible under such conditions. In addition to TOF and intensity, other criteria, such as those based on the pulse width of the received signal between detectors in any defined area, can also be applied.

[0062] Some embodiments of the adaptive blooming LiDAR system of the present teachings use a blooming mode of operation in which two or more sets of system parameters are employed. Each set of system parameters can be a fixed set, or in various embodiments, one or more sets of parameters can be adaptively determined. The different sets of system parameters used by the LiDAR system can be for multiple performance improvements including range, resolution, frame rate, intensity, reflectometry, stray light correction, and blooming correction, among other factors.

[0063] For example, in one simple embodiment, a second set of system parameters in blooming mode is used that reduces the laser output power or excited laser light transmission power used to measure the FOV. In this example, the first set of system parameters has a higher optical power that allows for the longest range of the system. The second set of parameters reduces the optical power so that at closer ranges there is less chance of the detector saturating. It should be understood that the sets of system parameters do not have to be used evenly. The second or additional sets of system parameters may be used only a small percentage of the time. A set of system parameters with lower optical power may, for example, allow for less saturation of the intensity image at closer ranges, thereby allowing better identification of items such as lane markings or letters on a sign. The intensity, TOF, and / or pulse characteristic data from the second set of system parameters may be compared to the same data from the first set of performance parameters, and areas of significant differences may be identified as areas where blooming may be occurring.

[0064] The above description illustrates possible modes of operation in which two or more sets of system parameters are used. Another mode of operation may involve adaptive changes to averaging or histogramming parameters used to process TOF and / or intensity data. One feature of the methods and apparatus of the present teachings is that the systems and methods are configured such that two (or more) data sets available periodically may be compared to each other. Thus, differences in intensity, TOF, and / or pulse width of these two data sets may indicate potential blooming, from which the system takes additional steps to confirm and correct the effects of blooming. The two data sets may be obtained, for example, from two different detector FOVs within the region of interest, two different laser FOVs within the region of interest, two different optical transmission powers of an excited laser within a single laser FOV within the region of interest, and various combinations of these and other parameters.

[0065] Once a LiDAR system has determined that blooming is possible, various algorithms can be used to confirm the blooming and reduce its effect on the reported data. One class of algorithms relies on illuminating the detector with a lower optical power to obtain measurements. By lowering the illumination so that the detector does not saturate, the blooming effect is reduced, thus making it possible to distinguish affected areas.

[0066] One way to reduce the illumination on a particular detector is to perform measurements on that detector using a laser with an FOV adjacent to the laser used for the standard TOF measurement. Due to the divergence and parallax of the laser beams, there is some amount of optical power from the adjacent laser that extends into the FOV of the adjacent detector region. This optical power provides such a weak illumination that the detector in the blooming region is no longer saturated. The adjacent lasers may be from the same laser array or may be from laser arrays where the FOV of the two laser arrays are optically interleaved. Measurements with the adjacent lasers can be obtained independently or simultaneously while scanning is being performed for the primary laser. When scanning with the primary laser, data can be obtained for both the detector in the main FOV of the primary laser and the adjacent detector. The LiDAR system performs a comparison of the TOF measurements of each detector obtained with the primary and adjacent lasers. For any measurements where the TOF is the same for the two data sets, a conclusion can be made that the object is "real".

[0067] If the TOF is a different measurement for the two data sets, a conclusion can be made that the difference corresponds to the presence of blooming. In areas where blooming is determined, the system can use measurement data from adjacent lasers, or if the system is recording multiple echoes, the echo from the first data set that corresponds to the blooming region can be removed from the data set and the system reports only the other echoes. Alternatively, once blooming is determined, the system can report all of the measurement data without removing the effects of blooming, and instead issue a flag or error indicating which data is likely affected by blooming, and a higher level system can decide how to use the data.

[0068] In another embodiment of the method, the transmitter power (excited laser optical transmission power) is reduced. In this embodiment, measurements are made with the primary laser, and the optical transmission power from the primary laser is reduced to a level where saturation in the region of interest does not occur. In this case, the LiDAR system must incorporate a laser driver circuit that can modify the current and / or voltage applied to the laser. One advantage of using a primary laser compared to an adjacent laser is the more direct control over the optical power used.

[0069] One possible drawback of using a primary laser compared to an adjacent laser is that lower power measurements cannot be taken simultaneously with higher power (primary) measurements. However, in systems that use multiple laser pulses to generate averages or histograms, the impact can be minimized by using lower power for either a few or a single pulse in the overall set of laser pulses. There may be an impact on SNR, but the impact can be small depending on the system parameters. For example, if 32 laser pulses were used for an average or histogram measurement and only one of those 32 pulses corresponded to a lower power, the impact on SNR and range would be negligible. Also, the low power pulses could be implemented only for the region of interest rather than the entire frame, further mitigating any frame rate and range effects. This method requires a laser driver that can change the laser power (voltage / current) on individual lasers in the array for short periods of time, sometimes even between pulses (<5 μsec). The system then performs a comparison of TOF and intensity measurements taken at standard operating mode optical power and reduced optical power. Any measurements where the TOF is the same can lead to the conclusion that the object is "real". Any measurements where there are different TOFs can lead to the conclusion that blooming is present. In areas where blooming is present, the system will remove returns corresponding to the areas of blooming and / or flag or report with errors those data points where blooming is likely to occur.

[0070] In yet another embodiment of the method, the optical pulse width is analyzed. LiDAR systems often use the pulse width as a simple noise filter. For example, a pulse width as long as the gating time of a TOF measurement does not correspond to any real physical object. Similarly, very small pulse widths can often be distinguished as electronic noise. A simple pulse width noise filter with a maximum and minimum pulse width is easily implemented. In the case of blooming, the pulse width can also be used in the blooming determination method. The laser pulse that reflects off the object and returns the image to the corresponding centroid position on the detector array has the maximum optical power density. In this case, the pulse width received by the detector at the corresponding centroid position will be the largest since the rise time, fall time, and length of the pulse are all maximized by the received power level. In the area affected by blooming, the light being received is, by definition, outside the centroid of the received optical signal, and the optical power will be reduced even if the detector is saturated. In this case, the rise time and fall time are expected to slow down, and the length of the pulse should also be reduced. Thus, by comparing the pulse widths of pixels that are within the blooming region, pixels associated with the true extent of the object will have a longer pulse width than pixels that are affected by blooming. However, many factors can affect the pulse width. One feature of the present teachings is that various adaptive algorithms can be implemented to take into account these various factors to determine the appropriate pulse width criteria.

[0071] Another embodiment of the adaptive blooming algorithm according to the present teachings utilizes a matrix-addressable or individually addressable transmitter with the ability to change the illuminated FOV. One-way blooming can occur when the transmitter FOV is large enough such that a portion of the laser beam strikes a highly reflective object. For a laser beam with a smaller FOV, the smaller divergence will intersect a smaller portion of the scene, thus mitigating the possibility of blooming.

[0072] FIG. 6 illustrates a process flow diagram 600 of a blooming mode of an embodiment of an adaptive LiDAR system of the present teachings. Once blooming mode operation is initiated in 602, the first step 604 is for the system to select a set of operating modes, which may be a set of operating parameters based on the blooming estimate occurrence metric and / or blooming criteria. Possible operating modes are described herein. In a second step 606, using the selected operating mode / parameters of the first step 604, the system then performs TOF and intensity measurements for the determined region of interest. The determined region of interest in this case may correspond exactly to the region of interest used to determine the metric for the occurrence of blooming, or a different region of interest may be selected in response to the selection of the operating mode. Once the measurement data from the selected operating mode is acquired, the system uses an algorithm to determine a corrected set of TOF and intensity data to be reported to the system in a third step 608, where this corrected data set is expected to have been corrected in some way for the occurrence of blooming. The process flow diagram 600 ends in step 610, where the system transitions to a standard operating mode as described herein.

[0073] There are design tradeoffs between transmitter FOV and detector FOV. One tradeoff is frame rate and scan time. Another tradeoff is that, all other aspects being equal, a smaller transmitter FOV slows down the frame rate. Therefore, it is not always desirable to have a small transmitter FOV. One embodiment of the present teaching is to adapt the FOV when blooming is detected, adjusting the transmitter FOV for the entire frame, or ideally only for the portion of the FOV affected by blooming. In this situation, the blooming region uses the TOF and intensity data associated with the smaller transmitter FOV to reduce blooming.

[0074] Another feature of the present teachings is that the LiDAR system can adapt in response to blooming conditions, although some embodiments can also adapt in response to non-blooming conditions via a similar set of analysis steps described herein. In this situation, instead of adapting the system to reduce the laser area in response to a blooming condition, the illuminated laser area can be increased by transitioning from standard operation to a "no blooming" operating condition when there is no blooming in the scene at all.

[0075] Equivalent While the applicant's teachings have been described in conjunction with various embodiments, it is not intended that the applicant's teachings be limited to such embodiments. On the contrary, the applicant's teachings encompass various alternatives, modifications, and equivalents, which can be made without departing from the spirit and scope of the teachings, as will be appreciated by those skilled in the art.

Claims

1. 1. A method for light detection and ranging (LiDAR), the method comprising: a) configuring a laser array such that each laser in the laser array, upon excitation, produces a light beam having a laser field of view and an intensity that illuminates a region of interest; b) configuring a detector array such that each detector in the detector array receives light from a detector field of view; c) exciting at least one laser in the laser array; and d) receiving light from the illuminated region of interest of the at least one excited laser in the laser array with the detector array, at least one detector in the detector array having a detector field of view located within the laser field of the at least one excited laser in the laser array; e) processing received light for selected ones of the detectors in the detector array with standard parameters to determine time-of-flight data for the selected ones of the detectors in the detector array and generate a three-dimensional point cloud representation of an object positioned within the illuminated region of interest of the excited at least one laser in the laser array; f) analyzing the determined time-of-flight data for the selected ones of the detectors in the detector array to determine whether the processed received light satisfies a blooming criterion indicating that the three-dimensional point cloud representation of the object is larger than a true three-dimensional point cloud representation of the object; g) re-exciting the at least one laser in the laser array if the processed received light satisfies the blooming criterion; h) receiving light from the illuminated region of interest of the re-pumped at least one laser in the laser array with the detector array, at least one detector in the detector array having a detector field of view located within the laser field of the re-pumped at least one laser in the laser array; i) processing the received light for selected ones of the detectors in the detector array with a blooming parameter to determine time-of-flight data for the selected ones of the detectors in the detector array that at least partially compensates for the three-dimensional point cloud representation of the object being larger than the true three-dimensional point cloud representation of the object, thereby generating a more accurate image of the object; A method comprising:

2. 2. The method of claim 1, wherein processing the received light for selected ones of the detectors in the detector array with standard parameters further determines intensity data for the selected ones of the detectors in the detector array to generate the three-dimensional intensity image of an object positioned within the illuminated region of interest of the excited at least one laser in the laser array.

3. 3. The method of claim 2, further comprising: analyzing the determined intensity data for the selected ones of the detectors in the detector array to determine whether the processed received light satisfies the blooming criterion.

4. 3. The method of claim 2, wherein processing the received light for selected ones of the detectors in the detector array further comprises processing the received light for selected ones of the detectors in the detector array with a blooming parameter to determine time-of-flight and intensity data for the selected ones of the detectors in the detector array that at least partially compensates for the three-dimensional point cloud representation of the object being larger than the true three-dimensional point cloud representation of the object.

5. 4. The method of claim 3, wherein processing the received light for selected ones of the detectors in the detector array further comprises processing the received light for selected ones of the detectors in the detector array using parameters to determine time-of-flight and intensity data for the selected ones of the detectors in the detector array that at least partially compensates for the three-dimensional point cloud representation of the object being larger than the true three-dimensional point cloud representation of the object.

6. 2. The method of claim 1 , wherein analyzing the determined time-of-flight data for the selected ones of the detectors in the detector array to determine whether the processed received light meets the blooming criterion comprises comparing time-of-flight data associated with the excited at least one laser.

7. The method of claim 1 , wherein exciting the at least one laser in the laser array comprises exciting a first and a second laser in the laser array.

8. The method of claim 7 , wherein the first and second lasers are adjacent to one another in the laser array.

9. 9. The method of claim 8, wherein analyzing the determined time-of-flight data for the selected ones of the detectors in the detector array to determine whether the processed received light meets the blooming criterion comprises comparing time-of-flight data associated with the excited first and second lasers.

10. The method of claim 3 , wherein exciting the at least one laser in the laser array comprises exciting a first and a second laser in the laser array.

11. The method of claim 10 , wherein the first and second lasers are adjacent to one another in the laser array.

12. 12. The method of claim 11 , wherein analyzing the determined time-of-flight and intensity data for the selected ones of the detectors in the detector array to determine whether the processed received light meets the blooming criterion comprises comparing intensity data associated with the excited first and second lasers.

13. 4. The method of claim 3, wherein exciting the at least one laser in the laser array comprises exciting the at least one laser in the laser array to produce two different optical transmit powers at two different times.

14. 14. The method of claim 13, wherein analyzing the determined time-of-flight and intensity data for the selected ones of the detectors in the detector array to determine whether the processed received light meets the blooming criterion comprises comparing the processed received light at the two different optical transmit powers generated at the two different times.

15. 2. The method of claim 1 , wherein analyzing the determined time-of-flight data for the selected ones of the detectors in the detector array to determine whether the processed received light meets the blooming criterion includes determining pulse characteristics of the processed received light.

16. The method of claim 15 , wherein the pulse characteristics include a pulse width.

17. The method of claim 15 , further comprising comparing the determined characteristic of the processed received light to a predetermined pulse characteristic.

18. 4. The method of claim 3, wherein analyzing the determined time-of-flight and intensity data for the selected ones of the detectors in the detector array to determine whether the processed received light meets the blooming criterion comprises determining pulse characteristics of the processed received light.

19. The method of claim 18 , wherein the pulse characteristics include a pulse width.

20. The method of claim 18 , further comprising comparing the determined pulse characteristics of the processed received light to predetermined pulse characteristics.

21. The method of claim 1 , wherein the blooming criteria varies with time.

22. The method of claim 1 , wherein the blooming criterion varies with ambient light conditions.

23. The method of claim 1 , wherein the blooming criteria varies with weather conditions.

24. 2. The method of claim 1, wherein processing the received light with the blooming parameters for selected ones of the detectors in the detector array to determine time-of-flight data for the selected ones of the detectors in the detector array that at least partially compensates for the three-dimensional point cloud representation of the object being larger than the true three-dimensional point cloud representation of the object includes reducing a laser field of view for at least one of the multiple lasers in the laser array.

25. 4. The method of claim 3, wherein processing the received light with the blooming parameters for selected ones of the detectors in the detector array to determine time-of-flight data for the selected ones of the detectors in the detector array that at least partially compensates for the three-dimensional point cloud representation of the object being larger than the true three-dimensional point cloud representation of the object includes reducing a laser field of view for at least one of the multiple lasers in the laser array.

26. 4. The method of claim 3, wherein processing the received light with the blooming parameters for selected ones of the detectors in the detector array to at least partially compensate for the three-dimensional point cloud representation of the object being larger than the true three-dimensional point cloud representation of the object comprises reducing the optical transmission power generated by at least one of the plurality of lasers in the laser array.

27. 2. The method of claim 1, wherein processing the received light with the blooming parameters for selected ones of the detectors in the detector array to determine time-of-flight data for the selected ones of the detectors in the detector array that at least partially compensates for the three-dimensional point cloud representation of the object being larger than the true three-dimensional point cloud representation of the object includes reducing a pulse width generated by a laser of the plurality of lasers.

28. 4. The method of claim 3, wherein processing the received light with the blooming parameters for selected ones of the detectors in the detector array to determine time-of-flight and intensity data for the selected ones of the detectors in the detector array that at least partially compensates for the three-dimensional point cloud representation of the object being larger than the true three-dimensional point cloud representation of the object includes reducing a pulse width generated by a laser of the plurality of lasers.

29. 2. The method of claim 1, wherein processing the received light for selected ones of the detectors in the detector array with the blooming parameters to determine time-of-flight data for the selected ones of the detectors in the detector array that at least partially compensates for the three-dimensional point cloud representation of the object being larger than the true three-dimensional point cloud representation of the object includes processing the received light of detectors corresponding to detector fields of view that are located outside the laser field of view of the at least one excited laser in the laser array.

30. 4. The method of claim 3, wherein analyzing the determined time-of-flight and intensity data for the selected ones of the detectors in the detector array comprises comparing the processed received light of detectors corresponding to detector fields of view located inside the laser field of view of the excited at least one laser in the laser array with the processed received light of detectors corresponding to detector fields of view located outside the laser field of view of the excited at least one laser in the laser array.

31. 31. The method of claim 30, wherein processing the received light for selected ones of the detectors in the detector array with the blooming parameters to determine time-of-flight data for the selected ones of the detectors in the detector array that at least partially compensates for the three-dimensional point cloud representation of the object being larger than the true three-dimensional point cloud representation of the object includes processing the received light for detectors corresponding to detector fields of view that are located outside the laser field of view of the at least one excited laser in the laser array.

Images