Method and apparatus for reducing halation in LiDAR systems

Abstract

The LiDAR receiver includes an optical element configured with a focal power to project light received at an input to a region where the received light overlaps. The apodized aperture in the region where the received rays overlap is configured with an optical transmission profile that varies radially across the apodized aperture to provide a desired modulation of the received overlapping rays. A detector optic positioned adjacent the apodization aperture focuses the modulated light on a detector plane. A two-dimensional pixelated detector array is positioned on a detector plane. The detector optics and the desired modulation are selected such that the ratio of the intensity of the modulated light on one pixel of the two-dimensional pixel array to the intensity of the modulated light on an adjacent pixel of the two-dimensional pixel array is a desired value.

Description

[0001] The section headings used herein are for organizational purposes only and should not be construed in any way as limiting the subject matter described in this application.

[0002] Cross-references to related applications

[0003] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 496,061, filed April 14, 2023, entitled "Method and Apparatus for Reducing Halo Effects in LiDAR Systems". The entire contents of that U.S. Provisional Patent Application No. 63 / 496,061 are incorporated herein by reference.

[0004] introduce

[0005] Autonomous, self-driving, and semi-autonomous vehicles, as well as many other stationary and mobile systems that require mapping of their surroundings, use combinations of different sensors and technologies, such as radar, image recognition cameras, and sonar, to detect and locate objects in their environment. These sensors enable a range of improvements in driver safety, including collision warning, automatic emergency braking, lane departure warning, lane keeping assist, adaptive cruise control, and autonomous driving. Among these sensor technologies, light detection and ranging (LiDAR) systems play a crucial role, enabling real-time, high-resolution 3D mapping of the surrounding environment.

[0006] LiDAR systems need to be able to operate under a variety of environmental and driving conditions, including various combinations of near and far distances containing objects, as well as diverse weather and ambient lighting conditions. Importantly, LiDAR must be able to provide accurate object size information under these and other conditions. Data collection from various optical sensors (including those used in LiDAR systems) leads to the need for optical systems and signal processing that compensate for optical artifacts that distort optical images, such as blooming and ghosting. Attached Figure Description

[0007] The present teachings and their further advantages, according to preferred and exemplary embodiments, will be described in more detail below with reference to the accompanying drawings. Those skilled in the art will understand that the drawings described below are for illustrative purposes only. The drawings are not necessarily drawn to scale; rather, they generally focus on illustrating the principles of the present teachings. The drawings are not intended to limit the scope of the applicant's teachings in any way.

[0008] Figure 1 A partial pixel map of a single-photon avalanche diode (SPAD) detector is shown, illustrating the cause of unwanted halos in the receiver of a known LiDAR system.

[0009] Figure 2AThis is a schematic diagram of a LiDAR receiver, showing the receiver optics and SPAD detector array.

[0010] Figure 2B This is a diagram of the optical beam profile at the SPAD detector array.

[0011] Figure 3 A schematic diagram of a LiDAR receiver with an apodized filter configured according to this teaching is shown.

[0012] Figure 4 A graph showing a comparison of optical power transmission across aperture as a function of distance for a standard aperture stop of a known system and optical power transmission of an apodized aperture according to this teaching is presented.

[0013] Figure 5 A graph showing a comparison of the relative intensities of light from a single point target after passing through a conventional aperture and after passing through an apodization aperture filter according to this teaching, focused and concentrated on the pixels of the SPAD detector array. Detailed Implementation

[0014] The present teachings will now be described in more detail with reference to exemplary embodiments shown in the accompanying drawings. Although the present teachings are described in conjunction with various embodiments and examples, they are not intended to limit the present teachings to such embodiments. Rather, the present teachings encompass various alternatives, modifications, and equivalents as will become apparent to those skilled in the art. With the benefit of the present teachings, those skilled in the art will recognize additional implementations, modifications, and embodiments, as well as other areas of application, all of which are within the scope of this disclosure as set forth herein.

[0015] References to "an embodiment" or "an embodiment" in the specification mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of this teaching. The phrase "in an embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment.

[0016] It should be understood that the steps of the methods described herein can be performed in any order and / or simultaneously, provided that the teachings remain operational. Furthermore, it should be understood that the apparatus and methods described herein can include any number or all of the described embodiments, provided that the teachings remain operational.

[0017] LiDAR systems used in autonomous vehicles must be able to operate with high precision in a variety of driving scenarios and lighting conditions. For example, they must obtain accurate ranging and image data (which can be in the form of a 3D point cloud) for a highly reflective traffic cone a few meters away and a dark, non-reflective vehicle tire 150 meters away on the road. From an optical perspective, these two scenarios present drastically different light power inputs to the sensor system.

[0018] An optical effect that can significantly impact the performance of LiDAR systems is commonly referred to as "blooming." Blossoming occurs when relatively intense light from a point or area illuminated by the LiDAR system, or even from another light source, propagates to the input of the imaging sensor at a power high enough to saturate the LiDAR detector. In real-world driving scenarios, objects with high reflectivity are frequently encountered. For example, highly reflective traffic signs can be encountered every few minutes in urban traffic. Automotive LiDAR sensors are particularly sensitive to high-intensity reflected light from such objects. In fact, these reflected signals from relatively close and / or highly reflective objects often saturate the detector, resulting in blooming.

[0019] There are two types of halos in LiDAR systems. The first type is electronic blooming, which occurs in conventional CCD sensors when strong light illuminates only a single detector pixel. When the light reaches a certain intensity, a single detector pixel will generate a charge sufficient to overcome the electrical isolation between pixels. In this case, even though no light illuminates adjacent pixels, the adjacent pixels are recorded as illuminated.

[0020] The second type of vignetting in LiDAR systems is optical vignetting, where adjacent pixels are illuminated due to the non-ideal nature of the input focusing optics. This is at least partly because the wave-like nature of light does not allow for perfect focusing. In practical systems, the focused beam will have a tail extending beyond the apparent size of the focused beam. Sensors with extremely high dynamic ranges (such as SPADs) are susceptible to optical vignetting because SPADs can detect the tail of the focused beam falling on adjacent pixels. Charge-coupled device (CCD) detectors are less likely to experience optical vignetting compared to SPADs due to their limited dynamic range.

[0021] More specifically, the LiDAR system processor generates a data point map from the LiDAR data processing system. A data point map is a dataset collected by the LiDAR system and processed by the LiDAR processor, representing a specific geographic area and terrain within the LiDAR field of view scanned by the LiDAR system. The term "halo" can refer to the situation where the point cloud determined by the LiDAR system processor represents an image with the outline of an actual highly reflective object appearing magnified or expanded in the region surrounding it. The result is a point cloud that represents an image larger than the actual size it would represent without halos.

[0022] For example, image magnification or dilation can be caused by light beams falling outside the central region of the image, such as light diffracted from components of a LiDAR system. The intensity of light falling within the central region of the image can be directed to a pixel of the pixelated detector. The intensity of light falling outside the central region can fall on one or more pixels adjacent to said pixel.

[0023] Additionally, optical halos can cause "ghost" images. Ghosting occurs when a highly reflective object enters the LiDAR system's field of view and enough light is detected and processed by the LiDAR system to form a dataset of "ghost" point clouds representing points with the same or similar shape and size but located at different positions in the point cloud (representing different physical locations). The location of the "ghost" in the point cloud typically differs for different types of LiDAR sensors. Again, these "ghosts" can be caused by light rays falling outside the central region of the image and illuminating pixels adjacent to the central pixel of the image.

[0024] In addition to halos and ghosting, current LiDAR systems, which receive large dynamic range reflected light and ambient light, generate imperfect point cloud data. One type of imperfect point cloud data is called "point cloud distortion," which leads to inaccurate ranging, causing the LiDAR system to report inaccurate information. Another type of imperfect point cloud data is called "point cloud missing data," which results in objects not being detected at certain distances. The result of generating imperfect point cloud data is that the LiDAR system reports data with "blind spots" to the driver. These "blind spots" are missing object and ranging information, which may lead to hazards to the driver and / or unnecessary actions by the vehicle, such as automatic braking and lane changes.

[0025] Figure 1A portion of pixel map 100 of a single-photon avalanche diode (SPAD) detector is shown, illustrating the cause of unwanted halos in receivers in known LiDAR systems. SPADs are commonly used in solid-state LiDAR systems. A characteristic of SPAD detectors is their ability to detect at the single-photon level, meaning that SPADs will detect light from a low-intensity tail around the focal point. This contrasts with standard CCD detector arrays, which are insensitive to extremely low light levels.

[0026] Square 102 represents pixels arranged in a two-dimensional spatial arrangement of SPAD pixels. In this illustration, a point target 104 is shown as its image centered on a single pixel, which we refer to as the primary receiver pixel. For descriptive purposes, it is assumed that the target is a single point at infinity. Receiver pixels 106 in the first row and 108 in the second adjacent row are shown in pixel diagram 100. On pixel diagram 100, the term "vagueness," as used herein, is shown as some focused light extending beyond the target pixel onto pixels such as pixels 106 in the first adjacent row and pixels 108 in the second adjacent row of pixel diagram 100.

[0027] Figure 2A This is a schematic diagram of a LiDAR receiver 200, showing the receiver optics and SPAD detector array. The receiver optics are shown as a lens 202, which is shown to collect light from two point targets 204, 206. The lens 202 then focuses the transmitted light onto a surface configured as shown in the diagram. Figure 1 The SPAD detector array 208, showing rows and columns of pixels, is positioned at a focal length from lens 202, such that point targets 204, 206 are imaged onto pixels within the SPAD detector array 208. It should be understood that this schematic diagram of the LiDAR receiver 200 does not precisely represent the ratio of distance to component size. Although not apparent in the schematic diagram of the LiDAR receiver 200, the rays from point targets 204, 206 are essentially parallel rays because they are positioned at a distance from lens 202 much greater than the focal length of the lens, approaching infinity in an ideal case. In a practical configuration, the rays from point target 204 are almost parallel, and the rays from point target 206 are almost parallel, but the rays from point targets 204 and 206 are not parallel to each other.

[0028] Figure 2BThis is a diagram of the optical beam profile 250 at the SPAD detector array. A row of SPAD pixels is shown, where each pixel 252 has a length and width of 30 micrometers. No lens can focus the beam to a perfect point of zero width. Any focal point has some width. The spot size can be minimized to some extent. However, as a practical problem, any real target being imaged will at least partially have light spillover onto adjacent SPAD pixels.

[0029] refer to Figure 2A and Figure 2B The beam profile 250 illustrates a first beam 254 corresponding to the first point target 204 and a second beam profile 256 corresponding to the second point target 206. The first and second beam profiles 254 and 256 represent a certain level of optical power extending to adjacent pixels of the SPAD detector array. The optical power experienced by the pixels in the SPAD detector array can be obtained by numerically integrating the two-dimensional power distribution falling on the desired pixels. The power distribution is... Figure 2B The outlines are schematically represented in one dimension as 254 and 256.

[0030] One object of the present invention is a method and apparatus for mitigating the effects of halos, ghosting, point cloud distortion, and missing point cloud data events in LiDAR systems. In one aspect of this teaching, these objects are achieved using apodized apertures or filters. The term "apodized," as used herein in conjunction with the term "aperture," refers to a situation where the aperture has a gradual transmission change from 100% to 0% rather than a hard "on-off" transition. The transmission change can occur radially across the aperture and has circular symmetry. Therefore, light passing through the center of the aperture experiences a different transmission or optical density than light at or near the aperture edges. The term "apodized," as used herein in conjunction with the term "filter," refers to an optical filter constructed with ports having a gradual transmission change from 100% to 0% rather than a hard "on-off" transition. In practice, apodizing the ports of a sharp aperture or filter will suppress long tails at the image plane by suppressing diffraction artifacts generated by sharp edges. Another characteristic of apodization apertures is that the intensity of light hitting the central spot in the image plane far from the aperture is suppressed. Therefore, the ratio of the intensity of light hitting the image plane far from the central spot to the intensity at the peak of the central spot is small and can be controlled according to the transmission profile of the apodization aperture. This ratio of the apodization aperture is significantly smaller compared to the same ratio of a conventional hard stop aperture. The result is an improvement in resolution beyond the diffraction limit of conventional apertures or filter ports, because the diffraction limit assumes a sharp transition, thus producing intensity ripples at the edges. Therefore, apodization filters (including apodization apertures) according to this teaching are configured to reduce or even substantially eliminate undesirable intensity variations in optical systems caused by diffraction and other optical effects, particularly those caused by hard stop apertures.

[0031] In one specific embodiment, the apodization filter of this teaching is a gradient filter designed to reduce or substantially eliminate undesirable intensity variations in an optical system caused by diffraction and other optical effects. For example, the gradient filter may be a continuously variable gradient filter, wherein the optical density varies across the substrate. Such filters are commercially available, for example, from Reynard Corporation of San Clemente, California. The gradient filter may also include a continuously variable pattern that causes variations in optical density across the filter.

[0032] In one specific embodiment, the optical density of the apodization filter increases radially from a clear or relatively clear central region. In such a filter, light will reach its peak intensity at the center and then gradually decrease in intensity towards the outer edges of the filter. In another specific embodiment, the optical density of the apodization filter is highest at the center, and the optical density increases from the edges of the filter towards the center.

[0033] In one specific embodiment, the apodization filter is designed for a combination of Gaussian beam distributions or Gaussian beam profiles. However, in general, the apodization filter according to this teaching can be configured for any type of beam profile.

[0034] Figure 3 A schematic diagram of a LiDAR receiver 300 configured according to this teaching is shown. The receiver 300 includes input optics 304 configured to receive light from targets 306, 306', 306''. Figure 3 The figure shown illustrates the light traces from three targets 306, 306', and 306'' positioned at infinity for the purposes of this discussion. Input optics 304 directs light from targets 306, 306', and 306'' to an apodization filter 302, which includes a filter. The apodization aperture 302, having the desired optical transmission profile, is positioned at the point where the light rays from targets 306, 306', and 306'' overlap. In one specific embodiment, the diameter of the apodization aperture 302 with the desired optical transmission profile at the 50% transmission position is the same as the aperture stop diameter of a known system without an apodization filter. The dimensions (including shape and width) of the transition region 308 of the apodization aperture 302 are chosen to suppress a desired amount of diffraction artifacts generated by the edges of the aperture stop. For example, in some embodiments, the width of the transition region 308 is several hundred micrometers.

[0035] The detector optics 310 are positioned behind the apodization aperture 302 in the propagation direction of the light from the three targets 306, 306', and 306''. The detector optics 310 focuses the light from the three targets 306, 306', and 306'' onto the image plane 312 of the SPAD detector array 314, which includes a two-dimensional pixel array. Specifically, the detector optics 310 receives the light, which has been modulated by the apodization aperture, at the image plane 312. The detector optics have optical power and are positioned between the apodization aperture 302 and the image plane 312 such that the modulated light can be projected onto the detector positioned at the image plane, thereby forming a speckle profile. The image plane 312 may be referred to as the detector plane. In some embodiments, the detector plane is located at the focal plane of the detector optics.

[0036] Figure 4A graph 400 shows a comparison of optical power transmission across the aperture as a function of distance for a standard aperture stop and for an apodized aperture according to this teaching. A first optical power transmission curve 402 is shown for a standard aperture with sharp edges. As expected, the optical power abruptly changes from full transmission within the aperture to essentially zero transmission outside the aperture. A second optical power transmission curve 404 is shown for an apodized aperture according to this teaching. For an apodized aperture filter function, the optical power transitions more gradually from full transmission to zero transmission across the aperture as a function of distance. In various embodiments of this teaching, the apodized aperture is configured to provide desired transmission characteristics to suppress a desired amount of diffraction artifacts generated by the edges of the aperture stop. In other words, the apodized aperture is configured to reduce the intensity of light in the region outside the central transmission region compared to a system without an apodized filter. Thus, in some embodiments, for a two-dimensional pixel array positioned on a detector plane, a desired ratio can be achieved between the intensity of the modulated light projected onto one pixel of the two-dimensional pixel array and the intensity of the modulated light projected onto adjacent pixels of the two-dimensional pixel array. For example, the intensity of modulated light projected onto adjacent pixels of a two-dimensional pixel array can be much lower than the intensity of modulated light projected onto a single pixel of the two-dimensional pixel array. For example, the single pixel can be closer to the center of the optical illumination on the detector array, and adjacent pixels can be closer to the edge of the optical illumination on the detector array.

[0037] In some embodiments, the apodization aperture is circularly symmetrical about a center point. In these embodiments, transmission as a function of distance varies radially across the aperture. In some embodiments, transmission is high at the center of the aperture and gradually decreases at the edges of the aperture, which reduces the effect of diffraction at the aperture edges.

[0038] Figure 5 A graph 500 is shown, illustrating a comparison of the relative intensities of light from a single point target, focused in a speckled profile onto a pixel of a SPAD detector array, after passing through a conventional aperture and after passing through an apodization aperture filter according to the present teachings. This speckled profile is caused after the aperture-modulated light is projected onto the detector plane by the detector optics. The relative intensity of the light is plotted as a function of distance from the center pixel of the array. In this particular SPAD detector array, the length and width of the pixel are approximately 30 micrometers. The intensities shown from 15 micrometers to 45 micrometers represent the light intensity in adjacent pixels.

[0039] Figure 502 illustrates the relative intensity of light from a single point target, focused and concentrated on the central pixel of the SPAD detector array after passing through a conventional aperture approaching a step function transmission profile. More specifically, Figure 502 shows that light transmission peaks at the center of the single pixel and then rapidly decreases as a function of distance from the edge of that single pixel at 15 micrometers. The relative intensity at the edge of the single pixel decreases by 10 at that edge. -4 Range. Graph 502 also shows a magnified view of the relative intensity of light from a single point target, focused and concentrated on a single pixel, ranging from 15 micrometers to 45 micrometers (which illuminates adjacent pixels). The relative intensity of light illuminating adjacent pixels exceeds 3*10 near the edge of that single pixel. -5 When the relative intensity of light is integrated over the area of ​​adjacent pixels, the power of the apodization aperture is reduced by ~5dB compared to the regular aperture, while the integrated power on the main pixel remains unchanged.

[0040] Figure 504 illustrates the relative intensity of light from a single point target, focused and concentrated on a pixel of the SPAD detector array after passing through an apodization aperture filter according to this teaching. Figure 504 (similar to Figure 502) shows that light transmission peaks at the center of a single pixel and then rapidly decreases as a function of distance from the edge of that single pixel at 15 micrometers. The relative intensity at the edge of the single pixel also decreases to around 10 micrometers. -4 The range is similar to a graph with a conventional aperture. Graph 504 also shows a magnified view of the relative intensity of light from a single point target, focused and concentrated on that single pixel, ranging from 15 micrometers to 45 micrometers (which illuminates adjacent pixels). The magnified view of graph 504 shows the relative intensity of light illuminating adjacent pixels near the edge of that single pixel at a high 10. -6 The range is significantly improved compared to the conventional aperture, which is close to the step function, as shown in Figure 502. Figures 502 and 504 clearly show that the apodization filter significantly reduces the intensity of light entering adjacent pixels of the SPAD detector array due to diffraction.

[0041] Therefore, a key feature of this teaching is to reduce vignetting and ghosting effects in LiDAR detectors by reducing the intensity of light falling on pixels outside the central region or spot in the image plane associated with a real object image. An apodization aperture is selected to provide desired modulation of the overlapping light rays originating from the object and projected by optics at the LiDAR input. This desired modulation results in lower intensity in regions away from the central focal region of the detector optics that project the modulated overlapping light rays onto the detector plane. A two-dimensional pixelated detector array is positioned in the image plane that receives the projected modulated overlapping light rays. In some embodiments, the desired modulation provided by the apodization aperture is selected such that a desired ratio is achieved between the intensity of the projected modulated light rays falling on one pixel of the two-dimensional pixel array and the intensity of the projected modulated light rays falling on adjacent pixels of the two-dimensional pixel array. This desired ratio can be a ratio that reduces vignetting effects in the two-dimensional pixelated detector array. In other words, the LiDAR system of this teaching utilizes an apodization aperture that modulates the intensity across a focused set of overlapping rays reflected from an object, such that when this focused set of modulated overlapping rays is projected onto a pixelated detector array by detector optics, the ratio of the intensity in the tail of the transmitted beam extending beyond the apparent size of the focused light to the intensity within the apparent size of the focused light is set to a desired ratio. This desired ratio can be low enough to reduce vignetting in the pixelated detector array.

[0042] Equivalent

[0043] Although the applicant's teachings have been described in conjunction with various embodiments, it is not intended to limit the applicant's teachings to such embodiments. Rather, the applicant's teachings encompass various alternatives, modifications, and equivalents, which, as will be apparent to those skilled in the art, can be modified without departing from the spirit and scope of these teachings.

Claims

1. A LiDAR receiver, comprising: a) An input terminal configured to receive light from an object illuminated by a LiDAR transmitter; b) An optical element positioned adjacent to the input end and configured with an optical power that projects light received at the input end onto an area where the received light overlaps. c) Apodization aperture, which is positioned in the region where the received light overlaps, the apodization aperture being configured with an optical transmission profile that varies radially over the apodization aperture in order to provide desired modulation of the overlapping received light. d) A detector optics element positioned adjacent to the apodization aperture to receive modulated light, the detector optics element having an optical power and position for projecting the modulated light onto a detector plane; and e) A two-dimensional pixelated detector array, the two-dimensional pixelated detector array being positioned at the detector plane, wherein the desired modulation is selected such that the intensity of the modulated light projected onto one pixel of the two-dimensional pixel array is a desired ratio to the intensity of the modulated light projected onto the adjacent pixels of the two-dimensional pixel array.

2. The LiDAR receiver according to claim 1, wherein, The apodization aperture includes a gradient filter.

3. The LiDAR receiver according to claim 1, wherein, The apodization aperture is configured to increase optical transmission in the radial direction toward the center of the apodization aperture.

4. The LiDAR receiver according to claim 3, wherein, The optical transmission increases linearly in the radial direction toward the center.

5. The LiDAR receiver according to claim 3, wherein, The optical transmission increases in the radial direction toward the center according to a mathematical function related to the beam profile.

6. The LiDAR receiver according to claim 1, wherein, The apodization aperture is configured to provide the desired optical transmission gradient for a Gaussian beam.

7. The LiDAR receiver according to claim 1, wherein, The apodization aperture is configured to reduce diffraction effects occurring at the edges of the apodization aperture.

8. The LiDAR receiver according to claim 1, wherein, The apodization aperture reduces the intensity of higher-order optical modes at the edges of the apodization aperture.

9. The LiDAR receiver according to claim 1, wherein, The pixelated detector array includes a single-photon avalanche diode array.

10. The LiDAR receiver according to claim 1, wherein, The detector plane is positioned at the focal point of the detector optics.

11. A method for LiDAR detection, the method comprising: a) Receive light from objects illuminated by a LiDAR transmitter; b) Project the light received from the object illuminated by the LiDAR transmitter onto the area where the received light overlaps; c) Transmit overlapping light rays through an apodization aperture having an optical transmission profile across the apodization aperture, the optical transmission profile modulating the overlapping light rays to provide the desired modulation of the overlapping received light rays. as well as d) Project the overlapping modulated light onto a two-dimensional pixelated detector array positioned on the detector plane such that the intensity of the modulated light projected onto one pixel of the two-dimensional pixelated detector array is achieved at a desired ratio to the intensity of the modulated light projected onto the adjacent pixels of the two-dimensional pixelated detector array.

12. The method of claim 11, further comprising selecting an optical transmission profile across the apodization aperture such that it gradually decreases optical transmission radially from the center of the aperture toward the edge.

13. The method according to claim 12, wherein, The transmission profile decreases linearly from the center of the apodization aperture towards the edge.

14. The method according to claim 12, wherein, The transmission profile decreases radially from the center of the apodization aperture toward the edge according to a predetermined mathematical function.

15. The method according to claim 14, wherein, The mathematical function is related to the beam profile of light received from an object illuminated by a LiDAR transmitter.

16. The method according to claim 15, wherein, The beam profile is a Gaussian beam profile.

17. The method of claim 11, further comprising selecting an optical transmission profile across the apodization aperture such that the intensity of higher-order optical modes is reduced at the edges of the aperture.

18. The method of claim 11, further comprising positioning the detector plane at the focal point of the detector optics.

19. The method of claim 11, wherein the expected ratio is greater than 10. 3 .

20. The method of claim 11, wherein the expected ratio is greater than 10. 4 .

21. The method of claim 11, wherein the expected ratio is greater than 10. 5 .

22. The method of claim 11, wherein the size of at least one pixel of the two-dimensional pixelated detector array is in the range of 15 micrometers to 45 micrometers.

Images