Holographic light detection and ranging

Abstract

A light detection and ranging "LIDAR" system arranged to survey a scene. The LIDAR system includes a spatial light modulator arranged to display a diffraction pattern of a hologram including a light footprint. The LIDAR system also includes a light source arranged to illuminate the diffraction pattern to form a holographic reconstruction of the light footprint on a holographic replay plane in the scene. In accordance with the present disclosure, the light footprint includes an array of light features, e.g., an array of light points. The LIDAR system also includes a display driver arranged to control the spatial light modulator and to change the diffraction pattern over time. The diffraction pattern varies over time such that each light feature in the array of light features scans a respective sub-region of the scene. The LIDAR system includes a detection system having a plurality of light detection elements. The detection system is configured such that each light detection element detects light from a respective individual field of view within the scene. Each sub-region of the scene contains a plurality of individual fields of view.

Description

Technical Field

[0001] This disclosure relates to light projectors. More specifically, this disclosure relates to holographic projectors, holographic projection systems, holographic projection methods, and holographic projection systems. Embodiments relate to light detection and ranging systems. Some embodiments relate to light detection and ranging methods. Some embodiments relate to methods for surveying scenes using computer-controlled light footprints. Background Technology

[0002] Light scattered from an object contains amplitude and phase information. This amplitude and phase information can be captured on, for example, a photosensitive plate using well-known interferometry techniques to form a holographic record, or "hologram," including interference fringes. The hologram can be reconstructed by illuminating it with appropriate light to form a two-dimensional or three-dimensional holographic reconstruction or replay image representing the original object.

[0003] Computer-generated holography can numerically simulate interference processes. Computer-generated holograms can be calculated using techniques based on mathematical transformations such as Fresnel or Fourier transforms. These types of holograms are called Fresnel / Fourier transform holograms or simply Fresnel / Fourier holograms. A Fourier hologram can be considered a Fourier domain / plane representation of an object, or a frequency domain / plane representation of an object. For example, coherent ray tracing or point cloud techniques can also be used to calculate CGH.

[0004] CGH can be encoded on a spatial light modulator (SLM) arranged to modulate the amplitude and / or phase of the incident light. For example, optical modulation can be achieved using electrically addressable liquid crystals, optically addressable liquid crystals, or micromirrors.

[0005] An SLM can comprise multiple individual addressable pixels, which may also be referred to as cells or elements. The optical modulation scheme can be binary, multilevel, or sequential. Alternatively, the device can be sequential (i.e., excluding pixels), so the optical modulation can be sequential on the device. An SLM can be reflective, meaning that modulated light is reflected from the SLM output. An SLM can also be transmissive, meaning that modulated light is transmitted from the SLM output.

[0006] The system described herein can be used to provide a holographic projector for imaging. Such projectors are already used in head-up displays (HUDs) and head-mounted displays (HMDs), including near-eye devices. The holographic projector can also be used for light detection and ranging. Light detection and ranging systems can be used in a variety of applications, including portable devices and vehicles.

[0007] This disclosure relates to improvements in optical detection and ranging systems. In particular, such improvements may include faster, more reliable, and / or more accurate techniques for surveying areas of a scene using optical detection and ranging in order to detect features of interest. Summary of the Invention

[0008] Various aspects of this disclosure are defined in the appended independent claims.

[0009] A light detection and ranging "LIDAR" system is provided, arranged to survey a scene. The LIDAR system includes a spatial light modulator arranged to display a diffraction pattern comprising a hologram of a light footprint. The LIDAR system also includes a light source arranged to illuminate the diffraction pattern to form a holographic reconstruction of the light footprint. The holographic reconstruction of the light footprint is projected onto the scene. In some embodiments, an intermediate holographic reconstruction is formed between the spatial light modulator and a projection lens arranged to project the light footprint onto the scene by imaging the intermediate holographic reconstruction. In other embodiments, the holographic reconstruction is projected directly onto the scene (i.e., no intermediate holographic reconstruction is formed and no projection lens is required). According to this disclosure, the light footprint includes an array of light features. For example, each light feature in the light feature array may be at least one of a light spot, a light spot pattern, a scan line, and a plurality of parallel scan lines. Each light feature (e.g., each light spot) may include a plurality of image pixels. The LIDAR system also includes a display driver arranged to control the spatial light modulator and change the diffraction pattern over time. The diffraction pattern changes over time such that each light feature in the light feature array scans a corresponding sub-region of the scene. In essence, the diffraction pattern is altered to move each optical feature to a different location along a scan path within its corresponding sub-region. The scan path can be continuous / uninterrupted or discontinuous, including discrete steps / jumps within the sub-region. Therefore, each optical feature in the array is uniquely associated with a given sub-region. The LIDAR system includes a detection system with multiple optical detection elements. The detection system is configured such that each optical detection element detects light from a corresponding individual field of view within the scene. Each sub-region of the scene contains multiple individual fields of view.

[0010] Therefore, the scene surveyed by the LIDAR system (also referred to herein as the "field of view" of the LIDAR system) is divided into multiple sub-regions. These multiple sub-regions can form a substantially continuous area of ​​the scene. The sub-regions can be substantially non-overlapping. Each sub-region can be a quadrilateral. According to this disclosure, each of the multiple sub-regions is simultaneously illuminated by an array of light features forming a light footprint. Specifically, each light feature (formed within the initial or zero diffraction order) is uniquely associated with a sub-region of the LIDAR system's field of view. Therefore, each light feature moves along a scan path within its corresponding sub-region to multiple scan positions to scan that sub-region individually. Thus, the light power in each sub-region can be controlled. As discussed herein, it is desirable to control the light power of the projected laser for eye safety reasons. In the embodiments, the light power of the light footprint in each sub-region is "eye-safe." As discussed further below, although standards vary in different jurisdictions and depend on the wavelength of light, light power distributed within a 7 mm circular aperture diameter below a threshold of approximately 1.0 mW (at 905 nm) is generally considered "eye-safe."

[0011] By using a LiDAR illumination pattern (light footprint) comprising an array of light features, where each light feature corresponds to a specific sub-region of the scene, and each sub-region receives light simultaneously, and by altering the diffraction pattern so that each light feature of the illumination pattern scans its corresponding sub-region, all sub-regions of the scene can be scanned simultaneously. As described herein, this method allows for faster and more accurate scanning of the entire scene.

[0012] An array of light features may include a regular or ordered array of light features that form an illumination pattern (light footprint). For example, in some examples, the array includes an ordered array of rows and columns of discrete light points, where each light point is formed at the same relative position within its respective sub-region. Thus, in this example, the light feature is a single light point formed to scan each sub-region. In other examples, the array may include more than one light point arranged in a pattern formed in the respective sub-region, such that the array includes an ordered arrangement of rows and columns of light point patterns. Thus, in this example, the light feature is a light point pattern, where the light points may be adjacent to each other (continuous) or spatially separated from each other (discrete). In particular, each light feature including a light point pattern may form multiple scan points (discrete light points), or one or more scan lines, etc., for individually scanning a respective sub-region of the scene. In either case, each light point may include multiple consecutive image pixels arranged in quadrilaterals, circles / ellipses, etc. In some embodiments, each light feature includes at least one ray, such as a straight ray, referred to herein as a "scan line". Scan lines can be formed by multiple consecutive light spots forming a straight line, or by consecutive discrete light features of a straight line shape. Each scan line may include multiple consecutive image pixels arranged in a straight line over the thickness of at least one image pixel.

[0013] In some embodiments, the display driver is arranged to change the diffraction pattern over time by altering the hologram of the diffraction pattern in order to scan sub-regions of the scene. Specifically, the hologram can be modified to reposition the array of light features such that each light feature is repositioned within its corresponding sub-region. By repositioning the light features within their respective sub-regions, the entire sub-region can be scanned. The array of light features scans each sub-region of the scene simultaneously. Therefore, by changing the diffraction pattern according to different hologram sequences, the entire area of ​​the scene can be scanned using an array of light features located at multiple different positions.

[0014] In other embodiments, the display driver is arranged to move the projected light footprint across the scene. Specifically, the array of light features of the projected light footprint is sequentially moved to multiple different locations on the scene. Specifically, the multiple different locations of the light footprint can be selected to form each light feature in the light feature array at multiple different locations within a corresponding sub-region of the scene. By forming light features at multiple different locations within a corresponding sub-region, the entire sub-region can be scanned. The light feature array scans each sub-region of the scene simultaneously. Therefore, the entire area of ​​the scene is scanned by sequentially moving the light footprint to multiple different locations. In this case, the hologram may remain unchanged.

[0015] In some embodiments, the diffraction pattern includes a grating function (also referred to as a "software grating") that determines the position of the light footprint on the holographic reproduction plane. Specifically, the grating function controls the position of the holographic reproduction field on the reproduction plane. The holographic reconstruction of the light footprint is formed on the holographic reproduction plane, which may be an intermediate plane or within a scene. Therefore, the grating function of the diffraction pattern controls the spatial position of the projected light footprint within the scene. In some embodiments, the display driver is arranged to change the grating function of the diffraction pattern to scan sub-regions of the scene. It can be said that by changing the grating function, the holographic reconstruction is spatially shifted or translated on the reproduction plane. In some embodiments, the grating function is a phase ramp function, such as a wound or repeated phase ramp function or a modulo 2p phase ramp function, for example, having a sawtooth shape. Those skilled in the art will understand how to change the grating spacing (or the gradient of the phase ramp) to change the translation of the zero-order reproduction field on the reproduction plane used for scanning.

[0016] The "propagation axis" can be defined as the axis of the modulated light that propagates to the reproduction plane that forms the zero-order (hereinafter referred to as "zero-order") reproduction field. In other words, the propagation axis is the line connecting the center of the pixel array of the spatial light modulator to the center of the zero-order reproduction field. As further described below, a grating function has the effect of changing the direction of the propagation axis. For perpendicular incidence of light on the spatial light modulator, and without a grating function, the propagation axis extends substantially perpendicularly through the center of the zero-order reproduction field. Therefore, by simply changing the grating function of the diffraction pattern, for example without changing the hologram of the light footprint, the array of light features (e.g., light spots) of the light footprint can be moved to multiple different locations to scan all sub-regions of the scene simultaneously.

[0017] Due to the diffractive nature of the process, the holographic reconstruction formed by the hologram includes a zero-order reproducing field at the center and multiple higher-order reproducing fields extending along the + / -x and + / -y directions (on the xy holographic reproducing plane). In a manner familiar to those skilled in optics and diffraction, the higher-order reproducing fields are lower-brightness repetitions of the zero-order reproducing field. Due to the diffractive nature of the holographic process, the maximum intensity on the holographic reproducing plane is non-uniform. That is, diffraction has the effect of modulating the intensity of the holographic reconstruction according to a non-uniform intensity envelope (or cover). In some embodiments, the envelope is a sinc function, or more specifically, sinc 2 The term "intensity non-uniformity" in the reproduction plane, as mentioned here, is a shorthand for the envelope of the intensity of the modulated holographic reconstruction. For example, if the hologram is uniformly bright across the entire reproduction field, the holographic reconstruction formed using a pixelated spatial light modulator at the actual reproduction field will be non-uniform in brightness due to the intensity envelope. Therefore, in short, the intensity distribution on the reproduction plane can be described as a sinc function extending in both the x and y directions (specifically, sinc squared - sinc). 2 The function, because it has no negative intensity value. According to sinc 2 The intensity envelope of the function arises with the spatial light modulator including quadrilateral light modulation pixels. More precisely, the intensity envelope can be said to include the first sinc in the x-direction. 2 The function and the second sinc in the y direction 2 Function. If the pixels of the spatial light modulator are not square, then the first sinc 2 The function is different from the second sinc 2 Function. When the grating function is applied, the center of the zeroth-order reproduced field is relative to sinc. 2 Intensity envelope translation.

[0018] In some embodiments, the optical footprint comprises an array of light spots—that is, each optical feature is a discrete light spot. Typically, each discrete light spot is formed by multiple consecutive image pixels in a desired light spot shape (e.g., quadrilateral or circular). It may be desirable to form light spots of uniform brightness. A hologram of a uniform brightness light spot array can be computed. However, due to the intensity envelope, the brightness of the reconstructed light spots may vary across the playback plane. For example, a light spot formed at the center of the optical footprint reconstructed in a zero-order hologram may have a higher intensity than light spots formed around its periphery. When computed, sinc can be applied. 2 Compensation schemes. For example, sinc can be applied. 2 A compensation function is used to change the intensity value of the light spot to compensate for changes in intensity with spatial location. Therefore, sinc 2 Compensation minimizes the intensity variations of image points in (zero-order) holographic reconstruction, thus providing a more uniform intensity distribution in the playback field. However, when forming a hologram that includes an optical footprint comprising an array of light spots (or other optical features such as scan lines), and moving the array of light spots without altering the hologram (e.g., through a grating function as described above), intensity variations between light spots can still occur, as described herein.

[0019] Specifically, in some embodiments, the hologram is configured (e.g., calculated) based on sinc applied to the light footprint. 2 A compensation function is used to compensate for intensity non-uniformity on the holographic reproduction plane. During scanning, sinc... 2 After compensation is applied to the target image (i.e., the desired light footprint), the hologram is computed. However, according to this disclosure, the light pattern is continuously moved / translated to simultaneously scan multiple sub-regions using corresponding light features. This scanning perturbs the sinc 2 Compensation, because the translation of the replay field relative to sinc 2 Intensity envelope occurs. sinc 2 This deterioration of the compensation scheme means that the compensation is suboptimal—the inhomogeneity or variation of light feature intensity used to detect different parts of the scene may increase (without minimization).

[0020] The holographic replay plane can be defined as an xy plane that receives spatially modulated light from a spatial light modulator, which propagates around a propagation axis extending in the z-direction. Therefore, the replay plane is spatially separated from the spatial light modulator by a propagation distance in the z-direction. The grating function may include an x-direction grating and a y-direction grating, which control the orientation of the propagation axis, thereby controlling the position of the light footprints along the x and y directions on the replay plane, respectively. In some embodiments, the target image used to compute the hologram is modified prior to the hologram computation to compensate for the intensity envelope and the use of varying grating functions. The hologram can be arranged to modulate the intensity of each light feature in the target image based on its distance from the center of the intensity envelope. In an improved embodiment, the hologram can be arranged to modulate the intensity of each light feature during scanning based on the average distance of each light feature in the x-direction and the average distance in the y-direction. More specifically, the brightness of each light feature in the array of light features of the target image (used to compute the hologram) is modulated based on the distance of the corresponding sub-region from the center of the intensity envelope. This provides more optimized compensation by minimizing the intensity variations of light features formed in different sub-regions on the scene during scanning.

[0021] In some embodiments, sinc is used for each optical feature (e.g., a light spot). 2 The compensation value (in terms of location) corresponds to the middle of its sub-region (e.g., the middle x-position and the middle y-position). It can be said that, via sinc... 2 The target image used for compensation and computation of the hologram is such that each optical feature in the optical feature array is positioned at the midpoint of its respective sub-region (in the x and / or y directions). Furthermore, positive and negative x and y raster functions are used to provide the desired translation of each optical feature from the midpoint of its sub-region, in order to provide multi-region scanning according to this disclosure.

[0022] In a surprising further improvement, in sinc 2 Before compensation (and therefore before hologram calculation), the intensity of light features in the central region of the light footprint (target image) is enhanced (i.e., increased, for example, by a factor / number greater than 1) to improve the overall uniformity of scene illumination during the scanning of sub-regions. This is counterintuitive, as the center of the replay field is typically the brightest part of the holographic replay field. According to this further improvement, the target intensity of light features in the target image is not equal (in sinc...). 2 (Before compensation). In some embodiments, selection is made for sinc 2 The compensation factor modifies (e.g., magnifies or enhances) the target image in the central region to increase or even maximize the uniformity of the optical feature array—or at least most of the optical feature array—across all scan locations. This factor can be a constant or a function of location on the playback field, such as distance from the center of the playback field or distance from sinc.2 The distance to the maximum value of the intensity envelope. In a variation, at sinc 2 After compensation, the intensity of each optical feature is modified, either additionally or alternatively, to improve illumination uniformity across all scanning positions. In another variation, further processing, such as sinc... 2 Before compensation and hologram calculation, the intensity of peripheral light features (i.e., light features near the edges or outside the center region) of the target image is reduced.

[0023] However, in some embodiments, the hologram is configured (e.g., calculated) such that the optical power of the optical features in the optical feature array of the light footprint varies with distance from the propagation axis. In particular, in some LiDAR applications, regions scanned at the periphery of the scene require different scan power than regions scanned at the center of the beam. For example, the center of the scene may require a longer scan with higher power, while the periphery may require a shorter scan with lower power. Therefore, the power of the optical features formed in the corresponding sub-regions can be adjusted according to the properties / requirements of the respective scene.

[0024] Each of the multiple sub-regions of a scene can be scanned individually and simultaneously in the same manner. In some embodiments, individual scans of sub-regions can be formed in a systematic sequence of successive scan positions along a scan path. The scan positions of the scan pattern or path are selected so that the entire sub-region is scanned. For example, the diffraction pattern can be changed such that each optical feature performs a raster scan (in the case of a spot) or a horizontal / vertical line scan (in the case of a scan line) of its corresponding sub-region. Thus, each sub-region is scanned by a spot of light moving in a raster scan order or sequence (from left to right and top to bottom of its corresponding sub-region) or by a scan line moving in a vertical or horizontal direction. In other embodiments, the diffraction pattern is changed such that each optical feature performs a “patterned scan” of its sub-region. In particular, a patterned scan can be defined as a sequence of different positions of optical features within their corresponding sub-regions, in a random or semi-random order or in a defined (time) pattern or order, such that a region of sub-region with a defined / minimum size is not exposed for more than a threshold time period.

[0025] Each light detection element (also referred to herein as a light detection element) of the detection system is capable of receiving light from a defined area of ​​the scene corresponding to its field of view. Each individual light detection element can be said to have a corresponding individual field of view (referred to herein as an "IFOV"). According to this disclosure, each sub-region of the scene contains multiple IFOVs. Therefore, each IFOV of the light detection element corresponds to only a portion of the sub-region of the scene. Thus, light reflected from the scene illuminated by the array of light features of the light footprint will be detected at any given time only by a subset of the multiple light detection elements (e.g., for each display event / corresponding exposure time). For example, each IFOV of the light detection element may receive light from an area illuminated by a single light feature of the array (i.e., an area with the same size (e.g., the same solid angle) as a portion of a sub-region illuminated by a single light feature at a specific scan position). In such an example, it can be said that there is a one-to-one association between a single light feature of the light footprint and a single light detection element. In other examples, each IFOV of the light detection element may receive light from an area illuminated by one or more light features formed at one or more scan positions, or a portion of a light feature.

[0026] In one embodiment, the detection system may include an array detector comprising multiple photodetector elements. The array detector / multiple photodetector elements may include a charge-coupled device (CCD) camera, wherein each photodetector element is a separate CCD of the CCD element array. In other embodiments, the array detector / multiple photodetector elements may include a single-photon avalanche diode (SPAD) array, which comprises an array of SPAD elements. Any other suitable form of photodetector including multiple photodetector elements is possible and contemplated. In this embodiment, the array detector / multiple photodetector elements are static. Therefore, it can be said that the IFOV of each photodetector element uniquely corresponds to a sub-region of the total field of view of the LIDAR system.

[0027] For optimal operation of the photodetector array, it is important that the light detected by the photodetector elements is not within a wide dynamic range, which could cause some photodetector elements to saturate within a fixed exposure time. Specifically, a constant exposure time is typically chosen for all photodetector elements. This balances the need for sufficient detection of low-intensity reflections by some photodetector elements while preventing other photodetector elements from saturating (overexposing) due to high-intensity reflections. Saturation of a photodetector element renders it inoperable for a period of time. Therefore, in some embodiments, the exposure time associated with each photodetector element is constant, and the display driver is also configured to change the hologram so that if a detection signal from a particular sub-region indicates that the corresponding detector element is saturated, the optical power of the light in that sub-region is reduced, and optionally, the optical power of the light in other sub-regions is increased simultaneously.

[0028] In some embodiments, the LIDAR system also includes an optical system arranged to amplify sub-regions. For example, optical elements can be provided that extend the reproduction field to a large emission aperture. This has the effect of propagating light over a large area of ​​the aperture / window in a short optical path, which advantageously increases the total amount of light that can be emitted from the aperture / window without exceeding the laser safety power limit within, for example, a 7 mm diameter aperture.

[0029] In some embodiments, the periodic continuation or extension of the array of light features forming the illumination pattern (light footprint) into at least one first-order holographic reconstructed field. In such embodiments, the detection system is arranged to detect light from a scene region illuminated by light features of the zero-order holographic reconstructed field as well as light from a scene region illuminated by light features of at least one higher-order holographic reconstructed field, in order to extend the field of view of the LIDAR system. In particular, the field of view of the projected illumination pattern (where the light footprint / holographic reconstruction is formed in the scene) includes some spatially modulated light from both higher-order and primary (zero-order) light. It can be said that the LIDAR system uses a “hybrid reconstructed field” that includes a portion of the zero-order and one or more first-order light. Using some first-order light is particularly suitable for covering illumination patterns with periodically varying reconstructed fields, as the combination of first-order and zero-order light provides a seamless pattern. Because optical power is inherently used to form the first order, using a portion of the light from one or more first orders (which would otherwise be lost) for LIDAR illumination improves the optical efficiency of delivering illumination into the scene. Furthermore, it increases the field of view of the LIDAR system. In particular, for applications requiring sensing in a field of view with a high aspect ratio (e.g., a field of view in the horizontal direction that is much larger than the field of view in the vertical direction), such as automotive applications, one or two first-order replay fields in the + / -x directions can be used to provide a corresponding illumination pattern with a high aspect ratio.

[0030] In some embodiments, optical features of at least one higher-order holographic reproducing field, detectable by a detection system, are formed in a region of the at least one higher-order holographic reproducing field that is immediately adjacent to (or adjacent to) the zero-order holographic reproducing field. In some examples, the region of the at least one higher-order holographic reproducing field constitutes less than 50%, for example, less than 30%, of the total area of ​​the at least one higher-order holographic reproducing field. A combined approach of illuminating sub-regions of a scene with an optical footprint / illumination pattern comprising an array of optical features is well-suited for use in conjunction with illuminating the scene using some first-order holographic reconstruction, since the repeating pattern of the optical feature array is reproduced at the zero-order edges (in the x and y directions).

[0031] The term "hologram" is used to refer to a record containing amplitude or phase information about an object, or some combination thereof. The term "holographic reconstruction" is used to refer to the optical reconstruction of an object formed by projecting a hologram. The system disclosed herein is described as a "holographic projector" because the holographic reconstruction is a real image and is spatially separated from the hologram.

[0032] The term "reproduced field" is used to refer to a 2D region within which a holographic reconstruction can be formed and fully focused. If a hologram is displayed on a spatial light modulator including pixels, the reproduced field will be repeated in the form of multiple diffraction orders, where each diffraction order is a copy of the zero-order reproduced field. The zero-order reproduced field typically corresponds to the preferred or master reproduced field because it is the brightest reproduced field. Unless otherwise explicitly stated, the term "reproduced field" should be considered to refer to the zero-order reproduced field. The term "reproduced plane" is used to refer to a plane in the space containing all reproduced fields. The terms "image," "reproduced image," and "image region" refer to the region of the reproduced field illuminated by the light reconstructed by the holography. In some embodiments, "image" may include discrete "image pixels."

[0033] In this disclosure, the term "light footprint" is generally used to refer to an illumination pattern formed in a scene by the reconstruction of a hologram. Each light footprint corresponds to the formation of a holographic reconstruction in the scene. Thus, a light footprint is a region of light within a scene (more specifically, within a playback field). A light pattern may include multiple discrete regions of light (e.g., "light spots" or "scan lines") separated by dark regions. The light detection and ranging system disclosed herein can be used to form a time series of light footprints within a scene.

[0034] The terms “scanning” and “surveying” are used here as synonyms, referring to the process of probing an area of ​​a scene by illuminating it with one or more light footprints. Similarly, “scanning” or “surveying” generally includes a time series of light footprints used in the process of probing an area of ​​a scene. The term “light” is used here in its broadest sense. The embodiments also apply to visible light, infrared light, and ultraviolet light, and any combination thereof.

[0035] Advantageously, the dynamically reconfigurable holographic technique disclosed herein can be used to control the parameters of the light footprint in real time in order to form a time series of the light footprint for scanning a scene, as described herein.

[0036] The terms “encoding,” “writing,” and “addressing” are used to describe the process of providing multiple corresponding control values ​​to multiple pixels of an SLM, each determining the modulation level of the pixel. In other words, the pixels of an SLM are configured to “display” an optical modulation distribution in response to receiving multiple control values. Therefore, it can be said that an SLM “displays” a hologram, and a hologram can be considered an array of optical modulation values ​​or levels.

[0037] The embodiments describe monochromatic light footprints by way of example only. In these embodiments, the light footprint is a multicolor light footprint. In these embodiments, a composite color light footprint is provided by combining multiple monochromatic light footprints. In these embodiments, multiple monochromatic computer-generated holograms can be used to form each composite color light footprint. This wavelength diversity can increase throughput.

[0038] The embodiments described are by way of example only, illustrating 1D and 2D optical footprints. In other embodiments, the optical footprint is a 3D optical footprint. That is, in the embodiments, each computer-generated hologram forms a 3D holographic reconstruction.

[0039] It has been found that acceptable-quality holographic reconstructions can be formed from "holograms" containing only phase information related to the Fourier transform of the original object. Such holographic records can be referred to as phase-only holograms. While the embodiments relate to phase-only holograms, this disclosure is equally applicable to amplitude-only holography.

[0040] This disclosure is equally applicable to forming holographic reconstructions using amplitude and phase information associated with the Fourier transform of the original object. In some embodiments, this is achieved by using complex modulation of a so-called fully complex hologram containing amplitude and phase information associated with the original object. Because the value (gray level) assigned to each pixel of the hologram has amplitude and phase components, such a hologram may be called a fully complex hologram. The value (gray level) assigned to each pixel can be represented as a complex number with amplitude and phase components. In some embodiments, a fully complex computer-generated hologram is computed.

[0041] The term "phase delay" can be a shorthand for the phase value, phase component, phase information, or simply phase of a pixel in a computer-generated hologram or spatial light modulator. That is, any phase value described is actually a number representing the amount of phase delay provided by that pixel (e.g., in the range of 0 to 2π). For example, a spatial light modulator is described as a pixel having a π / 2 phase value causing a π / 2 radian phase delay in the received light. In some embodiments, each pixel of a spatial light modulator can operate on one of a plurality of possible modulation values ​​(e.g., phase delay values). The term "gray level" can be used to refer to a plurality of available modulation levels. For example, the term "gray level" can be used for convenience to refer to a plurality of available phase levels in a phase modulator, even if different phase levels do not provide different shades of gray. For convenience, the term "gray level" can also be used to refer to a plurality of available complex modulation levels in a complex modulator.

[0042] A hologram is a diffraction pattern because it is a pattern that causes diffraction when displayed on a spatial light modulator and illuminated with light whose wavelength is relative to (typically less than) the pixel spacing of the spatial light modulator. Reference is made herein to combining holograms with other diffraction patterns, such as those used as lenses or gratings. For example, a diffraction pattern used as a grating can be combined with a hologram to translate the playback field on the playback plane, or a diffraction pattern used as a lens can be combined with a hologram to focus the holographic reconstruction onto the playback plane in the near field. Therefore, the term "diffraction pattern" can refer to a diffraction pattern consisting solely of a hologram or in combination with one or more other diffraction patterns. Attached Figure Description

[0043] Specific embodiments are described by way of example only with reference to the following figures:

[0044] Figure 1 This is a schematic diagram illustrating a reflective SLM that generates holographic reconstruction on a screen;

[0045] Figure 2A The first iteration of the example Gerchberg-Saxton type algorithm is shown;

[0046] Figure 2B The second and subsequent iterations of the example Gerchberg-Saxton type algorithm are shown;

[0047] Figure 2C Alternative second and subsequent iterations of the example Gerchberg-Saxton type algorithm are shown;

[0048] Figure 3 This is a schematic diagram of a reflective LCOS SLM;

[0049] Figure 4 A time series of example light footprints for surveying a scene, according to an embodiment, is shown;

[0050] Figure 5 Another example optical footprint according to a further embodiment is shown;

[0051] Figure 6A -D illustrates a time series of an example light footprint for a surveying scene according to yet another embodiment, similar to... Figure 4 Example light footprint;

[0052] Figure 7 The comparison example is shown. Figure 4 The intensity distribution map of the first row of light spots formed by the light footprint sequence;

[0053] Figure 8 The following is illustrated according to an embodiment: Figure 4 The intensity distribution map of the first row of light spots formed by the time series of the light footprint;

[0054] Figure 9 This illustrates the formation of zero-order and first-order holographic reproduction fields on the reproduction plane. Figure 4 Holographic reconstruction of the first optical footprint of the optical footprint sequence;

[0055] Figure 10A-10D A further embodiment is shown. Figure 9 The holographic reconstruction is performed in which the position of the first-order replay field in the y-direction on the replay plane is occluded in the time sequence, and the widened field of view forms a mixed replay field.

[0056] Figure 11 Another example of a widened field of view for a hybrid replay field forming a scene in a car example, according to an embodiment, is shown;

[0057] Figure 12 An example is shown. Figure 10A A mixed playback field with a noise dump formed at the top and bottom edges;

[0058] Figure 13 A LIDAR system according to an embodiment is shown;

[0059] Figure 14A This shows the target and corresponding sinc of four light spots in a row of light spots formed in the light footprint. 2 The compensated intensity distribution profile, the optical footprint including the midpoint formed in the x-direction within its respective sub-region. Figure 4 An array of light spots, where the target intensities are all the same;

[0060] Figure 14B The following is illustrated according to an embodiment: Figure 4 The time series of the first to fourth light footprints were formed Figure 14A The intensity distribution profile of the four light spots in a row;

[0061] Figure 14C It shows the equivalent of Figure 14A The target of a line of four light spots in the light footprint and the corresponding optimized sinc 2 Compensation intensity distribution profile, wherein the target intensity varies according to the embodiment, and

[0062] Figure 14D The following is illustrated according to an embodiment: Figure 4 The time series of the first to fourth light footprints were formed Figure 14C The intensity distribution profile of the four light spots in this row.

[0063] In all the accompanying drawings, the same reference numerals will be used to refer to the same or similar parts. Detailed Implementation

[0064] This invention is not limited to the embodiments described below, but extends to the full scope of the appended claims. That is, the invention may be implemented in different forms and should not be construed as limited to the described embodiments, which are illustrated for illustrative purposes.

[0065] Unless otherwise stated, singular terms may include plural forms.

[0066] A structure described as being formed above or below another structure should be interpreted as including situations where the structures are in contact with each other, and also including situations where a third structure is placed between them.

[0067] When describing temporal relationships, such as when the chronological order of events is described as “after,” “following,” “next,” “before,” etc., this disclosure should be considered to include both consecutive and discontinuous events, unless otherwise stated. For example, unless terms such as “exactly,” “immediately,” or “directly” are used, the description should be considered to include discontinuous cases.

[0068] Although the terms “first,” “second,” etc., may be used herein to describe various elements, these elements are not limited by these terms. These terms are used only to distinguish individual elements. For example, without departing from the scope of the appended claims, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

[0069] Features of different embodiments may be coupled or combined with each other in part or in whole, and may interoperate differently with each other. Some embodiments may be implemented independently of each other, or may be implemented together in a mutually dependent relationship.

[0070] Optical configuration

[0071] Figure 1 An embodiment is illustrated in which a computer-generated hologram is encoded on a single spatial light modulator. The computer-generated hologram is a Fourier transform of the object used for reconstruction. Therefore, a hologram can be described as a Fourier domain, frequency domain, or spectral domain representation of the object. In this embodiment, the spatial light modulator is a reflective liquid crystal on silicon (LCOS) device. The hologram is encoded on the spatial light modulator, and a holographic reconstruction is formed at the playback field, such as a light-receiving surface like a screen or diffuser.

[0072] A light source 110, such as a laser or laser diode, is configured to illuminate the SLM 140 via a collimating lens 111. The collimating lens causes the approximately plane wavefront of the light to be incident on the SLM. Figure 1 In this embodiment, the wavefront is oriented off-normal (e.g., two or three degrees away from a plane that is truly orthogonal to the transparent layer). However, in other embodiments, a generally planar wavefront is provided with normal incidence, and beam splitters are arranged to separate the input and output optical paths. Figure 1In the illustrated embodiment, the arrangement is such that light from the light source is reflected from the mirrored rear surface of the SLM and interacts with the light modulation layer to form an outgoing wavefront 112. The outgoing wavefront 112 is applied to an optics device including a Fourier transform lens 120, the focal point of which is located at a screen 125. More specifically, the Fourier transform lens 120 receives the modulated beam from the SLM 140 and performs a frequency-space transformation to produce a holographic reconstruction at the screen 125. The screen 125 is optional.

[0073] It is worth noting that in this type of hologram, each pixel of the hologram contributes to the overall reconstruction. There is no one-to-one correlation between a specific point (or image pixel) on the playback field and a specific optical modulation element (or hologram pixel). In other words, the modulated light leaving the optical modulation layer is distributed across the entire playback field.

[0074] In these embodiments, the spatial position of the holographic reconstruction is determined by the diopter (focusing) of the Fourier transform lens. Figure 1 In the illustrated embodiment, the Fourier transform lens is a physical lens. That is, the Fourier transform lens is an optical Fourier transform lens and performs a Fourier transform optically. Any lens can act as a Fourier transform lens, but the lens's performance will limit the accuracy of the Fourier transform it performs. Those skilled in the art will understand how to use lenses to perform optical Fourier transforms.

[0075] Holographic computation

[0076] In some embodiments, the computer-generated hologram is a Fourier transform hologram, or simply a Fourier hologram or a Fourier-based hologram, wherein the image is reconstructed in the far field by utilizing the Fourier transform properties of a positive lens. The Fourier hologram is computed by Fourier transforming the desired light field in the reproduction plane back to the lens plane. The Fourier transform can be used to compute computer-generated Fourier holograms.

[0077] Algorithms such as the Gerchberg-Saxton algorithm can be used to compute Fourier transform holograms. Furthermore, the Gerchberg-Saxton algorithm can be used to compute holograms in the Fourier domain (i.e., Fourier transform holograms) based solely on amplitude-only information in the spatial domain (e.g., a photograph). This effectively "retrieves" phase information related to the object from the amplitude-only information in the spatial domain. In some embodiments, the Gerchberg-Saxton algorithm or its variants are used to compute computer-generated holograms from amplitude-only information.

[0078] The Gerchberg-Saxton algorithm takes into account the fact that the intensity cross section I of the beams in planes A and B is known. A (x,y) and IB (x,y) and I A (x,y) and I B The case where (x,y) is correlated via a single Fourier transform. For a given intensity cross section, the approximate phase distribution Ψ in planes A and B is obtained. A (x,y) and Ψ B (x,y). The Gerchberg-Saxton algorithm finds a solution to the problem by following an iterative process. More specifically, the Gerchberg-Saxton algorithm iteratively applies spatial and spectral constraints while repeatedly transferring the representation I between the spatial and Fourier (spectral or frequency) domains. A (x,y) and I B A dataset (x, y) containing amplitude and phase. A corresponding computer-generated hologram in the spectral domain is obtained through at least one iteration of the algorithm. The algorithm is convergent and arranged to produce a hologram representing the input image. The hologram can be an amplitude-only hologram, a phase-only hologram, or a fully complex hologram.

[0079] In some embodiments, the phase-only hologram is computed using an algorithm based on the Gerchberg-Saxton algorithm, such as the algorithm described in British Patents 2498170 or 2501112, the entire contents of which are incorporated herein by reference. However, the embodiments disclosed herein are described by way of example only when calculating the phase-only hologram. In these embodiments, the Gerchberg-Saxton algorithm retrieves the phase information Ψ[u,v] of the Fourier transform of a dataset, which produces known amplitude information T[x,y], where the amplitude information T[x,y] represents the target image (e.g., a photograph). Since amplitude and phase are inherently combined in the Fourier transform, the transformed amplitude and phase contain useful information about the accuracy of the computed dataset. Therefore, the algorithm can be used iteratively with feedback of amplitude and phase information. However, in these embodiments, the phase-only information Ψ[u,v] is used as a hologram to form a holographic representation of the target image at the image plane. The hologram is a dataset of phase values ​​(e.g., a 2D array).

[0080] In other embodiments, an algorithm based on the Gerchberg-Saxton algorithm is used to compute a fully complex hologram. A fully complex hologram is a hologram having amplitude and phase components. A hologram is a dataset (e.g., a 2D array) comprising an array of complex data values, where each complex data value includes an amplitude component and a phase component.

[0081] In some embodiments, the algorithm processes complex data, and the Fourier transform is a complex Fourier transform. Complex data can be viewed as comprising (i) real and imaginary components, or (ii) amplitude and phase components. In some embodiments, the two components of the complex data are processed differently at different stages of the algorithm.

[0082] Figure 2A A first iteration of an algorithm for computing a phase-only hologram, according to some embodiments, is shown. The input to the algorithm is an input image 210 comprising a 2D array of pixel or data values, where each pixel or data value is an amplitude or oscillation value. That is, each pixel or data value of the input image 210 does not have a phase component. Therefore, the input image 210 can be considered as an amplitude-only, oscillation-only, or intensity-only distribution. An example of such an input image 210 is a photograph or a frame of a video comprising a time-series of frames. The first iteration of the algorithm begins with a data formation step 202A, which includes assigning random phase values ​​to each pixel of the input image using a random phase distribution (or random phase seed) 230 to form an initial complex dataset, where each data element of the dataset includes both amplitude and phase. In other words, the initial complex dataset represents the input image in the spatial domain.

[0083] First processing block 250 receives an initial complex dataset and performs a complex Fourier transform to form a complex dataset of Fourier transforms. Second processing block 253 receives the complex dataset of Fourier transforms and outputs a hologram 280A. In some embodiments, hologram 280A is a phase-only hologram. In these embodiments, second processing block 253 quantizes each phase value and sets each amplitude value to 1 to form hologram 280A. Each phase value is quantized according to the phase level that can be represented on the pixel of the spatial light modulator that will be used to "display" the phase-only hologram. For example, if each pixel of the spatial light modulator provides 256 different phase levels, each phase value of the hologram is quantized to one of the 256 possible phase levels. Hologram 280A is a phase-only Fourier hologram representing an input image. In other embodiments, hologram 280A is a fully complex hologram comprising an array of complex data values ​​(each including an amplitude component and a phase component) derived from the received complex dataset of Fourier transforms. In some embodiments, the second processing block 253 constrains each complex data value to one of a plurality of permissible complex modulation levels to form a hologram 280A. The constraint step may include setting each complex data value to the closest permissible complex modulation level in the complex plane. The hologram 280A can be said to represent an input image in the spectral, Fourier, or frequency domain. In some embodiments, the algorithm stops at this point.

[0084] However, in other embodiments, the algorithm continues, such as Figure 2AAs shown by the dashed arrow in the image. In other words, follow... Figure 2A The steps indicated by the dashed arrows are optional (i.e., not essential for all embodiments).

[0085] The third processing block 256 receives the modified complex dataset from the second processing block 253 and performs an inverse Fourier transform to form a complex dataset with an inverse Fourier transform. The complex dataset with the inverse Fourier transform can be said to represent the input image in the spatial domain.

[0086] The fourth processing block 259 receives the complex dataset of the inverse Fourier transform and extracts the distribution of amplitude values ​​211A and the distribution of phase values ​​213A. Optionally, the fourth processing block 259 evaluates the distribution of amplitude values ​​211A. Specifically, the fourth processing block 259 can compare the distribution of amplitude values ​​211A of the complex dataset of the inverse Fourier transform with the input image 510, which itself is, of course, the distribution of amplitude values. If the difference between the distribution of amplitude values ​​211A and the input image 210 is sufficiently small, the fourth processing block 259 can determine that the hologram 280A is acceptable. That is, if the difference between the distribution of amplitude values ​​211A and the input image 210 is sufficiently small, the fourth processing block 259 can determine that the hologram 280A is a sufficiently accurate representation of the input image 210. In some embodiments, for comparison purposes, the distribution of phase values ​​213A of the complex dataset of the inverse Fourier transform is ignored. It will be understood that any number of different methods can be used to compare the distribution of amplitude values ​​211A with the input image 210, and this disclosure is not limited to any particular method. In some embodiments, the mean squared error is calculated, and if the mean squared error is less than a threshold, the hologram 280A is considered acceptable. If the fourth processing block 259 determines that the hologram 280A is unacceptable, further iterations of the algorithm can be performed. However, this comparison step is not required, and in other embodiments, the number of iterations of the algorithm performed is predetermined, preset, or user-defined.

[0087] Figure 2B This represents the second iteration of the algorithm and any further iterations of the algorithm. The distribution of the phase values ​​213A from previous iterations is fed back through the algorithm's processing block. Distributions of amplitude values ​​211A are rejected, favoring the distribution of amplitude values ​​of the input image 210. In the first iteration, data formation step 202A forms a first complex dataset by combining the distribution of amplitude values ​​of the input image 210 with the random phase distribution 230. However, in the second and subsequent iterations, data formation step 202B includes forming a complex dataset by combining (i) the distribution of phase values ​​213A from previous iterations of the algorithm with (ii) the distribution of amplitude values ​​of the input image 210.

[0088] Then, with reference Figure 2AThe same method described is handled by Figure 2B The complex dataset formed in step 202B is used to form the second iterative hologram 280B. Therefore, the description of this process will not be repeated here. The algorithm can stop when the second iterative hologram 280B has been computed. However, any number of further iterations of the algorithm can be performed. It will be understood that the third processing block 256 is only needed if a fourth processing block 259 is required or further iterations are needed. The output hologram 280B generally improves with each iteration. However, in practice, a point is often reached where measurable improvement is no longer observable, or the positive benefits of performing further iterations are offset by the negative impact of the additional processing time. Therefore, the algorithm is described as iterative and convergent.

[0089] Figure 2C This represents an alternative embodiment for the second and subsequent iterations. The distribution of the phase value 213A from the previous iteration is fed back through the algorithm's processing block. The distribution of the amplitude value 211A is rejected, favoring an alternative distribution of the amplitude value. In this alternative embodiment, the alternative distribution of the amplitude value is derived from the distribution of the amplitude value 211 from the previous iteration. Specifically, processing block 258 subtracts the distribution of the amplitude value of the input image 210 from the distribution of the amplitude value 211 from the previous iteration, scales the difference by a gain factor α, and subtracts the scaled difference from the input image 210. This is mathematically expressed by the following equation, where the subscript text and numbers represent the iteration number:

[0090] R n+1 [x,y]=F'{exp(iψ n [u,v])}

[0091] ψ n [u,v]=∠F{ηexp(i∠R n [x,y])}

[0092] η=T[x,y]-α(|R n [x,y]|-T[x,y])

[0093] in:

[0094] F' is the inverse Fourier transform;

[0095] F is the forward Fourier transform;

[0096] R[x,y] is the complex number dataset output by the third processing block 256;

[0097] T[x,y] is the input or target image;

[0098] ∠ is the phase component;

[0099] Ψ is a phase-only hologram 280B;

[0100] η is a new distribution of amplitude value 211B; and

[0101] α is the gain factor.

[0102] The gain factor α can be fixed or variable. In some embodiments, the gain factor α is determined based on the size and rate of the input target image data. In some embodiments, the gain factor α depends on the number of iterations. In some embodiments, the gain factor α is only a function of the number of iterations.

[0103] In all other respects, Figure 2C Implementation examples and Figure 2A and Figure 2B The implementation is the same. It can be said that only the phase hologram Ψ(u,v) includes the phase distribution in the frequency or Fourier domain.

[0104] In some embodiments, a spatial light modulator is used to perform a Fourier transform. Specifically, holographic data is combined with second data that provides optical power. That is, the data written to the spatial light modulator includes holographic data representing an object and lens data representing a lens. The data written to the spatial light modulator can be said to include a diffraction pattern that combines the functions of a hologram and a lens. When displayed on the spatial light modulator and illuminated with light, the lens data simulates a physical lens—that is, it focuses light in the same way as a corresponding physical optical element. Therefore, the lens data provides optical power or focusing power. In these embodiments, [the following can be omitted] Figure 1A physical Fourier transform lens 120 is used. Data representing the lens is known. This data can be referred to as a software lens. For example, a phase-only lens can be formed by calculating the phase delay caused by the optical path length at each point of the lens due to its refractive index and spatial variation. For example, the optical path length at the center of a convex lens is greater than the optical path length at the edge of the lens. An amplitude-only lens can be formed from Fresnel zone plates. In the field of computer-generated holography, it is also known how to combine data representing the lens with a hologram to perform a Fourier transform of the hologram without requiring a physical Fourier lens. In some embodiments, the lensed data is combined with the hologram by simple addition, such as simple vector addition. In some embodiments, a physical lens is used in combination with a software lens to perform the Fourier transform. Alternatively, in other embodiments, the Fourier transform lens is omitted entirely, allowing holographic reconstruction to occur in the far field. In further embodiments, the hologram can be combined with grating data—i.e., data arranged to perform grating functions such as image steering—in the same manner. Again, how to calculate such data is known in the art. The data written into a spatial light modulator can be said to include a diffraction pattern that combines holographic and grating functions. For example, a phase-only grating can be formed by modeling the phase delay caused by each point on the surface of a blazed grating. An amplitude-only grating can be simply superimposed on an amplitude-only hologram to provide angular steering for holographic reconstruction. The second data providing lensing and / or steering can be referred to as an optical processing function or optical processing pattern to distinguish it from the holographic data, which can be referred to as an image forming function or image forming pattern.

[0105] In some embodiments, the Fourier transform is performed jointly by a physical Fourier transform lens and a software lens. That is, the software lens provides some of the optical power that contributes to the Fourier transform, while one or more physical optics provide the remaining optical power that contributes to the Fourier transform.

[0106] In some embodiments, a real-time engine is provided, arranged to receive image data using an algorithm and compute holograms in real time. In some embodiments, the image data is video comprising a sequence of image frames. In other embodiments, the holograms are pre-computed, stored in computer memory, and retrieved as needed for display on an SLM. That is, in some embodiments, a library of predetermined holograms is provided.

[0107] The embodiments described herein are by way of example only and involve Fourier holography and Gerchberg-Saxton type algorithms. This disclosure is equally applicable to Fresnel holography and Fresnel holograms that can be computed using similar methods. This disclosure is also applicable to holograms computed using other techniques, such as point cloud-based methods.

[0108] Optical modulation

[0109] Spatial light modulators can be used to display diffraction patterns, including computer-generated holograms. If the hologram is a phase-only hologram, a spatial light modulator is needed to modulate the phase. If the hologram is a fully complex hologram, a spatial light modulator that modulates both the phase and amplitude can be used, or a first spatial light modulator that modulates the phase and a second spatial light modulator that modulates the amplitude can be used.

[0110] In some embodiments, the light modulation element (i.e., pixel) of the spatial light modulator is a cell comprising liquid crystal. That is, in some embodiments, the spatial light modulator is a liquid crystal device in which the optically active component is liquid crystal. Each liquid crystal cell is configured to selectively provide multiple light modulation levels. That is, each liquid crystal cell is configured at any time to operate at one light modulation level selected from multiple possible light modulation levels. Each liquid crystal cell can be dynamically reconfigured to a light modulation level different from the multiple light modulation levels. In some embodiments, the spatial light modulator is a reflective liquid crystal on silicon (LCOS) spatial light modulator, but this disclosure is not limited to this type of spatial light modulator.

[0111] LCOS devices provide a dense array of light-modulating elements or pixels within a small aperture (e.g., a few centimeters wide). Pixels are typically about 10 micrometers or smaller, resulting in a diffraction angle of a few degrees, meaning the optical system can be compact. The small aperture of an LCOS SLM is much easier to fully illuminate than the larger apertures of other liquid crystal devices. LCOS devices are typically reflective, meaning the circuitry driving the LCOS SLM pixels can be buried beneath the reflective surface. This results in a higher aperture ratio. In other words, the pixels are densely packed, meaning there are virtually no dead zones between pixels. This is advantageous because it reduces optical noise in the playback field. LCOS SLMs use a silicon substrate, which has the advantage of optically flat pixels. This is particularly important for phase modulation devices.

[0112] The following are just examples for reference. Figure 3 To describe a suitable LCOS SLM, an LCOS device is formed using a single-crystal silicon substrate 302. It has a 2D array of square planar aluminum electrodes 301, spaced apart by gaps 301a, arranged on the upper surface of the substrate. Each electrode 301 can be addressed by circuitry 302a buried in the substrate 302. Each electrode forms its own planar mirror. An alignment layer 303 is disposed on the electrode array, and a liquid crystal layer 304 is disposed on the alignment layer 303. A second alignment layer 305 is disposed on a planar transparent layer 306, for example, made of glass. A single transparent electrode 307, for example made of ITO, is disposed between the transparent layer 306 and the second alignment layer 305.

[0113] Each square electrode 301, together with the area covered by the transparent electrode 307 and the intermediate liquid crystal material, defines a controllable phase modulation element 308, commonly referred to as a pixel. Taking into account the space between pixels 301a, the effective pixel area, or fill factor, is the percentage of the total number of optically active pixels. By controlling the voltage applied to each electrode 301 relative to the transparent electrode 307, the properties of the liquid crystal material of the individual phase modulation elements can be altered, thereby providing a variable delay for light incident upon them. The effect is to provide phase-only modulation to the wavefront, i.e., without amplitude effects.

[0114] The described LCOS SLM outputs spatially modulated light in a reflective manner. The advantage of a reflective LCOS SLM is that the signal lines, grating lines, and transistors are located below the mirror, resulting in a high fill factor (typically greater than 90%) and high resolution. Another advantage of using a reflective LCOS spatial light modulator is that the thickness of the liquid crystal layer can be half that required when using a transmissive device. This significantly improves the switching speed of the liquid crystal (a key advantage for projecting moving video images). However, the teachings of this disclosure can also be implemented using a transmissive LCOS SLM.

[0115] Area Scan

[0116] Various methods for using holographic projectors to provide improved image quality and head-up displays have been previously disclosed. Holographic projectors can also be used in LiDAR. WO2018 / 134618 discloses a scanning LiDAR system in which a variable grating function (instead of physical optics such as rotatable prisms) is used to move the holographic replay field to perform continuous scanning of the light footprint across a scene. WO2019 / 224052 discloses a structured light LiDAR system in which a scene is scanned by continuously changing a hologram, the structured light pattern changing with each projection event. This disclosure relates to a further improvement on holographic-based LiDAR systems, wherein multiple sub-regions or partitions of a scene are simultaneously scanned by scanning a structured light pattern comprising an array of light features. Notably, unlike WO2019 / 224052, this disclosure relates to an arrangement in which each sub-region of the scene scanned by a corresponding light feature of the structured light pattern (or light footprint) contains multiple separate fields of view for the detection system. In some embodiments (e.g.) Figure 4 Each light feature is a continuous light region, such as a single light spot. In other words, each light feature is a discrete light region. In other embodiments, each light feature includes multiple discrete light spots. In some embodiments, each light feature is a ray or multiple parallel rays (e.g., Figure 5 (Three parallel scan lines are shown). To avoid confusion, each light feature (or light feature component) can be formed by multiple image pixels.

[0117] During scanning, each light source feature scans its entire sub-region. Typically, no part of a sub-region is illuminated twice; that is, no part of each sub-region is “double-exposed” (or exposed more than once) during the scan. The reader will understand how to configure the size and shape of the light features in each sub-region to achieve scanning of its entire sub-region without double exposure. In some embodiments, the scan is a 2D scan of x and y, where the plane containing the light footprint (at all times during the scan) is the xy plane. In other embodiments, the scan is a 1D scan of x or y.

[0118] Figure 4 A time series comprising multiple light footprints according to an embodiment of the present disclosure is shown, which can be formed by a holographic projector for LiDAR scanning of a scene. The footprints are shown from the perspective of a light detector and thus correspond to the detector's field of view. In this embodiment, each light feature is a single light spot having a quadrilateral shape, and a 2D scan is performed.

[0119] In particular, Figure 4 The diagram illustrates a sequence of sixteen optical footprints, including a first optical footprint 451 formed during a first time interval 401 of the total scan time 400, a second optical footprint 452 formed during a second time interval 402 of the scan time 400, and so on, up to the final sixteenth optical footprint 466 formed during the sixteenth time interval 416 of the scan time 400. As those skilled in the art will understand, for ease of illustration, Figure 4 The third through fifteenth light footprints are not shown. Each light footprint comprises sixteen light spots, such as light spot 430 of the first light footprint 451. As described above, each light footprint is formed in the (zero-order) playback field by a "display event". The display event includes the display of the diffraction pattern on the spatial light modulator and the illumination of the spatial light modulator to form a holographic reconstruction of the light footprint. Optionally, a projection lens can be used to project the light footprint onto the scene. Therefore, each display event corresponds to a "projection event".

[0120] According to this disclosure, the scene (i.e., the field of view) surveyed by LIDAR is divided into multiple sub-regions, such as quadrilateral sub-regions. The sub-regions shown in the figure correspond to illumination at a single plane within the field, i.e., the sub-regions correspond to a solid angle starting from the playback field. In embodiments, the sub-regions are substantially non-overlapping. Typically, the sub-regions are continuous. Multiple sub-regions can form a substantially continuous region of the scene. It should be understood that in embodiments where the holographic reconstruction is formed on the playback plane and then projected onto the scene, the playback plane is similarly divided into corresponding multiple sub-regions. Therefore, although the specification generally uses the term "sub-region" to refer to a sub-region of the scene (i.e., the field of view of the detection system), it can also refer to the corresponding sub-region on which the playback plane on which the holographic reconstruction is formed. It is noteworthy that the position of the sub-region is fixed on the playback plane throughout the scanning process, and therefore also fixed within the scene. According to this disclosure, a portion of each of the multiple sub-regions is simultaneously illuminated by the light features of the projected light footprint.

[0121] exist Figure 4 In one embodiment, each optical feature (in the optical feature array) is a single point of light scanned in the x and y directions.

[0122] Figure 4 An example light footprint according to this disclosure is shown, applicable to surveying a scene divided into multiple consecutive quadrilateral sub-regions. The example light footprint comprises a regular or ordered array of rows and columns of light spots 430. Specifically, the array comprises individual discrete light spots 430 having a quadrilateral shape, these light spots being spatially separated from each other and formed at regular or periodic intervals along the x and y directions on a holographic replay plane, thereby forming in the scene. Each light spot 430 is formed in a corresponding partition or sub-region 440 of the surveyed scene 420. In this embodiment, each sub-region includes one light spot. It can be said that each sub-region receives the light of one light spot 430. In this embodiment, each light spot 430 is arranged to scan its corresponding individual sub-region 440, as further described below. Although the example light footprint forms only a single light spot 430 in each individual sub-region 440, this is not necessary. Other examples of light footprints may form more than one light spot in each sub-region, or adjacent groups of light spots may form larger features, such as horizontal or vertical rays, as will be understood from other examples of light footprints described below.

[0123] exist Figure 4In the example optical footprint, each point 430 in the point array is spatially separated from its adjacent points in the same row by a distance corresponding to the size of the sub-region 440 in the x-direction. Similarly, each point 430 in the point array is spatially separated from its adjacent points in the same column by a distance corresponding to the size of the sub-region 440 in the y-direction. Therefore, each point 430 in the point array of the optical footprint forms a substantially identical position within its corresponding sub-region 440. In other words, each point 430 in the point array forms at the same relative position within its corresponding sub-region 440. For example, in the first optical footprint 451 of the sequence, each point 430 forms at the upper left corner of its corresponding sub-region 440, while in the last optical footprint 466 of the sequence, each point 430 forms at the lower right corner of its corresponding sub-region 440.

[0124] In addition, Figure 4 In the example light footprint, the size of each light spot 430 formed in the scene corresponds to a defined proportion or fraction of the size of a sub-region 440 of the scene. Specifically, each light spot 430 illuminates 1 / n (or at least 1 / n) of the total size of the sub-region 440. It can be said that the area of ​​a quadrilateral light spot 430 is 1 / n of the area of ​​a quadrilateral sub-region 440. The entire sub-region 440 can be scanned by a sequence of n consecutive positions of the corresponding light spots 430. In the illustrated example, n = 16. Therefore, each light spot 430 illuminates one-sixteenth of its corresponding sub-region 440 and moves to its sixteen consecutive scan positions. Figure 4 In the illustrated light footprint sequence, each light spot 430 in the light spot array of the light footprint moves from the upper left corner to the lower right corner through a sequence of consecutive scan positions within its corresponding sub-region 440 in a raster scan order. Therefore, since each sub-region 440 of the scene 420 is simultaneously illuminated by its corresponding light spot 430, it is possible to... Figure 4 The entire area of ​​scene 420 is scanned using a time sequence of sixteen light footprints. Therefore, the total scan can be completed in a scan time of 400, which includes 16 display / projection events or frame intervals 451, 452 to 466.

[0125] Figure 5 Another example of a light footprint according to this disclosure is shown, applicable to surveying a scene divided into multiple consecutive quadrilateral sub-regions. Figure 5 In one embodiment, each optical feature (in the optical feature array) is scanned only as a set of three parallel vertical scan lines in the x direction.

[0126] Example optical footprint 520 comprises a regular or ordered array of three scan lines. That is, each optical feature consists of three scan lines that are spatially separated and parallel to each other. The size of each scan line is equal to the corresponding size of the sub-region. Figure 5In this embodiment, the height (dimension in the y-direction) of each scan line is equal to the height (dimension in the y-direction) of each sub-region. Each sub-region is scanned simultaneously by its three scan lines. The three scan lines of each sub-region collectively scan each portion of its sub-region only once (i.e., no double exposure of any portion of the sub-region). Figure 5 In the example shown, the light feature pattern is arranged to form first, second, and third scan lines 530a, 530b, and 530c in each corresponding sub-region 540 of the scene. For example, each scan line 530a, 530b, and 530c may include Figure 4 A column of four adjacent light spots is used to form a vertical scan line. As described herein, each light spot may include multiple image pixels, so each scan line may be wider than one image pixel. Each first vertical scan line 530a is spatially separated from the second vertical scan line 530b in a corresponding sub-region 540. Each second vertical scan line 530b is spatially separated from the third vertical scan line 530c in a corresponding sub-region 540. The first, second, and third scan lines 530a, 530b, and 530c are formed in an ordered array, i.e., periodically spaced along the x and y directions on the holographic reproduction plane, so that the first, second, and third scan lines 530a, 530b, and 530c are formed at the same relative positions in each sub-region 540 in the scene. Therefore, each first / second / third scan line is spatially separated from the adjacent first / second / third scan lines in the same row. However, there is essentially no spatial spacing in the y direction between each first / second / third scan line 530 in the same column. Each optical feature, comprising a set of first, second, and third scan lines 530a, 530b, and 530c, is formed in a corresponding sub-region 540 of the surveyed scene. It can be said that there is a one-to-one correspondence between each of the first, second, and third scan lines 530a, 530b, and 530c and a single sub-region 540 of the scene. This is because each set of scan lines 530a, 530b, and 530c is arranged to scan its corresponding individual sub-region 540, as further described below. As those skilled in the art will understand, in this example optical footprint, a pattern of multiple light points can be arranged to form a set of scan lines 530a, 530b, and 530c in each individual sub-region 540.

[0127] In some embodiments, m scan lines are used to scan each sub-region, and the interval between adjacent scan lines in each sub-region is equal to the sub-region size divided by m.

[0128] As technicians will understand, it is possible to form (and project) according to Figure 5The example light footprint 520 is a time sequence of light footprints such that each light feature, comprising a set of three scan lines 530a, 530b, and 530c, scans its corresponding sub-region 540. Because each light footprint scans all sub-regions 540 of the scene simultaneously, the sequence of light footprints will scan the entire scene. As those skilled in the art will further understand, in Figure 5 The example uses scan lines instead of Figure 4 The discrete light spots in the example require fewer display events to scan the entire scene in a time series. Therefore, the scan time is reduced. Other example light footprints can use a single scan line provided by forming a single column of light spots in each sub-region, or one or more horizontal scan lines provided by forming one or more rows of light spots in each sub-region. In the case of one or more horizontal scan lines, there is essentially no spatial spacing in the x-direction between each scan line in the same row.

[0129] In one embodiment, each scan line is one image pixel wide (in the x-direction of the vertical scan line), and there are many scan lines in each sub-region. For example, each sub-region may have more than 20 scan lines, such as 32 scan lines per sub-region, and adjacent scan lines of a sub-region may be separated by fewer than 12 image pixels, such as 4 image pixels. In other embodiments, each light feature / sub-region includes multiple light spots, where each light spot comprises only one image pixel. That is, each light spot is formed by only one image pixel. The single image pixel light spot of each sub-region may be separated by, for example, 2 to 8 pixels in the x and y directions, such as 4 pixels. These schemes are advantageous because they require only a very small raster function to cover the entire sub-region (e.g., + / - 2 image pixels), so the entire scene is coarsely mapped in a short time (total scan time) (i.e., no unscanned areas). It is worth noting that this can be accomplished using relatively low-resolution holograms.

[0130] According to this disclosure, a LIDAR system includes a display driver (or equivalent system controller) arranged to control a spatial light modulator of a holographic projector. The display driver is arranged to change the diffraction pattern displayed by the spatial light modulator over time. In particular, the diffraction pattern can change over time to form a time series of light footprints, thereby scanning a scene. Furthermore, the LIDAR system includes a detection system comprising a plurality of photodetector elements arranged to detect light reflected from the scene. The detected light can be processed by the detection system to determine time-of-flight measurements, identify features of the scene from the reflected light, etc., as known in the art.

[0131] In some implementations, the detection system includes an array of photodetectors. In some examples, the multiple photodetectors include charge-coupled device (CCD) cameras, where each photodetector is a separate CCD in an array of CCD elements. In other examples, the multiple photodetectors include an array of single-photon avalanche diodes (SPADs), where each photodetector is a SPAD element in an array of SPAD elements. Any other suitable form of photodetector including an array of photosensitive elements is possible and contemplated.

[0132] According to this disclosure, the detection system is arranged such that each optical detection element detects light from a corresponding individual field of view (“IFOV”) within the scene being surveyed (i.e., the total field of view of the detector). Each optical detection element of the detection system is capable of receiving light from a defined (fixed) region in the scene corresponding to its field of view. Therefore, each individual optical detection element has a corresponding IFOV. The array of optical detection elements is typically static during field-of-view scanning of the LIDAR system.

[0133] As described above, the scene is divided into an array of sub-regions, typically a continuous array of quadrilateral sub-regions, and these sub-regions are scanned individually while forming a time series of light footprints in the scene. It is noteworthy that, according to embodiments of this disclosure, each sub-region of the scene contains multiple areas of view (IFOVs) of the light detection elements. Therefore, each IFOV of the light detection elements corresponds only to a portion of the sub-region of the scene (i.e., arranged to receive light from it). Thus, a sub-array of multiple light detection elements corresponds to each sub-region of the scene (i.e., from which light can be received). Light reflected from the scene illuminated by the light feature array of the light footprints in the light footprint time series will be detected at a time only by a subset of the multiple light detection elements. Specifically, when the light feature array of a light footprint illuminates the area corresponding to its IFOV (i.e., where light is projected), the light detection element will detect reflected light from the scene. Conversely, when the light feature array of a light footprint does not illuminate the area corresponding to its IFOV, the light detection element will not detect reflected light from the scene. According to embodiments, since each continuous light footprint of the projected light footprint time series illuminates a different portion of the sub-region of the scene, different subsets of the light detection elements will detect reflected light during continuous light detection intervals (i.e., exposure times). By providing recovery time to the photodetector after saturation occurs, the saturation problem of the photodetector can be mitigated.

[0134] In some implementations, the IFOV of each photodetector element can correspond to a single light feature—typically a light spot—formed in a sub-region of the scene. In this case, it can be said that there is a one-to-one correlation between each light spot and photodetector element in the light footprint's array of light spots.

[0135] For example, in use Figure 4In the implementation of the light footprint sequence, each light spot 430 illuminates 1 / n of a corresponding sub-region 440 of the scene, and the IFOV of each photodetector element corresponds to 1 / n of the sub-region 440 of the scene. Therefore, for example, each sub-region 440 of the scene is associated with a sub-array of n photodetector elements of the detector array. Figure 4 In the example shown, n = 16, and the scene comprises 16 quadrilateral sub-regions, which corresponds to n × n photodetectors in the detector array (i.e., 16 × 16 = 256 photodetectors). Therefore, during each display / projection event that forms an array of n discrete light points in the scene, according to... Figure 4 In one of the light footprint sequences shown, n photodetectors (i.e., 16 photodetectors) of the detector array will simultaneously receive reflected light, while the remaining (n-1) x n photodetectors (i.e., 15 x 16 = 240 photodetectors) will not detect any reflected light (because their IFOV is not illuminated). Therefore, only a spatially separated subset of the photodetector array, specifically 1 / n of the total number of photodetectors, will detect reflected light at any given time. Furthermore, during each consecutive display event forming the corresponding light footprint in the light footprint sequence, different subsets of the photodetector array will detect reflected light. Therefore, using... Figure 4 The example sequence of light footprints shown indicates that a single light detection element will receive light every 16 display events, thus allowing sufficient recovery time in the event that the light detection element is saturated.

[0136] Similarly, in use Figure 5 In an example implementation of the light footprint, each light spot in a column forming scan lines 530a, 530b, and 530c can correspond to the IFOV of a light detection element. In this example, a subset of the light detection elements of the detector array (comprising three spatially separated columns) can simultaneously receive reflected light from the scene. For example, each scan line may include a column of four light spots, such that the four light detection elements in the corresponding column of the array detector can simultaneously receive reflected light from the corresponding sub-region of the scene. However, different subsets of the light detection elements (corresponding to different columns) will receive reflected light from the scene during consecutive display / projection events, forming a corresponding light footprint for scanning the time series of the light footprint of the scene.

[0137] In the example implementation, the size of each sub-region of the scene is selected such that the size of the corresponding region at the exit window of the LIDAR system is greater than the 7mm aperture diameter. More precisely, a 7mm circle can be fitted within each sub-region at all locations from the exit aperture to the plane of interest in the scene. The 7mm aperture size corresponds to the size of the human iris and is therefore used to ensure compliance with eye safety requirements in laser-based applications. In particular, eye safety requirements typically define a threshold for the maximum power level of the laser allowed by a 7mm aperture size—typically around 1.0mW for a 905nm laser wavelength. Therefore, by matching the size of the sub-regions to be equal to or greater than the 7mm aperture diameter, the power of the light spot illuminating each sub-region within the scene can be controlled such that the power level is below the safety requirement threshold while delivering light several times greater than the eye safety limit into the scene.

[0138] Figures 6A-6D Another example time series including multiple light footprints according to this disclosure is shown, which can be formed by a holographic projector used for LiDAR scanning of a scene.

[0139] Figures 6A-6D The light footprint sequence shown uses with Figure 4 The same example light footprint includes a regular array of rows and columns of discrete light points with quadrilateral shapes. Therefore, this sequence is suitable for surveying a scene 600 divided into multiple consecutive quadrilateral sub-regions 650. Thus, each light point is formed in a corresponding sub-region 650 of the surveyed scene 600. Specifically, a single light point is formed (projected) in each individual sub-region 650 of the scene for its scanning. Multiple sub-regions of the scene can be scanned simultaneously and individually by corresponding light points formed at the same relative position within each sub-region. Figures 6A-6D The first four optical footprints in the time series of 16 optical footprints arranged to fully scan the scene are shown (since the number of light spots / scan positions within sub-region n = 16).

[0140] However, with Figure 4 Compared to the time series of the light footprints shown, the scanning of each sub-region was not performed in raster scan order. Instead, in Figures 6A-6DIn the time series of 16 light footprints, each consecutive light footprint forms a light spot in a defined order at one of 16 consecutive scan positions within its corresponding sub-region to provide a patterned scan. As previously described, the patterned scan comprises a sequence of different positions of the light spot within its corresponding sub-region, in a random or semi-random order or in a defined (time) pattern or order, such that areas of a defined / minimum size within a sub-region of the scene do not remain unlit for a period exceeding a threshold time. In some applications, it may be necessary for areas of a sub-region of the scene with a certain minimum size (e.g., greater than a threshold, such as greater than 20%) not to remain unlit for a period exceeding a threshold time (e.g., greater than a threshold number or threshold time of the display event interval). Areas that remain unlit for such a long time are referred to herein as “blanks”. Those skilled in the art will understand that when a single light spot scans its corresponding sub-region in a raster scan order, as Figure 4 The sequence of light footprints may create gaps in certain areas of the scene. For example, the area in the lower right corner of a sub-region, which is one-quarter of the sub-region's size, will not receive any light during the first to tenth light footprints in the sequence. Therefore, a gap may appear in this area within the time interval of the first 10 display events out of a total of 16 display events corresponding to this sequence (i.e., a gap exists for 5 / 8 of the total scan time 400). This is undesirable because information in the gaps may be lost or discovered too late. Therefore, patterned scanning, where the time sequence of light footprints is ordered to form light spots positioned in the corresponding sub-regions in random / semi-random or defined patterns, prevents gaps from occurring, for example, preventing areas (e.g., one-quarter of a sub-region) from remaining unlit for a threshold number of consecutive display events / time intervals.

[0141] exist Figures 6A-6D In the illustrated patterned scan, each sub-region is divided into four quarter-sized regions (referred to herein as "quarter-regions"). Specifically, the four quarter-regions correspond to the first quarter-region at the upper left corner of the sub-region, the second quarter-region at the upper right corner of the sub-region, the third quarter-region at the lower left corner of the sub-region, and the fourth quarter-region at the lower right corner of the sub-region. Furthermore, in this example, during scanning using patterned scanning, each consecutive light spot forms in a different region within the four quarter-regions of its respective sub-region. As those skilled in the art will understand, other methods are possible and can be anticipated.

[0142] Figures 6A-6D An example area of ​​the scene is shown, including a single illustrative sub-region 650 and portions of adjacent sub-regions—specifically, three adjacent sub-regions in an array of sub-regions, located to the right, below, and lower right of sub-region 650, respectively. For example... Figure 6AAs shown, during the first display interval, the first light footprint of the time series forms a first light spot 601 at the upper left corner of each sub-region 650. The position of the first light spot 601 corresponds to... Figure 4 The first position in the ordered sequence of the grating scan. Therefore, the first optical footprint forms a spot in the first quarter region, as shown. Figure 6A As shown. Figure 6A It also shows the light spots in the light spot array formed simultaneously at equivalent positions (the same relative positions) in each of the three adjacent sub-regions to the right, below, and lower right of sub-region 650. For example... Figure 6B As shown, during the second display interval, the second light footprint of the time series forms a second light spot 602 at the fourth quarter position of sub-region 650. Figure 6C As shown, during the third display interval, the third light footprint of the time series forms a third light spot 603 at the third quarter position of sub-region 650. Figure 6C The image also shows light spots in an array of light spots formed simultaneously at equivalent positions in an adjacent sub-region to the right of sub-region 650. The fourth display event of the time series forms a fourth light spot 602 at the second-quarter position of sub-region 650 during the fourth display interval, as shown. Figure 6D As shown. Figure 6D The image also shows light spots in an array of light spots formed simultaneously at equivalent positions in adjacent sub-regions below sub-region 650. Figure 6B , 6C In each of the 6D models, the locations of light spots formed by previous light footprints in a time series are shown to illustrate how to prevent the formation of blank areas, including regions not illuminated across multiple display intervals / light footprints. Specifically, as... Figure 6D As shown, after the first four light footprints in the time series, the illuminated or surveyed areas are uniformly distributed throughout the scene. In this example, this is achieved by moving the light spot in a random, semi-random order or in a pattern to locate it in different areas of the four quarter regions of the sub-region during the successive light footprints of the sequence.

[0143] Therefore, a light detection and ranging method, "LIDAR," for surveying a scene is provided. The method includes dividing the scene into multiple sub-regions. The method also includes displaying a diffraction pattern comprising a hologram of light footprints on a spatial light modulator. The light footprints comprise an array of light features, such as light spots. The method further includes illuminating the diffraction pattern to form a holographic reconstruction of the light footprints on a holographic replay plane and projecting the reconstructed light footprints into the scene. The method also includes controlling the spatial light modulator to change the diffraction pattern over time, such that each light feature in the regular array of light features scans a corresponding sub-region of the scene. The method also includes detecting light through a plurality of light detection elements, wherein each light detection element is arranged to receive light from a corresponding individual field of view within the scene. The method includes configuring the plurality of light detection elements such that each sub-region of the scene contains multiple individual fields of view for the light detection elements.

[0144] Power / intensity control in individually scanned sub-regions

[0145] According to this disclosure, before scanning begins, the LIDAR system divides the scene (or field of view) into multiple sub-regions for simultaneous, individual scanning. This allows for control of the power used to survey different areas of the scene, thereby controlling the illumination intensity. As those skilled in the art will understand, the higher the power used for illumination, the larger the range (i.e., the greater the distance the light can travel into the scene). Furthermore, for a given range, higher power results in higher illumination intensity of the scene, and therefore higher reflection intensity from the scene, leading to more sensitive and / or more accurate measurements. Thus, the range, accuracy, and / or sensitivity of the scan increase with increasing optical power used. For example, the range can be approximately proportional to the square root of the optical power used.

[0146] Therefore, in some embodiments, the power of optical features (e.g., light spots or scan lines) in each sub-region of the scene is controlled based on the properties / requirements of the corresponding portion of the field of view. For example, optical features formed in a corresponding sub-region at the center of the scene / field of view may require higher power than those formed in a corresponding sub-region at the periphery of the scene / field of view. Specifically, in automotive LiDAR scanning applications, the area "directly in front" of the scene (the center of the field of view) requires high power for long-distance scanning (e.g., long distances to the road ahead), while areas at the periphery of the scene (the edges of the field of view), such as high-angle and / or far from the sides, may require lower power for shorter-distance scanning (e.g., identifying nearby objects, such as roadside signs or hazards).

[0147] Furthermore, in some embodiments, the power of the light features formed in each sub-region is adjusted such that the reflected light detected by the photodetector does not have a wide dynamic range (intensity variation). Specifically, it is desirable that the displayed events have a constant display interval (display time), and each photodetector has a also constant associated exposure time (photosensing interval)—referred to as the "global exposure time." Importantly, the photodetector does not saturate (lacking sufficient recovery time). Therefore, in some embodiments, the detection system monitors the saturation of the photodetector during the time series of the light footprint, and if saturation is detected, provides a feedback signal to the holographic projector accordingly. The display driver is configured to modify the hologram (e.g., recalculate the hologram) to reduce the light power of the associated light features as subsequent light footprints of the time series are formed. Specifically, for those light features used to scan the light footprint of a particular sub-region, the light power (light intensity) is reduced, and for these sub-regions, the feedback signal indicates that the photodetector with IFOV is saturated.

[0148] According to this disclosure, the power of individual optical features in the optical feature array of the optical footprint can be controlled and dynamically adjusted by changing the hologram (e.g., recalculating the hologram—either in real time or by retrieving a predetermined hologram from a database).

[0149] In this embodiment, it is important to not exceed the maximum eye-safe optical power within a given sub-region. Alternatively, in sub-regions where high power is not required for sensing, the optical power can be reduced—for example, to increase optical efficiency.

[0150] Moving image points for LIDAR scanning

[0151] As mentioned above, in Figure 4 , 5 In the examples of 6, LIDAR scanning uses an illumination pattern comprising an array of light features, where each light feature is formed in a corresponding sub-region of the scene, and scanning is performed on that sub-region. This allows for the simultaneous scanning of multiple sub-regions of the scene. This scanning is achieved by forming a time series of light footprints in the scene. In particular, the light footprint sequence can be formed by a corresponding time series of display events of a holographic projector, as described herein, where successive display events move or reposition the array of light spots of the light footprints in the scene.

[0152] In some embodiments, a time series of light footprints is formed by dynamically altering a hologram, which is written to and displayed on the spatial light modulator of a holographic projector, and thus holographically reconstructed on the playback plane. Similarly, the holographic reconstruction can be formed in a scene, or it can be formed on a playback plane including an intermediate plane and the projection lens employed. Specifically, a hologram can be computed for each light footprint in the sequence (i.e., each different location of the light feature array) and sequentially written to the spatial light modulator at the start of each display event to form the light footprint in the scene. The hologram can be computed by a hologram engine and written to the spatial light modulator in real time. Alternatively, the hologram can be pre-computed and stored in a database, and can be retrieved from the database and written to the spatial light modulator in real time.

[0153] In other embodiments, a time series of the light footprint is formed by displaying the same hologram, comprising an array of light features, on the spatial light modulator of a holographic projector. Alternatively, the time series of the light footprint is formed by spatially repositioning the light footprint on the playback plane, thus spatially repositioning the array of light features that forms the holographically reconstructed light footprint. This can be achieved using techniques known as “beam control” or “image control.” In such embodiments, the diffraction pattern written into the spatial light modulator includes a grating function (also called a “software grating”) combined with the hologram of the light footprint, as described above. The grating function determines the position of the playback field on the holographic playback plane, thereby determining the light footprint. Specifically, the diffraction pattern written into the spatial light modulator can include grating data—that is, data arranged to perform the grating function. In the field of computer-generated holography, how to compute grating data and combine it with holographic data representing an image is known. For example, a pure phase grating can be formed by simulating the phase delay caused by each point on the surface of a blazed grating. Amplitude-only holographic gratings can be simply superimposed on an amplitude-only hologram representing an image to provide angular control of the amplitude-only hologram. Therefore, each display event can include writing a diffraction pattern to a spatial light modulator, which includes the same hologram data with different grating data, to move or reposition the holographic reconstruction of the projected light footprint in the scene, thereby forming a continuous light footprint in a sequence. Thus, in these embodiments, by simply changing the grating function of the diffraction pattern (e.g., without changing the hologram of the light footprint), an array of light features formed by the light footprint can be moved to multiple different locations to scan all sub-regions of the scene described herein. This embodiment may be more efficient (e.g., faster—allowing for shorter times between display events) than embodiments that change the diffraction pattern by changing the hologram of the light footprint.

[0154] Compensation for non-uniform intensity in holographic reconstruction

[0155] In an embodiment, the pixels of the spatial light modulator generate an intensity envelope, which results in undesirable brightness non-uniformity on the reproduction plane. Without countermeasures, the intensity envelope effectively determines the maximum brightness of the image pixel formed at each point on the reproduction plane. The intensity envelope is non-uniform. In some embodiments, the pixels of the spatial light modulator are rectangular or square, and the intensity envelope is a sinc function, or more specifically, a sinc function. 2 Function. The intensity envelope can be the first sinc in the x-direction. 2 The function or profile, and the second sinc in the y-direction. 2 Functions or contours. In some cases, the center (i.e., the maximum value) of the intensity envelope lies on the optical axis of the pixel array of the spatial light modulator. That is, a straight line extending perpendicularly from the surface of the pixel array will intersect the reproduction plane at the center of the intensity envelope. Therefore, in a conventional configuration, the center of the zero-order reproduction field is essentially the brightest part of the holographic reconstruction. Thus, for a hologram calculated using a Gerchberg-Saxton-based algorithm and without a grating function for uniform intensity at the reproduction field, the light spot of the LiDAR's optical footprint formed at the center of the (zero-order) reproduction field will have a higher intensity than the light spot of the optical footprint formed at the periphery.

[0156] In some embodiments, the hologram is arranged to compensate for non-uniform intensity envelopes. More specifically, in some embodiments, the target image (light footprint) used to compute the hologram is processed prior to the hologram computation to compensate for the non-uniform intensity envelope that will result from the reconstruction.

[0157] According to some embodiments, a grating function is used to translate the replay field on the playback plane. While the grating function translates the replay field (e.g., an array of optical features), it does not translate the intensity envelope. Therefore, the grating function may cause the optical axis of the spatial light modulator and the projection axis of the holographic reconstruction to become non-coincident. In other words, the point where the projection axis intersects the playback plane can be translated on the playback plane relative to the point where the optical axis intersects the playback plane. Thus, the grating function causes the optical axis and the projection axis to become non-collinear.

[0158] For example, sinc 2Compensation functions can be used to process the target image for projection before computing the hologram. However, if the projected light pattern is moved using a raster function, intensity inhomogeneities due to the intensity envelope are reintroduced. In other words, a compensation function for one playback field location is not applicable to other playback field locations. This is because the intensity envelope in the x and y directions is due to the structure of the spatial light modulator, specifically the pixel shape. Therefore, the intensity envelope remains in the same position on the playback plane, even though the raster function is changed to reposition the light footprint by moving the playback field. In particular, if a hologram is computed for the first light footprint, and the second and subsequent light footprints are formed in the time series of the light footprint using the same hologram by changing the raster function, the (compensated) intensity of the light spot will not match the compensation required for different raster positions relative to the intensity envelope. Therefore, intensity variations will occur. This is as follows: Figure 7 As shown.

[0159] Figure 7 (Bottom) shows the sequence of light footprints 711 and 712, including according to Figure 4 An example light footprint forms an array of light spots, where sinc 2 A compensation function is applied to the first optical footprint to compute the hologram. The same hologram is then used for subsequent optical footprints in the time series. Note that, unlike the previous... Figure 1 Similarly, light footprints 711 and 712 are shown from the perspective of the light detector, thus showing the location of the light spots formed in the corresponding sub-regions of the surveyed scene, as described above. Figure 7 (The top section is a graph showing the intensity of the light spot.) For ease of explanation, Figure 7 The intensity distribution of four groups of light spots in the corresponding sub-region of the first row of the light spot array formed by each of the first four light footprints in the time series is shown, plotted relative to their spatial positions in the x-direction of the playback plane. Each light spot is represented by a bar in the figure. Figure 7 sinc is also shown 2 The intensity envelope 700 and the edges 721 and 722 of the scene surveyed in the x direction (i.e., the boundaries of the field of view).

[0160] like Figure 7 As shown, since according to sinc 2The compensation scheme compensates for the corresponding hologram, and the intensity distribution of the first group of four light spots (corresponding to the first, fifth, ninth, and thirteenth bars in the figure) formed by the first light footprint 711 has substantially the same intensity value. It is noteworthy that the intensity of each of the four light spots in the first group is uniform over its region (including multiple image pixels), as shown by the flat top of the corresponding bar. In other embodiments, each light spot is formed by only one image pixel. However, the intensity distribution of the second group of four light spots (corresponding to the second, sixth, tenth, and fourteenth bars in the figure) formed by the second light footprint 712 differs due to their different positions on the playback plane and therefore in sinc 2 The intensity of the first group of light spots varies from position to position within the intensity envelope of 700, and also differs from each other. Specifically, this is due to their orientation towards sinc. 2 The displacement of the center of the intensity envelope 700 results in light spots in the corresponding first and second sub-regions formed by the second optical footprint 712 having a higher intensity than when they were formed by the first optical footprint 711. Conversely, due to the distance from sinc 2 The shift in the center of the intensity envelope 700 results in the light spots in the corresponding third and fourth sub-regions formed by the second light footprint 712 having lower intensities than when they were formed by the first light footprint 711. The same applies to the four light spots in the third and fourth groups formed by the third and fourth light footprints (not shown). It is noteworthy that the intensity of each light spot in the second, third, and fourth groups is not uniform across its region (i.e., multiple image pixels have different intensities), as indicated by the sloping tops of the corresponding bars, because the conventional sinc 2 The compensation technique does not compensate for intensity variations related to its different locations. Due to this non-uniformity, Figure 7 The difference between the highest and lowest intensity points of the light spot shown is relatively large, as indicated by arrow 730 (the change is approximately 2.3 times).

[0161] Therefore, in some embodiments, the hologram is not constructed by sinc 2 The compensation function is applied to the array of light spots located according to the first light footprint 711 of the sequence, because this causes a relatively large intensity change during scanning, as indicated by arrow 730. Conversely, in the embodiment, when the light spot is substantially located in the middle of its corresponding sub-region, the intensity is reduced when the sinc 2 The compensation function is applied to the target image before calculating the hologram. This improvement is as follows: Figure 8 As shown. From Figure 8 As can be seen, positive and negative grating functions are used to provide the translation of each light spot from the middle of its corresponding sub-region to its relevant scan position within it, based on the light footprint sequence. In contrast, in Figure 7 In this study, only the negative raster function was used to provide the translation of each spot from the upper left of its sub-region, but poor results were obtained by comparing arrows 730 and 732.

[0162] More in detail, Figure 8 It shows the relationship with Figure 7 The intensity distribution of an identical set of four light spots is determined by... Figure 7 Four identical light footprint sequences are formed. However, although the position of the light spot is the same as... Figure 7 The same as the light footprint sequence, but using optimization techniques to compute the holograms used to form the light footprint sequence.

[0163] Therefore, as Figure 8 As shown, the first and second light spots in the first row of the light spot array formed by the first, second, third, and fourth light footprints in the first and second sub-regions (corresponding to the first eight bars in the graph) have a continuously increasing intensity distribution. The third and fourth light spots in the first row of the light spot array formed by the first, second, third, and fourth light footprints in the third and fourth first sub-regions (corresponding to the last eight bars in the graph) have a continuously decreasing intensity distribution. Furthermore, the intensity of each light spot in the four groups is non-uniform across its region, as indicated by the sloping tops of the corresponding bars. Since the hologram is calculated to minimize the intensity variations of the light spots across all footprints in the sequence, Figure 8 The difference between the highest and lowest intensity points of the light spot shown is reduced, as indicated by arrow 732 (the change is approximately 1.5 times). This ensures a more uniform signal-to-noise ratio for LiDAR detection across the entire field of view.

[0164] In these optimized embodiments, each light spot in the target image (used for hologram calculation) is sinc based on the center of its sub-region. 2 Compensation. Furthermore, positive and negative gratings are used to translate each light spot from the center of its sub-regions to provide multi-region scanning according to this disclosure.

[0165] As those skilled in the art will understand, the above-described optimization techniques can be used in conjunction with any time series of optical footprints, including raster scans, line scans, random, semi-random, or patterned sequences, as described herein.

[0166] In other embodiments, intensity variations in the detector's field of view (surveying scene) are minimized by minimizing the magnitude of grating variations during scanning. Specifically, in embodiments that survey a scene by varying the grating function of a diffraction pattern (e.g., grating data) displayed by a spatial light modulator in a series of display events to form a sequence of light footprints, the grating variations should be small. This ensures that the sinc of the holograms used to form the sequence of light footprints is minimized. 2The compensation is closer to the ideal values ​​for all the gratings used. Meanwhile, using as many grating values ​​as possible can be advantageous because this increases the power at a given location in the scene during measurement (thus reducing the contribution of background light to the measurement data), since the light energy guiding each feature of the replay field depends on the amount of image content. Therefore, an alternative and potentially advantageous light footprint pattern is a sparse point grid spanning the entire region of interest (e.g., distributed at 4 times the angular interval between the individual fields of view (IFOVs) in the x and y directions), and then the grating displacement only shifts the points to cover all IFOVs (e.g., 4 gratings in the x direction and 4 gratings in the y direction, where each grating variation corresponds to an angle between IFOVs).

[0167] The uniformity of the detector's field of view center (survey scene) can be further improved by using different (i.e. non-uniform) spot target intensities in the corresponding sub-regions. Figure 14A The target intensity value 1410 of the four light spots in the first row of the light spot array formed in the corresponding sub-region is shown by black lines, for example, through... Figure 4 The hologram of the optical footprints formed by the midpoint between the second and third optical footprints in the sequence. Specifically, each of the four light spots shown in the optical footprints is located at the midpoint of the corresponding sub-region in the x-direction. Figure 14A It also shows the sinc-based 2 The corresponding sinc of the target intensity at different positions of the four light spots within the intensity envelope of 1400. 2 Compensation value, such as the first sinc of block A 2 Compensation value 1420 (shown by the gray line). Figure 14A The target light intensity 1410 is shown to be the same for each light spot and is constant over its corresponding region, as indicated by the solid lines at the top of the bars in each block A to D. 2 Compensation strength—for example, the first sinc of block A 2 The compensation value 1420 significantly increases the target intensity above each light spot (first and fourth light spots) formed in the outer or peripheral sub-regions, and is non-uniform across their respective areas, as indicated by the sloping lines at the top of the bars. However, for each light spot (second and third light spots) formed in the inner or central sub-regions, sinc 2 The compensation intensity only slightly increases above the target intensity of 1410, and is similarly non-uniform in its corresponding region. The sinc of the light spot based on the uniform target intensity value... 2 Calculate the compensation strength value to form a hologram Figure 4 The first to fourth optical footprint sequences resulted in a maximum intensity change of approximately ±15% during the scan, as shown in Figure 1404A. Figure 14B As shown, it illustrates the four scanning positions for each light spot.

[0168] In some embodiments, an alternative method is used, wherein the target intensity value of the light spot formed in the corresponding interior or central sub-region of the scene is higher than the target intensity of the light spot formed in the corresponding exterior or peripheral sub-region of the scene. Figure 14C An alternative embodiment is shown. Figure 14B An equivalent view. It can be seen that, compared to the target intensity 1410 of the first light spot (corresponding to block A) and the fourth light spot (corresponding to block D) illuminating the corresponding outer sub-region (in the x-direction), the second light spot (corresponding to block B) and the third light spot (corresponding to block C) illuminating the corresponding inner sub-region (in the x-direction) have an increased target intensity 1415 (therefore, sinc 2 (Compensation intensity). In particular, the target light intensity of the first and fourth light spots is 1410 and... Figure 14A All the light spots shown have the same target light intensity value, while the target light intensity of the second and third light spots increases by 1415, as indicated by the flat solid line at the top of each corresponding bar. Therefore, the sinc of the first and fourth light spots... 2 Compensation strength profile and Figure 14A The results are the same, but the sinc values ​​of the second and third light spots are different. 2 The compensation strength profile is increased to above Figure 14A The intensity profile shown is indicated by the sloping line at the top of the bar. This is based on the sinc of the non-uniform target intensity of the light spot. 2 Compensation is used to calculate the hologram to form Figure 4 The first to fourth optical footprint sequences cause intensity variations during the scan, such as... Figure 14D As shown, the maximum variation 1404C is reduced to approximately ±8%. It is noteworthy that in this example, the intensity variation is improved (reduced) by increasing the intensity in a sub-region at the scene center (in the x-direction), which is typically desirable for LiDAR applications requiring higher resolution at the scene center. Figure 14A The embodiment of B (with the same target intensity) and Figure 14C The embodiment of D (with non-uniform target intensity) shows the first sinc due to the intensity envelope in the x-direction. 2 This improves the compensation for intensity variations in light features within an optical feature array caused by the function. As those skilled in the art will understand, the same technique can be implemented to improve the compensation for the second sinc due to the intensity envelope in the y-direction. 2 Compensation for intensity changes caused by the function.

[0169] Therefore, with Figure 14BIn contrast, by selecting the target intensity of the light spot in the light footprint used for hologram calculation based on the location of the corresponding sub-region in the surveyed scene, it is possible to reduce the intensity variation of the light spot in the light footprint (especially in the scene center) in a sequence (e.g., when different gratings are applied to the same hologram) from ±15% to ±8%, such as... Figure 14D As shown. This effect of reducing intensity variation is surprising, because one would typically expect to reduce overall uniformity by using the non-uniform target intensity of light spots in a light footprint (holographic image).

[0170] In some examples, the target intensity value of the light spot used to scan the central sub-region of the surveyed scene can be selected based on the intensity value of the light spot used to scan the outer sub-region, with the outer sub-region used to bring them to a raster position near the central sub-region. See again... Figure 14D Each light spot is labeled with the letter A, B, C, or D and the number 1, 2, 3, or 4. The letter A, B, C, or D represents the corresponding sub-region AD, and the number 1, 2, 3, or 4 represents the footprint number in the sequence of footprints 1-4 that form the light spot. The intensities of the light spots labeled A3, A4, D1, and D2 formed in the outer sub-regions A and D immediately adjacent to the central sub-regions B and C can be used to select an appropriate target intensity for the central sub-region (i.e., the target intensity of the second and third sub-regions B and C relative to the first and fourth sub-regions A and D). As those skilled in the art will understand, the appropriate value will depend on the width of the relevant sinc2 function and the number of sub-regions.

[0171] Therefore, when combined with a grating, the relative intensity of the light spot used to scan a sub-region will depend on the intensity of adjacent / nearby sub-regions to provide a substantially uniform light spot intensity, or alternatively, to provide the desired non-uniform light spot intensity as required by the application.

[0172] For ease of description, the above example relates to compensation for intensity non-uniformity caused by the intensity envelope in the x-direction. As those skilled in the art will understand, there is a corresponding intensity variation in the y-direction caused by the intensity envelope. Therefore, the embodiments apply the above principle to compensate for intensity variations in both the x and y directions.

[0173] A reproduction field using a mixture of zero-order and first-order diffracted light

[0174] Holographic reconstructions formed by holographic projectors can include higher-order replicas of the zero-order reproduction field. This is in Figure 9 As shown in the figure, each playback field contains Figure 4 The first optical footprint of the optical footprint sequence is 451. Specifically, Figure 9This illustrates how a first-order reproduction field 901 is formed in the (positive and negative) x and y directions adjacent to the zero-order reproduction field 900. The intensities of the zero-order reproduction field 900 and the first-order reproduction field 901 (from the center point on the reproduction plane corresponding to the propagation axis of the spatially modulated light) are enveloped by the intensity in the x and y dimensions of the reproduction plane (e.g., sinc). 2 (Contour) attenuation. As those skilled in the art will understand, this forms the secondary and subsequent playback fields, far removed from the zero-order playback field; however, for ease of illustration, these are not shown in the text. Figure 9 As shown in the diagram. In this specification, the first-level playback field, together with any subsequent playback fields, is referred to as the "higher-level playback field".

[0175] Higher-order reproduction fields are typically undesirable. Therefore, traditionally, higher-order reproduction fields are blocked within the holographic projector (e.g., by a baffle) to prevent them from reaching the reproduction plane. However, the light forming higher-order reproduction fields can constitute a significant portion of the holographic projector's optical power, corresponding to the power of the light input to the system (e.g., light incident on a spatial light modulator). Therefore, blocking higher-order fields can be considered optically inefficient because optical power is lost from the system.

[0176] As described in this article, a LiDAR system includes a holographic projector that projects holographic reconstructions, forming a light footprint in the scene. The light footprint is the illumination pattern of structured light used to detect or survey a scene contained within the field of view. In some applications, such as automotive applications, LiDAR systems require a relatively high aspect ratio field of view. In particular, such applications may require a relatively large horizontal field of view (e.g., + / - 60°) and a relatively small vertical field of view (e.g., + / - 15°). Optical efficiency, in terms of the proportion of power emitted by the light source used to illuminate the scene, is a crucial metric for LiDAR systems. Systems with low optical efficiency require the use of higher-power light sources (e.g., lasers) to achieve a given illumination in the scene, which increases cost, increases heat dissipation requirements (i.e., size), and reduces overall electrical efficiency. Electrical efficiency is a particularly important consideration in electric vehicle applications.

[0177] Therefore, in some embodiments, the holographic projector forms a light footprint in the scene, wherein the field of view illuminated by the projection (corresponding to the scene being surveyed) includes light from higher-order reproduction fields in addition to light from the primary (zero-order) reproduction field.

[0178] Figure 10A-10D It shows the use of Figure 4 Examples of optical footprint sequences. For instance, each playback field comprises the same discrete array of light spots, which is repositioned on the playback plane in raster scan order, for example, using a software raster, to form an optical footprint sequence, as described herein. Thus, if adjacent optical footprints of a first-order playback field are not blocked from reaching the scene, they are repositioned accordingly.

[0179] exist Figure 10A-10D In the example, the projector's output illumination is formed in a so-called "mixed field of view" 1010. The mixed field of view includes the entire zero-order reproduction field and a portion of each first-order reproduction field on either side (in the + / -x direction). Therefore, the mixed field of view is an extended set of scene sub-regions corresponding to the detector's extended field of view. The mixed field of view is actually a fixed window that defines the extent of the field of view, encompassing the entire zero-order reproduction field (i.e., all grating positions) of all light footprints in the capture sequence, as well as different portions of the two first-order reproduction fields in the + / -x direction for different light footprints in the sequence, depending on the displacement of the software grating.

[0180] Figure 10A (Bottom) shows the first light footprint of the light footprint sequence formed in the scene, which is divided into multiple sub-regions, as shown in the reference above. Figure 4 As stated above. In particular, Figure 10A It shows that according to Figure 4 The first optical footprint 451 of the sequence is located as a spot in the spot array of the optical footprints in its corresponding sub-region. The first optical footprint is formed without applying a grating function, such that the playback field on the playback plane is relative to sinc. 2 The intensity distribution envelope remains unchanged. The hybrid field of view 1010A is formed by an array of sub-regions illuminated by a spot array of light footprints formed by the zero-order reproduction field 1006A, and corresponding portions of the sub-regions adjacent to it in the x-direction illuminated by spot arrays of light sequences formed by the corresponding first-order reproduction fields 1004A and 1008A (in the negative and positive x-directions, respectively). In the example shown, the portion of the adjacent sub-region corresponds to half of each adjacent sub-region. Therefore, in this example, the hybrid field of view extends the field of view of the LIDAR system in the x-direction by the width of the sub-regions (half the width of the sub-regions on each side). Thus, the hybrid field of view 1010A includes a column of light spots from the first-order reproduction field 1008A (i.e., on the right-hand side of the zero-order reproduction field 1006A). However, the hybrid field of view 1010A does not include any light spots from the first-order reproduction field 1004A (i.e., on the left side of the zero-order reproduction field 1006A). Figure 10A (Top) Also shown is the intensity profile of the top row of light spots in the spot array of the first light footprint formed in the hybrid field of view 1010A, which is relative to sinc 2 Intensity variations were compensated for to minimize illumination variations across the scene in the light footprint sequence, as described above. Figure 8 The corresponding sinc is shown. 2 Intensity envelope 1012A (first sinc) 2 (Function) for reference.

[0181] Figure 10B (Bottom) shows in Figure 10AThe second optical footprint is formed in a sequence of optical footprints in multiple sub-regions of the scene, and the optical spots in the optical footprint's spot array are based on... Figure 4 The second optical footprint 452 of the sequence is located in its corresponding sub-region. The second optical footprint is formed by applying a raster function such that, on the playback plane (along the positive x-direction), relative to sinc... 2 The intensity envelope exhibits a shift in the reproducing field. Therefore, the mixed field of view 1010B is formed by an array of sub-regions illuminated by a spot array of light footprints formed by the zero-order reproducing field 1006B, and corresponding portions of the sub-regions adjacent to it in the x-direction illuminated by spot arrays of light footprints formed by the corresponding first-order reproducing fields 1004B and 1008B. Thus, at this position, the mixed field of view 1010B includes a column of light spots from the first-order reproducing field 1008B (i.e., to the right of the zero-order reproducing field 1006B), which in the x-direction... Figure 10A The corresponding column of light spots in the first-order reproduction field 1008A is shifted. However, the mixed field of view 1010B again does not include any light spots in the first-order reproduction field 1004B (i.e., to the left of the zero-order reproduction field 1006B). Figure 10B (Top) Shows the intensity distribution of the top row of light spots in the spot array of the second light footprint formed in the hybrid field of view 1010B. 2 The intensity envelope 1012B is shown for reference.

[0182] Figure 10C (Bottom) shows in Figure 10A The third optical footprint of the optical footprint sequence formed in multiple sub-regions of the scene, the optical spots in the optical footprint's spot array according to Figure 4 The third optical footprint of the sequence is located in its corresponding sub-region. The third optical footprint is formed by applying a grating function such that, relative to sinc... 2 The intensity distribution envelope has a larger reproducible field displacement on the reproduction plane (along the positive x-direction). Therefore, the mixed field of view 1010C is formed by an array of sub-regions illuminated by a spot array of light footprints formed by the zero-order reproduction field 1006C, and corresponding portions of the sub-regions adjacent to it in the x-direction illuminated by spot arrays of light footprints formed by the corresponding first-order reproduction fields 1004C, 1008C. Thus, in this position, the mixed field of view 1010C includes a column of light spots from the first-order reproduction field 1004C (i.e., to the left of the zero-order reproduction field 1006C). However, in this case, the mixed field of view 1010C does not include any light spots from the first-order reproduction field 1008C (i.e., to the right of the zero-order reproduction field 1006C). Figure 10C (Top) Shows the intensity profile of the top row of light spots in the spot array of the third light footprint formed in the hybrid field of view 1010C. 2 The intensity envelope 1012C is shown for reference.

[0183] at last, Figure 10D (Bottom) shows in Figure 10A The fourth optical footprint is formed in a sequence of optical footprints in multiple sub-regions of the scene, and the optical spots in the optical footprint's spot array are based on... Figure 4 The fourth optical footprint of the sequence lies within its corresponding sub-region. The fourth optical footprint is formed by applying a grating function such that the replay field on the playback plane (along the positive x-direction) is relative to sinc. 2 The intensity envelope has a larger displacement. Therefore, the mixed field of view 1010D is formed by an array of sub-regions illuminated by a spot array of light footprints formed by the zero-order reproduction field 1006D and corresponding portions of the sub-regions adjacent to it in the x-direction illuminated by spot arrays of light footprints formed by the corresponding first-order reproduction fields 1004D and 1008D. Therefore, at this position, the mixed field of view 1010D includes a column of light spots from the first-order reproduction field 1004D (i.e., to the left of the zero-order reproduction field 1006D), which in the x-direction... Figure 10C The corresponding column of light spots in the first-order reproduction field 1004D is shifted. However, the mixed field of view 1010D again does not include any light spots in the first-order reproduction field 1008D (i.e., to the right of the zero-order reproduction field 1006D). Figure 10D (Top) Shows the intensity profile of the top row of light spots in the spot array of the fourth light footprint formed in the hybrid field of view 1010D. 2 The intensity envelope 1012D is shown for reference.

[0184] Therefore, in the formation Figure 10A-10D At the four grating positions of the first four light footprints in the shown scanning sequence, first-order light illuminates the additional region (the additional sub-region of the scene) beyond the zero order in the x-direction (positive and negative). Specifically, an additional column of light spots is formed by each light footprint in the sequence to scan the portion of the additional sub-region included in the extended field of view.

[0185] As those skilled in the art will understand, by Figure 10A-10D The combined intensity profile of all the light spots formed by the four light footprints (at the top) shows the relationship with... Figure 7 The illumination at the top (A) is essentially uniform. However, the intensity at the very edges of the field of view is (inevitably) lower than at the center. This reduction in intensity is generally acceptable.

[0186] For example, in automotive LIDAR applications, distance sensing requirements (and therefore the illumination power required to obtain a threshold signal-to-noise ratio) are lower at wide angles in the surveyed scene, as further described below. Figure 11As shown. The use of some first-order beams is particularly suitable for illumination patterns that cover a large area with periodic variations (e.g., a periodic array of light characteristics corresponding to multiple sub-regions) in the x and / or y directions, because the mixed field of view formed by first and zero orders provides a seamless pattern.

[0187] In some embodiments, the hologram may be computed or otherwise configured to control the intensity of the light spots in the zero-order and first-order reproduction fields (which form an array of light spots for each light footprint in the sequence in the mixed field of view). For example, it may be desirable to adjust (e.g., increase or decrease) the intensity of the light spots formed at the first-order, zero-order, or both. As described above, this can be achieved in a manner similar to reducing the intensity variation of the light spot array of the primary (zero-order) light footprint.

[0188] Figure 11 An example of a hybrid field of view including a survey scene in an automotive LIDAR application is shown. The zero-order playback field forms a light footprint in an array of sub-regions 1102 of the scene, including the field of view encompassing the road in front of the vehicle. In the illustrated example, the scene is divided into an array of 16×10 quadrilateral sub-regions, each sub-region being scanned individually by a projected light footprint comprising an array of light features described herein (e.g., an array of discrete light points (e.g., 16×10) or an array of one or more scan lines). The hybrid field of view includes all the sub-regions of the zero order. Additionally, the hybrid playback field includes a pair of columns of 10 sub-regions of the first order to the left of the zero-order 1104L and a pair of columns of 10 sub-regions of the first order to the right of the zero-order 1104R. Therefore, in this example, the hybrid field of view includes the complete first-order sub-regions. Thus, with Figure 10A-10D Compared to the example, the extended region of the mixed field of view receives first-order light throughout the entire light footprint sequence (i.e., at all grating locations).

[0189] Including a portion of the first-order reproduction field (in both the positive and negative x directions) in a hybrid reproduction field / field of view offers several advantages. First, it improves optical efficiency. Specifically, the efficiency of delivering illumination to the scene is enhanced because optical power must be used in the first order due to the holographic diffraction process. By illuminating the scene with at least some of the first-order light, the system's optical efficiency increases. Second, the increased or expanded field of view conveniently provides the high aspect ratio required for scene surveying in automotive applications.

[0190] Therefore, in some embodiments, as described above, the detection system is arranged to detect light spots in an extended hybrid field of view. Specifically, the detection system is arranged to detect light spots in the zero-order reproduction field and at least one light spot in a higher-order reproduction field to extend the field of view of the LIDAR system. Thus, the detection system is capable of detecting light reflected from additional sub-regions of the scene illuminated by the primary light spot.

[0191] In some embodiments, at least one higher-order reproduction field light feature (e.g., a light spot) detectable by the detection system is formed in a sub-region of the scene, which is directly adjacent to a (peripheral) sub-region corresponding to the light feature formed in the zero-order reproduction field. In some examples, the region including the sub-region of at least one higher-order reproduction field constitutes less than 50%, for example, less than 30%, of the total region of at least one higher-order reproduction field. The combined approach of illuminating a sub-region of the scene with a light footprint / illumination pattern comprising a regular array of light features is well-suited for use in conjunction with illuminating the scene using some first-order reproduction fields, since the repeating block pattern is reproduced at the zero-order edges (in the x and y directions).

[0192] In some embodiments, the hybrid field of view may be surrounded by "noise dump" regions on certain sides, such as the top and bottom. British Patent 2501112 describes the concept of noise dump in a holographic system in which a phase hologram is used to project a light pattern; this patent is incorporated herein by reference. In some embodiments, regions forming the edges of zero-order and first-order sub-regions (e.g., forming the hybrid field of view) can be used as noise dumps. For example… Figure 12 A hybrid field of view 1210 illuminated by a light footprint is shown. The hybrid field of view 1210 includes a zero order 1206 and portions 1222 of each first order 1204, 1206 adjacent to each side of the zero order 1206 in the x-direction, similar to... Figure 10A (Bottom). The regions at the top and bottom edges 1222 (which may be inside or outside the corresponding sub-regions of the scene) can be used as noise dumps. The embodiment combines the higher-order playback field in the x-direction and the noise dump in the y-direction, and vice versa, so that the zero-order and higher-order noise dump regions are outside the mixed field of view. In contrast, the reference... Figure 12 If noise dumps are provided to the left and right sides of playback field 1206, they will be within the system's field of view.

[0193] LIDAR System Diagram

[0194] Figure 13 An embodiment is shown that includes a spatial light modulator 1310 and a photodetector 1320, both comprising a holographic projector. The spatial light modulator 1310 is arranged to direct light to a scene 1300, and the photodetector 1320 is arranged to collect reflected light from the scene. The spatial light modulator 1310 is arranged to receive light from a light source (not shown) and output spatially modulated light according to a dynamically variable diffraction pattern comprising a computer-generated hologram presented or "displayed" on the spatial light modulator 1310. Figure 13A spatial light modulator 1310 outputs a first spatially modulated light 1331, which forms a first optical footprint 1351 of a zero-order reproduction field within a scene 1300 based on a first computer-generated hologram (not shown) presented on the spatial light modulator 1310. The first optical footprint 1351 includes an array of optical features comprising light spots. As described herein, a holographic projector can project the spatially modulated light directly onto the scene 1300, or it can include optics, such as magnifying optics and / or projection lenses, to project a holographic reconstruction formed on a reproduction plane within the holographic projector onto the scene 1300. One or more optical elements can be used to extend the reproduction field to a large emission aperture to magnify sub-regions, thereby controlling the size of the field of view and its sub-regions. Figure 13 The diagram shows that all the light spots in the light spot array are simultaneously formed by a first spatially modulated light 1331. As described herein, the photodetector 1320 may include an array detector having a field of view that includes the surveyed scene divided into sub-regions. Each photodetector element in the array has a separate field of view (IFOV) that includes a portion of a sub-region of scene 1100. The first light footprint may be part of a sequence of light footprints including the light spot array as described herein. The system controller 1370 may dynamically change the diffraction pattern displayed on the spatial light modulator 1310 by outputting a drive signal 1372 to form such a sequence of light footprints.

[0195] Figure 13 The image also shows a photodetector 1320 receiving reflected light 1341 from an area in scene 1300 illuminated by a first light footprint 1351. This is for illustrative purposes only. Figure 13Light reflected from only a portion of a scene is shown. For example, light from points in a spot array can be reflected by objects in the scene. The reader will understand that during an illumination event, light may be reflected from multiple portions of the scene—i.e., one frame of a LiDAR system. In response to receiving reflected light 1341, photodetector 1320 outputs a light response signal 1374. System controller 1370 is arranged to receive and process the light response signal 1374 and determine time-of-flight measurements, identify features of the scene from the reflected light, etc., as known in the art. In embodiments, controller 1370 may determine whether the light response signal 1374 indicates the presence of an object in one or more areas of the scene illuminated or “probe” by points in the spot array of the first light footprint. In some embodiments, the property of the light response signal 1374 is a maximum (or peak) intensity or an average (or mean) intensity of the light response signal 1374. In other embodiments, the characteristic of the light response signal 1374 is an intensity variation of the light response signal 1374. The properties of the light response signal can be any property of the light response signal 1374, or any feature of the light response signal 1374, which can provide information about the area or any object in the area detected by the first light footprint 1351. For example, the controller 1370 can determine whether the amplitude of the light response signal 1374 exceeds a threshold. A feedback signal can be provided to the controller 1370 by the photodetector 1320 together with the light response signal 1374. Alternatively, the feedback signal can be provided separately. The photodetector can be a CCD array or a SPAD array, and the light projected onto the scene can be visible light or infrared light.

[0196] The methods and processes described herein can be embodied on a computer-readable medium. The term "computer-readable medium" includes media arranged for temporary or permanent storage of data, such as random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, and cache memory. The term "computer-readable medium" should also be considered to include any medium or combination of media capable of storing instructions for machine execution, such that when the instructions are executed by one or more processors, the machine performs, wholly or partially, any or all of the methods described herein.

[0197] The term "computer-readable medium" also covers cloud-based storage systems. The term "computer-readable medium" includes, but is not limited to, one or more tangible and non-transitory data repositories (e.g., data volumes) in the example forms of solid-state storage chips, optical discs, disks, or any suitable combinations thereof. In some example embodiments, instructions for execution may be transmitted by a carrier medium. Examples of such carrier media include transient media (e.g., propagation signals for transmitting instructions).

[0198] It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the scope of the appended claims. This disclosure covers all modifications and variations within the scope of the appended claims and their equivalents.

Claims

1. A light detection and ranging "LIDAR" system arranged to survey a scene, the system comprising: a spatial light modulator arranged to display a diffraction pattern of a hologram comprising a light footprint, wherein the light footprint comprises an array of light features; a light source arranged to illuminate the diffraction pattern to form a holographic reconstruction of the light footprint, wherein the holographic reconstruction of the light footprint is projected onto the scene; a display driver arranged to control the spatial light modulator and to vary the diffraction pattern over time such that each light feature of the array of light features scans a respective sub-region of the scene; a detection system comprising a plurality of light detection elements, wherein the detection system is configured such that each light detection element detects light from a defined area within the scene corresponding to its respective individual field of view, and each sub-region of the scene contains a plurality of defined areas corresponding to a plurality of individual fields of view.

2. The system of claim 1, wherein, The plurality of sub-regions form a substantially continuous region of the scene, and / or the sub-regions are substantially non-overlapping.

3. The system of claim 1, wherein, Each light feature comprises at least one selected from the group comprising a light point, a pattern of light points, a scan line, and a plurality of parallel scan lines.

4. The system of claim 1, wherein, The display driver is arranged to control the spatial light modulator such that the optical power in each sub-region is eye-safe during its scan.

5. The system of any one of claims 1-4, wherein, The display driver is arranged to vary the hologram of the diffraction pattern in order to scan the sub-regions of the scene.

6. The system of any one of claims 1 to 4, wherein, The holographic reconstruction is formed on a holographic replay plane, and wherein the diffraction pattern further comprises a grating function which determines the position of the light footprint on the holographic replay plane, and the display driver is arranged to vary the grating function of the diffraction pattern in order to scan the sub-regions of the scene.

7. The system of claim 6, wherein, The system is arranged to configure the hologram to compensate for intensity non-uniformities on the holographic replay plane based on a sinc 2 compensation function to compensate for intensity non-uniformities on the holographic replay plane.

8. The system of claim 7, wherein, sinc 2 A compensation function is applied to the light footprint comprising the array of light features, each light feature substantially located in the middle of its respective sub-region.

9. The system of claim 7, wherein, The system is arranged to configure the hologram to increase the intensity of the light features in a central region of the light footprint in order to reduce variations in the intensity of the light features during scanning.

10. The system of claim 6, wherein, The holographic replay plane is an x-y plane which is spatially separated from the spatial light modulator by a propagation distance z, and the grating function comprises an x-direction grating and a y-direction grating.

11. The system of any one of claims 1-4, wherein, The display driver is arranged to vary the diffraction pattern such that each light feature performs a scan at a plurality of positions along a scan path within its sub-region.

12. The system of claim 11, wherein, The scan comprises a raster scan or a line scan of its sub-region.

13. The system of any one of claims 1 to 4, wherein, The display driver is arranged to vary the diffraction pattern such that each light feature performs a patterned scan of its sub-region.

14. The system of any one of claims 1 to 4, wherein, An exposure time associated with each light detection element is constant, and the display driver is further configured to vary the hologram in order to reduce the optical power in a sub-region if a detection signal from that sub-region indicates that the corresponding light detection element is saturated.

15. The system of any of claims 1 to 4, further comprising an optical system arranged to magnify the sub-regions.

16. The system of any one of claims 1 to 4, wherein, The light footprint projected onto the scene is such that the periodicity of the array of light features formed in a zeroth order replay field extends into at least one first order holographic replay field.

17. The system of any one of claims 1 to 4, wherein, The detection system is arranged to detect light features of the zeroth order holographic replay field and light features of a portion of at least one higher order holographic replay.

18. The system of claim 17, wherein, The light features of at least one higher order holographic replay field detected by the detection system are formed in a sub-region which is directly adjacent to the sub-region scanned by the light features of the zeroth order holographic replay field.

19. The system of claim 18, wherein, The area comprising the sub-area of the at least one higher order holographic replay field constitutes less than 50% of the total area of the at least one higher order holographic replay field.

20. The system of claim 18, wherein, The area comprising the sub-area of the at least one higher order holographic replay field constitutes less than 30% of the total area of the at least one higher order holographic replay field.

21. The system of any one of claims 1 to 4, wherein, The system is arranged to configure the hologram such that the intensity of the light features of the array of light features varies with distance from the propagation axis of the projected light.

22. A light detection and ranging, "LIDAR", method for surveying a scene, the method comprising: dividing a field of view into a plurality of sub-areas of the scene; displaying a diffraction pattern of a hologram comprising a light footprint on a spatial light modulator, wherein the light footprint comprises an array of light features; illuminating the diffraction pattern to form a holographic reconstruction of the light footprint and projecting the reconstructed light footprint onto the scene; controlling the spatial light modulator to change the diffraction pattern over time such that each light feature of the array of light features scans a respective sub-area of the scene; detecting light by a plurality of light detection elements, wherein each light detection element is arranged to receive light from a respective individual field of view within the scene, and configuring the plurality of light detection elements such that each sub-area of the scene contains a plurality of individual fields of view of the light detection elements.

Images