Increasing signal-to-noise ratio of pixels of LIDAR system
By periodically adjusting the alignment of IFOV parameters and light in the LIDAR system, the problem of pixel signal-to-noise ratio drop is solved, and the detection accuracy and reliability of the system under different environmental conditions are improved.
Patent Information
- Application Number
- CN202380036946.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-03-03
- Filing Date
- 2023-03-03
- Publication Date
- 2024-12-06
AI Technical Summary
When the LIDAR system detects distant objects under different conditions, the signal-to-noise ratio (SNR) of the pixels is difficult to maintain efficient efficiency, especially because some sub-pixels only sense noise rather than signals, resulting in a decrease in the SNR of the system.
By introducing an alignment unit and a processing unit in the LIDAR system, the alignment of the alignment of the IFOV parameters and light is periodically determined, so as to optimize the output data of the pixels, ensure the collection and processing of signals and improve the signal-to-noise ratio of the system.
It effectively improves the signal-to-noise ratio of the LIDAR system under different environmental conditions, ensures the reliability and accuracy of remote object detection, and enhances the performance of the system.
Smart Images

Figure CN119096163A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates generally to surveying techniques for scanning a surrounding environment, and more particularly, to systems and methods for detecting objects in a surrounding environment using LIDAR technology. Background Art
[0002] With the advent of driver assistance systems and autonomous vehicles, cars need to be equipped with systems that can reliably sense and interpret their surroundings, including identifying obstacles, hazards, objects, and other physical parameters that may affect the vehicle's navigation. To this end, many different technologies have been proposed, including radar, LIDAR, camera-based systems, operating alone or in a redundant manner.
[0003] One consideration for driver assistance systems and autonomous vehicles is the system's ability to determine the surrounding environment across different conditions, including rain, fog, darkness, bright light, and snow. Light detection and ranging systems (LIDAR, also known as LADAR) are examples of technologies that can work well in different conditions by illuminating the object with light and measuring the reflected pulses with a sensor. A laser is an example of a light source that can be used in a LIDAR system. As with any sensing system, in order for LIDAR-based sensing systems to be fully adopted by the automotive industry, the system should provide reliable data capable of detecting distant objects.
[0004] The sensing unit of the LIDAR system may include a plurality of pixels. A pixel may include a plurality of sub-pixels, each of which may include one or more sensing elements, such as but not limited to a photodiode.
[0005] Each sub-pixel can sense noise. Some sub-pixels can also sense signals. At least some other sub-pixels do not sense any signals (and only sense noise).
[0006] The noise can be infrared noise - but the sub-pixels can sense other noise.
[0007] The outputs of all sub-pixels may be summed to provide a pixel output signal.
[0008] When the outputs of all sub-pixels are added, the signal-to-noise ratio (SNR) of the pixel output signal is reduced at least in part due to sensing only the noisy sub-pixels.
[0009] There is an increasing demand to increase the SNR of pixels.
[0010] The systems and methods of the present disclosure are directed to improving the SNR of a pixel. Summary of the invention
[0011] According to some broad aspects of the present disclosure, a LIDAR system with improved SNR may be provided.
[0012] According to one broad aspect, the present disclosure provides a LIDAR system comprising at least one light source configured to scan a selected scene, a sensing unit comprising at least one pixel, the at least one pixel configured to generate output data indicative of light intensity collected by the at least one pixel; the processing unit configured and operative to periodically determine data regarding an alignment metric based on the output data received from the sensing unit, and to change at least one of an IFOV parameter, alignment of collected light reflected from the selected scene, and a readout of the at least one pixel of the sensing unit to improve a signal-to-noise ratio (SNR) of the system.
[0013] Typically, output data may relate to sensor output. Such output data may be in the form of an electrical signal generated by one or more photodetectors upon light detection. In some embodiments, the output data may be an analog signal indicating the sum or average output of a selected number of sub-pixels. In some additional embodiments, the output data may be a digital signal. The output data may be formed by an array of signals, each signal indicating the light intensity impinging on a selected sub-pixel. At least one pixel is preferably placed to collect light reflected from a scene to provide output data indicating light reflected from a scene.
[0014] According to some embodiments, the LIDAR system also includes a scanning unit that includes at least one light deflector positioned to receive collected light reflected from one or more objects in a selected scene and direct the collected light to the sensing unit.
[0015] According to some embodiments, the LIDAR system further comprises an alignment unit configured for aligning the path of the collected light to be incident on the IFOV of the at least one pixel, wherein the alignment unit is connected to a scanning unit comprising at least one light deflector to align the path of the collected light by deflecting the at least one light deflector.
[0016] According to some embodiments, the LIDAR system further comprises an alignment unit configured to align a path of the collected light to impinge on the IFOV of the at least one pixel.
[0017] According to some embodiments, an alignment unit is connected to the sensing unit to align a path of the collected light by changing a position of the at least one pixel.
[0018] According to some embodiments, the LIDAR system further comprises at least one optical element located in the path of the collected or transmitted light, and wherein the alignment unit is connected to the at least one optical element to align the path of the collected light by changing the orientation of the at least one optical element.
[0019] According to some embodiments, at least one pixel includes a plurality of sub-pixels and a readout circuit, and is configured to generate output data regarding light intensity collected by a selected number of sub-pixels among the plurality of sub-pixels, and wherein the processing unit generates operational instructions for selecting an arrangement and number of sub-pixels for generating the output data for changing at least one of the IFOV parameters and alignment.
[0020] According to some embodiments, the periodically determining data regarding the alignment metric comprises determining an average SNR over one or more scans of the scene based on a relationship between one or more signals associated with the collected light reflected from one or more objects in the scene. Typically, the LIDAR system can be configured to periodically determine the alignment metric during normal operation to improve the SNR and align the IFOV and light collection due to changes that may be caused by thermal changes, vibrations, or any other changes in the light collection path.
[0021] According to some embodiments, at least one pixel includes at least a first readout region and a second readout region, which provide readout data about light incident on at least the first region and the second region of the at least one pixel, and the periodic determination of data about the alignment metric includes: determining the relationship between the readouts from the at least first readout region and the second readout region.
[0022] In accordance with some embodiments, at least one pixel is configured to provide a top area readout and a bottom area readout indicating light incident on at least one of a top area and a bottom area or a right area and a left area of an area of the at least one pixel.
[0023] According to some embodiments, at least one pixel is formed by at least a first pixel position and a second pixel position for detecting at least a first portion and a second portion of the illumination spot, and the periodic determination of data about the alignment metric includes: determining a relationship between readouts from the at least two first pixels and the second pixel.
[0024] According to some embodiments, at least one pixel includes one or more additional light detectors, and wherein the sensing unit is aligned to direct the collected light to impinge on the at least one pixel and partially impinge on the one or more additional light detectors, and the periodic determination of data about the alignment metric includes: determining the intensity distribution of the portion of light impinging on the one or more additional detectors.
[0025] According to some embodiments, the at least one pixel includes an arrangement of multiple sub-pixel sensors, and wherein the sensing unit is configured to provide readouts of the multiple sub-pixel sensors, and the periodic determination of data regarding the alignment metric includes: processing the readout distribution of the multiple sub-pixels and determining signal data based on the sub-pixel spatial cluster readouts indicating data regarding the collected light.
[0026] According to some embodiments, a processing unit is configured to receive input data indicating individual readouts of the plurality of sub-pixels, and to process the input data to determine signal collection; the processing comprises: determining a sub-pixel spatial cluster readout indicating data about the collected light, and determining one or more parameters of the object based on the arrangement of the sub-pixel spatial clusters.
[0027] According to some embodiments, the one or more parameters of the object include the object center of mass position, the object dimension along at least one axis, and the object reflectivity.
[0028] According to some embodiments, the processing unit is configured and operable to periodically determine data regarding the alignment metric during typical ongoing operation.
[0029] According to one other broad aspect, the present disclosure provides a method for use in operation of a LIDAR system, the method comprising: receiving output data from at least one pixel, the output data indicating light reflected from a selected scene and incident on the at least one pixel, determining data regarding an alignment metric based on the output data from the at least one pixel, and calibrating light collection of the at least one pixel, the calibrating light collection comprising changing at least one of an IFOV parameter, alignment of collected light reflected from the selected scene, and readout of the at least one pixel of a sensing unit to improve a signal-to-noise ratio (SNR) of the system.
[0030] According to some embodiments, calibrating the light collection includes generating operating instructions for an alignment unit to align a path of the collected light to impinge on an active sensing area of the at least one pixel.
[0031] According to some embodiments, calibrating the light collection includes generating operating instructions for a scanning unit of the LIDAR to align a path of the collected light by deflecting the at least one light deflector.
[0032] According to some embodiments, calibrating the light collection comprises changing a spatial alignment of the at least one pixel relative to a path of the collected light.
[0033] According to some embodiments, at least one pixel includes a plurality of sub-pixels and a readout circuit, and is configured to generate output data regarding light intensity collected by a selected number of sub-pixels among the plurality of sub-pixels, and wherein changing the alignment of the collected light includes selecting an arrangement of sub-pixels to participate in generating the output data.
[0034] According to some embodiments, determining the alignment metric comprises determining an average SNR over one or more scans of the scene based on a relationship between one or more signals associated with collected light reflected from one or more objects in the scene and collected noise.
[0035] According to some embodiments, at least one pixel includes at least a first readout region and a second readout region, which provide readout data about light incident on the at least first region and the second region, and determining the alignment metric includes: determining the relationship between the readouts from the at least first readout region and the second readout region.
[0036] According to some embodiments, at least one pixel is configured to provide a top area readout and a bottom area readout indicative of light incident on a top half and a bottom half of an area of the at least one pixel.
[0037] According to some embodiments, at least one pixel is formed by at least a first pixel position and a second pixel position for detecting at least a first portion and a second portion of the illumination spot, and the periodic determination of data about the alignment metric includes: determining a relationship between readouts from the at least two first pixels and the second pixel.
[0038] According to some embodiments, at least one pixel includes one or more additional light detectors and is aligned to direct the collected light to be incident on the at least one pixel and the one or more additional light detectors, and determining the alignment metric includes: determining the intensity distribution of the light portion incident on the one or more additional detectors.
[0039] In accordance with some embodiments, at least one pixel includes an arrangement of multiple sub-pixel sensors, and wherein the sensing unit is configured to provide a readout of the multiple sub-pixel sensors, and determining the alignment metric includes: processing the readout distribution of the multiple sub-pixels and determining signal data based on the sub-pixel spatial cluster readouts indicating data about the collected light.
[0040] According to some embodiments, processing the readout distribution includes receiving input data indicating individual readouts of the plurality of sub-pixels, and processing the input data to determine a signal collection by determining a sub-pixel spatial cluster readout indicating data about the collected light and determining one or more parameters of an object based on the arrangement of the sub-pixel spatial clusters.
[0041] According to some embodiments, the one or more parameters of the object include an object center of mass position, an object size along at least one axis, and an object reflectivity.
[0042] According to yet another broad aspect, the present disclosure provides a program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform a method for use in operation of a LIDAR system, the method comprising receiving output data from at least one pixel, the output data indicating light incident on the at least one pixel, determining data regarding an alignment metric based on the output data from the at least one pixel, and calibrating light collection by the at least one pixel, the calibrating light collection comprising changing at least one of an IFOV parameter, alignment of the collected light reflected from the selected scene, and readout of the at least one pixel of a sensing unit to improve a signal-to-noise ratio (SNR) of the system. The program storage device may also include instructions for performing additional method actions as described herein.
[0043] According to yet another broad aspect, the present disclosure provides a computer program product comprising a computer usable medium having computer readable program code embodied therein for use in the operation of a LIDAR system, the computer program product comprising computer readable instructions for:
[0044] obtaining output data from at least one pixel;
[0045] determining data regarding an alignment metric based on the output data; and
[0046] calibrating light collection of the at least one pixel;
[0047] wherein the calibrating light collection comprises changing at least one of: an IFOV parameter, an alignment of the collected light reflected from the selected scene, and a readout of the at least one pixel of a sensing unit to improve a signal-to-noise ratio (SNR) of the system. The computer program product may also include computer readable instructions for performing additional method actions as described herein.
[0048] According to yet another broad aspect, the invention provides a method for determining an output of a pixel, the method comprising:
[0049] receiving a pixel output signal during one or more learning periods, wherein the pixel includes a first plurality of sub-pixels; wherein a value of the pixel output signal is based on a value of at least one sub-pixel output signal; and
[0050] A number of pixel output signals to be output from the pixel and a contribution of each of the first plurality of sub-pixels to the pixel output signals are determined based on one or more signal-to-noise ratio (SNR) criteria.
[0051] According to some embodiments, the one or more SNR criteria are selected from obtaining a maximum SNR, providing a maximum SNR under certain circumstances, or providing a maximum SNR under certain misalignments.
[0052] According to some embodiments, the one or more SNR criteria are associated with at least one of a time of day, lighting conditions, date, weather conditions, location, or one or more objects illuminated by the LIDAR system.
[0053] According to some embodiments, the aggregated length of the one or more learning periods is less than one second.
[0054] According to some embodiments, the aggregate length of the one or more learning periods is the duration of a single acquisition.
[0055] According to some embodiments, the aggregate length of the one or more learning periods is less than one minute.
[0056] According to some embodiments, the aggregate length of the one or more learning periods is greater than one hour.
[0057] According to some embodiments, the pixel output signal comprises a plurality of pixel output signals from more than one pixel.
[0058] According to some embodiments, each of the more than one pixel has the same number of sub-pixels.
[0059] According to some embodiments, the difference between the plurality of pixel output signals is used to determine the number of pixel output signals to be output. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate various disclosed embodiments. In the drawings:
[0061] Figure 1 An example of a LIDAR system is shown;
[0062] Figure 2 and Figure 3 Various configurations of the projection unit and its role in the LIDAR system are shown;
[0063] Figure 4-Figure 5 An example of a LIDAR system is shown;
[0064] FIG. 6A to FIG. 6C shows the pixel and sub-pixel arrangement, Fig. 6A shows the arrangement of sub-pixels forming a pixel, Figure 6B shows an illumination point incident on a portion of a pixel, and Figure 6C Another example of an illumination point falling on a portion of a pixel is shown;
[0065] Figure 7 and Figure 8 An example of iterative evaluation of SNR is shown.
[0066] Fig. 9 A pixel and additional circuitry are illustrated, wherein the pixel comprises at least a first pixel region and a second pixel region;
[0067] Fig.10 Pixels and additional circuitry are illustrated, wherein the pixel includes one or more additional photodetectors;
[0068] Fig.11 An example of a pixel and additional circuitry is shown, wherein the pixel is formed from a plurality of sub-pixels;
[0069] Fig.12 Another example of a pixel and additional circuitry is shown;
[0070] Fig.13A and Fig. 13B shows a block diagram describing a method according to some embodiments of the present disclosure; and
[0071] Fig.14 Additional examples showing LIDAR systems DETAILED DESCRIPTION
[0072] The disclosed embodiments may relate to optical systems. As used herein, the term "optical system" broadly includes any system for generating, detecting and / or manipulating light. By way of example only, an optical system may include one or more optical components for generating, detecting and / or manipulating light. For example, a light source, a lens, a reflector, a prism, a beam splitter, a collimator, a polarization optical device, an optical modulator, an optical switch, an optical amplifier, an optical detector, an optical sensor, an optical fiber, a semiconductor optical component, although not necessarily required, may each be a part of an optical system. In addition to one or more optical components, an optical system may also include other non-optical components, such as electrical components, mechanical components, chemical reaction components, and semiconductor components. Non-optical components may cooperate with optical components of an optical system. For example, an optical system may include at least one processor for analyzing detected light.
[0073] Consistent with the present disclosure, the optical system may be a LIDAR system. As used herein, the term "LIDAR system" broadly includes any system that can determine the value of a parameter indicating the distance between a pair of tangible objects based on reflected light. In one embodiment, the LIDAR system may determine the distance between a pair of tangible objects based on reflections of light emitted by the LIDAR system. As used herein, the term "determining the distance" broadly includes generating an output indicating the distance between one or more pairs of tangible objects. The determined distance may represent a physical size between a pair of tangible objects. By way of example only, the determined distance may include a flight line distance between the LIDAR system and another tangible object in the field of view of the LIDAR system. In another embodiment, the LIDAR system may determine the relative speed between a pair of tangible objects based on reflections of light emitted by the LIDAR system. Examples of output indicating the distance between a pair of tangible objects include: a number of standard units of length between the tangible objects (e.g., meters, inches, kilometers, millimeters), a number of arbitrary units of length (e.g., a number of LIDAR system lengths), a ratio between a distance and another length (e.g., a ratio to the length of an object detected in the field of view of a LIDAR system), an amount of time (e.g., given in standard units, arbitrary units, or ratios, such as the time it takes light to travel between the tangible objects), one or more positions (e.g., specified using an agreed-upon coordinate system, specified relative to a known position), and the like.
[0074] The LIDAR system can determine the distance between a pair of tangible objects based on reflected light. In one embodiment, the LIDAR system can process the detection results of the sensor, which creates time information indicating the time period between the emission of the light signal and the time when it is detected by the sensor. This time period is sometimes referred to as the "flight time" of the light signal. In one example, the light signal can be a short pulse whose rise and / or fall time can be detected in reception. Using known information about the speed of light in the relevant medium (usually air), information about the flight time of the light signal can be processed to provide the distance traveled by the light signal between emission and detection. In another embodiment, the LIDAR system can determine the distance based on frequency phase shift (or multi-frequency phase shift). Specifically, the LIDAR system can process information indicating one or more modulation phase shifts of the light signal (for example, by solving some simultaneous equations to give the final measurement). For example, the emitted light signal can be modulated with one or more constant frequencies. At least one phase shift of the modulation between the emitted signal and the detected reflection can indicate the distance traveled by the light between emission and detection. Modulation can be applied to a continuous wave light signal, a quasi-continuous wave light signal, or another type of emitted light signal. Note that the LIDAR system can use additional information to determine distance, such as position information between the projected locations (e.g., relative position), the detection location of the signal (especially if far away from each other), etc.
[0075] In some embodiments, a LIDAR system can be used to detect multiple objects in the environment of the LIDAR system. The term "detecting an object in the environment of the LIDAR system" broadly includes generating information indicating an object reflecting light toward a detector associated with the LIDAR system. If the LIDAR system detects more than one object, the information generated about the different objects can be interconnected, for example, a car driving on a road, a bird sitting on a tree, a person touching a bicycle, a truck moving toward a building. The size of the environment in which the LIDAR system detects objects can vary relative to the implementation. For example, a LIDAR system can be used to detect multiple objects in the environment of a vehicle on which the LIDAR system is mounted, up to a horizontal distance of 100m (or 200m, 300m, etc.), and up to a vertical distance of 10m (or 25m, 50m, etc.). In another example, a LIDAR system can be used to detect multiple objects in a vehicle environment or within a predetermined horizontal range (e.g., 25°, 50°, 100°, 180°, etc.) and up to a predetermined vertical height (e.g., ±10°, ±20°, ±40°-20°, ±90°, or 0°-90°).
[0076] As used herein, the term "detecting an object" may broadly refer to determining the presence of an object (e.g., an object may be present in a certain direction relative to a LIDAR system and / or another reference location, or an object may be present in a certain volume of space). Additionally or alternatively, the term "detecting an object" may refer to determining the distance between an object and another location (e.g., the location of a LIDAR system, a location on the earth, or the location of another object). Additionally or alternatively, the term "detecting an object" may refer to identifying an object (e.g., classifying types of objects such as cars, plants, trees, roads); recognizing specific objects (e.g., the Washington Monument); determining a license plate number; determining the composition of an object (e.g., a solid, a liquid, a transparent material, a translucent material); determining kinematic parameters of an object (e.g., whether it is moving, its speed, its direction of movement, the extension of an object). Additionally or alternatively, the term "detecting an object" may refer to generating a point cloud map, wherein each of one or more points of the point cloud map corresponds to a location in an object or a location on a surface thereof. In one embodiment, the data resolution associated with the point cloud representation of the field of view may be associated with 0.1°×0.1° or 0.3°×0.3° of the field of view.
[0077] Consistent with the present disclosure, the term "object" broadly includes a finite composition of matter that can reflect light from at least a portion of it. For example, an object can be at least partially solid (e.g., a car, a tree); at least partially liquid (e.g., a puddle on the road, rain); at least partially gaseous (e.g., smoke, clouds); made of a large number of different particles (e.g., a dust storm, fog, spray); and can have one or more magnitudes, such as ~1 millimeter (mm), ~5mm, ~10mm, ~50mm, ~100mm, ~500mm, ~1 meter (m), ~5m, ~10m, ~50m, ~100m, etc. Smaller or larger objects and any size between these examples can also be detected. Note that for various reasons, a LIDAR system may detect only a portion of an object. For example, in some cases, light may be reflected from only some sides of an object (e.g., only the side opposite the LIDAR system will be detected); in other cases, light may be projected onto only a portion of an object (e.g., a laser beam projected onto a road or building); in other cases, the object may be partially blocked by another object between the LIDAR system and the detected object; in other cases, the LIDAR's sensor may only detect light reflected from a portion of an object, for example, because ambient light or other interference items interfere with the detection of some portions of the object.
[0078] Consistent with the present disclosure, a LIDAR system may be configured to detect an object by scanning the environment of the LIDAR system. The term "scanning the environment of the LIDAR system" broadly includes illuminating the field of view or a portion of the field of view of the LIDAR system. In one example, scanning the environment of the LIDAR system may be achieved by moving or pivoting a light deflector to deflect light in different directions toward different portions of the field of view. In another example, scanning the environment of the LIDAR system may be achieved by changing the positioning (i.e., position and / or orientation) of a sensor relative to the field of view. In another example, scanning the environment of the LIDAR system may be achieved by changing the positioning (i.e., position and / or orientation) of a light source relative to the field of view. In yet another example, scanning the environment of the LIDAR system may be achieved by changing the position of at least one light source and at least one sensor to move rigidly relative to the field of view (i.e., maintaining the relative distance and orientation of at least one sensor and at least one light source).
[0079] As used herein, the term "field of view of a LIDAR system" may broadly include the range of the observable environment of the LIDAR system in which objects may be detected. Note that the field of view (FOV) of a LIDAR system may be affected by various conditions, such as, but not limited to: the orientation of the LIDAR system (e.g., the direction of the optical axis of the LIDAR system); the position of the LIDAR system relative to the environment (e.g., height above the ground and adjacent terrain and obstacles); operating parameters of the LIDAR system (e.g., transmit power, computational settings, defined operating angles), etc. The field of view of a LIDAR system may be defined, for example, by a solid angle (e.g., defined using angles φ, θ, where φ and θ are angles defined in a vertical plane, e.g., relative to the axis of symmetry of the LIDAR system and / or its FOV). In one example, the field of view may also be limited to a specific range (e.g., up to 200m).
[0080] Similarly, the term "instantaneous field of view" can broadly include the range of the observable environment in which the LIDAR system can detect objects at any given moment. For example, for a scanning LIDAR system, the instantaneous field of view is narrower than the entire FOV of the LIDAR system, and it can be moved within the FOV of the LIDAR system to enable detection in other parts of the FOV of the LIDAR system. Movement of the instantaneous field of view within the FOV of the LIDAR system can be achieved by moving the light deflectors of the LIDAR system (or outside the LIDAR system) to deflect light beams to and / or from the LIDAR system in different directions. In one embodiment, the LIDAR system can be configured to scan a scene in the environment in which the LIDAR system is operating. As used herein, the term "scene" can broadly include some or all objects that are within the field of view of the LIDAR system, in their relative positions and in their current state, for the duration of the operation of the LIDAR system. For example, a scene may include ground elements (e.g., earth, roads, grass, sidewalks, road surface markings), sky, man-made objects (e.g., vehicles, buildings, signs), vegetation, people, animals, light projecting elements (e.g., flashlights, the sun, other LIDAR systems), and the like.
[0081] Any reference to the term "actuator" shall apply mutatis mutandis to the term "manipulator." Non-limiting examples of manipulators include micro-electromechanical systems (MEMS) actuators, voice coil magnets, motors, piezoelectric elements, etc. It should be noted that the manipulator may be incorporated with the temperature control unit.
[0082] The disclosed embodiments may involve obtaining information for generating a reconstructed three-dimensional model. Examples of types of reconstructed three-dimensional models that can be used include point cloud models and polygonal meshes (e.g., triangle meshes). The terms "point cloud" and "point cloud model" are well known in the art and should be interpreted as including a set of data points spatially located in a certain coordinate system (i.e., having an identifiable position in the space described by the corresponding coordinate system). The term "point cloud point" refers to a point in space (which can be dimensionless, or a microcellular space, e.g., 1 cm3), and its position can be described by the point cloud model using a set of coordinates (e.g., (X, Y, Z), (r, φ, θ)). By way of example only, a point cloud model can store additional information for some or all of its points (e.g., color information for the points generated from a camera image). Similarly, any other type of reconstructed three-dimensional model can store additional information for some or all of its objects. Similarly, the terms "polygon mesh" and "triangle mesh" are well known in the art and should be interpreted as including a collection of vertices, edges, and faces that define the shape of one or more 3D objects (such as polyhedral objects), etc. The faces may include one or more of the following: triangles (triangle meshes), quadrilaterals, or other simple convex polygons, as this can simplify rendering. The faces may also include more general concave polygons or polygons with holes. Polygonal meshes may be represented using different techniques, such as: vertex-vertex meshes, face-vertex meshes, wing-edge meshes, and rendering dynamic meshes. Different parts of the polygonal mesh (e.g., vertices, faces, edges) are spatially located in a certain coordinate system directly and / or relative to each other (i.e., have identifiable positions in the space described by the corresponding coordinate system). The generation of the reconstructed three-dimensional model may be implemented using any standard, specialized, and / or novel photogrammetric techniques, many of which are known in the art. Note that the LIDAR system may generate other types of environment models.
[0083] Consistent with the disclosed embodiments, the LIDAR system may include at least one projection unit having a light source configured to project light. As used herein, the term "light source" refers broadly to any device configured to emit light. In one embodiment, the light source may be a laser (such as a solid-state laser, a laser diode, a high-power laser) or an alternative light source (such as a light source based on a light emitting diode (LED)). In addition, the light source 112 shown throughout the accompanying drawings may emit light in different formats, such as light pulses, continuous waves (CW), quasi-CW, etc. For example, one type of light source that can be used is a vertical cavity surface emitting laser (VCSEL). Another type of light source that can be used is an external cavity diode laser (ECDL). In some examples, the light source may include a laser diode configured to emit light having a wavelength between about 650nm and 1150nm. Alternatively, the light source may include a laser diode configured to emit light having a wavelength between about 800nm and about 1000nm, between about 850nm and about 950nm, or between about 1300nm and about 1600nm. Unless otherwise indicated, the term "about" with respect to a numerical value is defined as a variance of up to 5% relative to the stated value. Figure 2 and Figure 3 Reference is made to PCT patent application PCT / IB2020 / 055283 publication number WO20 / 245767 Figure 2 A- Figure 2 C describes additional details regarding the projection unit and the at least one light source, which are incorporated herein by reference.
[0084] Consistent with the disclosed embodiments, the LIDAR system may include at least one scanning unit having at least one light deflector configured to deflect light from a light source so as to scan a field of view. The term "light deflector" broadly includes any mechanism or module configured to deflect light from its original path; for example, a mirror, a prism, a controllable lens, a mechanical mirror, a mechanical scanning polygon, an active diffraction (e.g., a controllable LCD), a Risley prism, non-mechanical electro-optical beam steering (such as manufactured by Vscent), a polarization grating (such as provided by Boulder Nonlinear Systems), an optical phased array (OPA), etc. In one embodiment, the light deflector may include multiple optical components, such as at least one reflective element (e.g., a mirror), at least one refractive element (e.g., a prism, a lens), etc. In one example, the light deflector may be movable to deflect light to different degrees (e.g., discrete degrees, or over a continuous span of degrees). The optical deflector can optionally be controlled in different ways (e.g., deflect to α degrees, change the deflection angle by Δα, move a component of the optical deflector by M millimeters, change the speed at which the deflection angle changes). In addition, the optical deflector can optionally be operable to change the deflection angle within a single plane (e.g., the θ coordinate). The optical deflector can optionally be operable to change the deflection angle within two non-parallel planes (e.g., the θ and φ coordinates). Alternatively or in addition, the optical deflector can optionally be operable to change the deflection angle between predetermined settings (e.g., along a predefined scanning route) or in other ways. Regarding the use of optical deflectors in LIDAR systems, it should be noted that the optical deflector can be used in an outbound direction (also known as a transmit direction or TX) to deflect light from a light source to at least a portion of the field of view. However, the optical deflector can also be used in an inbound direction (also known as a receive direction or RX) to deflect light from at least a portion of the field of view to one or more optical sensors. Reference is now made to PCT patent application PCT / IB2020 / 055283 with publication number WO20 / 245767. Figure 3 A- Figure 3 C describes additional details regarding the scanning unit and at least one light deflector, which are incorporated herein by reference.
[0085] The disclosed embodiments may involve pivoting an optical deflector in order to scan a field of view. As used herein, the term "pivot" broadly includes the rotation of an object (especially a solid object) about one or more rotation axes while substantially keeping the center of rotation fixed. In one embodiment, the pivoting of the optical deflector may include the rotation of the optical deflector about a fixed axis (e.g., an axis), but not necessarily. For example, in some MEMS mirror embodiments, the MEMS mirror may be moved by actuating a plurality of benders connected to the mirror, and in addition to the rotation, the mirror may also undergo some spatial translation. However, such a mirror may be designed to rotate about a substantially fixed axis, and therefore, consistent with the present disclosure, it is considered to be pivoted. In other embodiments, some types of optical deflectors (e.g., non-mechanical-electro-optical beam steering, OPA) do not require any moving parts or internal movement to change the deflection angle of the deflected light. Note that any discussion relating to moving or pivoting an optical deflector also applies, mutatis mutandis, to controlling the optical deflector so that it changes the deflection behavior of the optical deflector. For example, controlling a light deflector may cause a change in the deflection angle of a light beam arriving from at least one direction.
[0086] The disclosed embodiments may involve receiving a reflection associated with a portion of a field of view corresponding to a single instantaneous position of an optical deflector. As used herein, the term "instantaneous position of an optical deflector" (also referred to as "the state of an optical deflector") broadly refers to the location or position of at least one controlled component of an optical deflector at an instantaneous point in time or in space over a short time span. In one embodiment, the instantaneous position of an optical deflector may be measured relative to a reference frame. The reference frame may relate to at least one fixed point in the LIDAR system. Or, for example, the reference frame may relate to at least one fixed point in a scene. In some embodiments, the instantaneous position of an optical deflector may include some movement of one or more components (e.g., mirrors, prisms) of the optical deflector, typically to a limited extent relative to a maximum degree of change during scanning of the field of view. For example, scanning the entire field of view of the LIDAR system may include changing the deflection of light over a span of 30°, and the instantaneous position of at least one optical deflector may include an angular displacement of the optical deflector within 0.05°. In other embodiments, the term "instantaneous position of a light deflector" may refer to the position of a light deflector during the acquisition of light that is processed to provide data for a single point of a point cloud (or another type of 3D model) generated by the LIDAR system. In some embodiments, the instantaneous position of a light deflector may correspond to a fixed position or orientation in which the deflector is paused for a short time during illumination of a particular sub-region of the LIDAR field of view. In other cases, the instantaneous position of a light deflector may correspond to a certain position / orientation along a scanning range of positions / orientations of the light deflector, which the light deflector traverses as part of a continuous or semi-continuous scan of the LIDAR field of view. In some embodiments, the light deflector may be moved so that during a scanning cycle of the LIDAR FOV, the light deflector is located at a plurality of different instantaneous positions. In other words, the deflector may move through a series of different instantaneous positions / orientations during the time period in which the scanning cycle occurs, and the deflector may reach each different instantaneous position / orientation at a different time during the scanning cycle.
[0087] Consistent with the disclosed embodiments, the LIDAR system may include at least one sensing unit having at least one sensor configured to detect reflections from an object in the field of view. The term "sensor" broadly includes any device, element, or system capable of measuring a characteristic of an electromagnetic wave (e.g., power, frequency, phase, pulse timing, pulse duration) and generating an output related to the measured characteristic. In some embodiments, at least one sensor may include a plurality of detectors consisting of a plurality of detection elements. At least one sensor may include one or more types of light sensors. Note that at least one sensor may include multiple sensors of the same type, which may differ in other characteristics (e.g., sensitivity, size). Other types of sensors may also be used. Combinations of several types of sensors may be used for different reasons, such as improving detection over a range span (particularly at close range); improving the dynamic range of the sensor; improving the temporal response of the sensor; and improving detection under changing environmental conditions (e.g., atmospheric temperature, rain, etc.).
[0088] In one embodiment, at least one sensor includes a SiPM (silicon photomultiplier), which is a solid-state single-photon sensitive device constructed from an array of avalanche photodiodes (APDs) and single-photon avalanche diodes (SPADs) used as detection elements on a common silicon substrate. In one example, the typical distance between SPADs can be between about 10 μm and about 50 μm, where each SPAD can have a recovery time between about 20 ns and about 100 ns. Similar photomultipliers from other non-silicon materials can also be used. Although SiPM devices operate in digital / switching mode, SiPMs are analog devices because all microcells can be read in parallel, so that signals can be generated over a dynamic range from a single photon to hundreds and thousands of photons detected by different SPADs. It should be noted that the outputs from different types of sensors (e.g., SPADs, APDs, SiPMs, PIN diodes, photodetectors) can be combined together into a single output that can be processed by the processor of the LIDAR system. Reference is made below to the present application Figure 4 and Figure 5 Reference is made to PCT patent application PCT / IB2020 / 055283 publication number WO20 / 245767 Figure 4 A- Figure 4 C describes additional details regarding the sensing unit and the at least one sensor, which are incorporated herein by reference.
[0089] In accordance with the disclosed embodiments, the LIDAR system may include or communicate with at least one processor configured to perform different functions. The at least one processor may constitute any physical device having a circuit that performs a logical operation on one or more inputs. For example, the at least one processor may include one or more integrated circuits (ICs), including application specific integrated circuits (ASICs), microchips, microcontrollers, microprocessors, all or part of a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or other circuits suitable for executing instructions or performing logical operations. The instructions executed by the at least one processor may be preloaded into a memory integrated with or embedded in a controller, for example, or may be stored in a separate memory. The memory may include a random access memory (RAM), a read-only memory (ROM), a hard disk, an optical disk, a magnetic medium, a flash memory, other permanent, fixed or volatile memory, or any other mechanism capable of storing instructions. In some embodiments, the memory is configured to store information representing data about an object in the environment of the LIDAR system. In some embodiments, the at least one processor may include more than one processor. Each processor may have a similar configuration, or the processors may have different configurations that are electrically connected or disconnected from each other. For example, the processors may be separate circuits or integrated in a single circuit. When more than one processor is used, the processors may be configured to operate independently or collaboratively. The processors may be coupled electrically, magnetically, optically, acoustically, mechanically, or by other means that allow them to interact. Figures 6A to 6C Reference is made to PCT patent application PCT / IB2020 / 055283 publication number WO2020 / 245767 Figure 5 A- Figure 5 C describes additional details regarding the processing unit and at least one processor, which is incorporated herein by reference.
[0090] Figure 1A LIDAR system 100 is shown that includes a projection unit 102, a scanning unit 104, a sensing unit 106, and a processing unit 108. The LIDAR system 100 can be mounted on a vehicle 110. Consistent with embodiments of the present disclosure, the projection unit 102 can include at least one light source 112, the scanning unit 104 can include at least one light deflector 114, the sensing unit 106 can include at least one sensor 116, and the processing unit 108 can include at least one processor 118. In one embodiment, the at least one processor 118 can be configured to coordinate the operation of the at least one light source 112 with the movement of the at least one light deflector 114 in order to scan a field of view 120. During a scanning cycle, each instantaneous position of the at least one light deflector 114 can be associated with a particular portion 122 of the field of view 120. Additionally, the LIDAR system 100 can include at least one optional optical window 124 for directing light projected toward the field of view 120 and / or receiving light reflected from objects in the field of view 120. The optional optical window 124 can be used for different purposes, such as collimation of the projected light and focusing of the reflected light. In one embodiment, the optional optical window 124 can be an opening, a flat window, a lens, or any other type of optical window.
[0091] Consistent with the present disclosure, the LIDAR system 100 can be used in autonomous or semi-autonomous road vehicles (e.g., cars, buses, vans, trucks, and any other land vehicles). An autonomous road vehicle with a LIDAR system 100 can scan its environment and drive to a destination vehicle without human input. Similarly, the LIDAR system 100 can also be used in autonomous / semi-autonomous aircraft (e.g., UAVs, drones, quadcopters, and any other aerial vehicles or equipment); or in autonomous or semi-autonomous watercraft (e.g., boats, ships, submarines, or any other vessels). Autonomous aircraft and watercraft with a LIDAR system 100 can scan their environment and navigate to a destination autonomously or using a remote human operator. According to one embodiment, a vehicle 110 (road vehicle, aircraft, or vessel) can use a LIDAR system 100 to help detect and scan the environment in which the vehicle 110 is operating.
[0092] It should be noted that the LIDAR system 100 or any of its components can be used with any of the example embodiments and methods disclosed herein. In addition, although some aspects of the LIDAR system 100 are described with respect to an exemplary vehicle-based LIDAR platform, the LIDAR system 100, any of its components, or any of the processes described herein can be applicable to LIDAR systems of other platform types.
[0093] In some embodiments, the LIDAR system 100 may include one or more scanning units 104 to scan the environment around the vehicle 110. The LIDAR system 100 may be attached or mounted to any portion of the vehicle 110. The sensing unit 106 may receive reflections from the surroundings of the vehicle 110 and transmit a reflection signal indicative of light reflected from an object in the field of view 120 to the processing unit 108. Consistent with the present disclosure, the scanning unit 104 may be mounted to or incorporated into a bumper, fender, side panel, spoiler, roof, headlight assembly, taillight assembly, rearview mirror assembly, hood, trunk, or any other suitable portion of the vehicle 110 that is capable of accommodating at least a portion of the LIDAR system. In some cases, the LIDAR system 100 may capture a complete surround view of the environment of the vehicle 110. Thus, the LIDAR system 100 may have a 360 degree horizontal field of view. In one example, as Figure 1 As shown, the LIDAR system 100 may include a single scanning unit 104 mounted on a rooftop vehicle 110. Alternatively, the LIDAR system 100 may include multiple scanning units (e.g., two, three, four, or more scanning units 104), each having a field of view such that the overall horizontal field of view is covered by a 360-degree scan around the vehicle 110. Those skilled in the art will appreciate that the LIDAR system 100 may include any number of scanning units 104 arranged in any manner, each scanning unit 104 having a field of view of 80° to 120° or less, depending on the number of units employed. In addition, a 360-degree horizontal field of view may also be obtained by mounting multiple LIDAR systems 100 on the vehicle 110, each LIDAR system 100 having a single scanning unit 104. However, it should be noted that one or more LIDAR systems 100 do not have to provide a full 360-degree field of view, and in some cases, a narrower field of view may be useful. For example, a vehicle 110 may require a first LIDAR system 100 with a 75° field of view looking forward of the vehicle, and may require a second LIDAR system 100 with a similar FOV looking backward (optionally with a lower detection range). It should also be noted that different vertical field of view angles may also be implemented.
[0094] Projection unit
[0095] Figure 2 and Figure 3 Various configurations of the projection unit 102 and its role in the LIDAR system 100 are depicted. Specifically, Figure 2 is a diagram showing a projection unit 102 having a single light source; Figure 31 is a diagram showing multiple projection units 102 with multiple light sources aimed at a common light deflector 114. Those skilled in the art will appreciate that the depicted configuration of the projection unit 102 may have many variations and modifications. Figure 2 C- Figure 2 Non-limiting examples are provided in G, which is incorporated herein by reference.
[0096] Figure 2 An example of a bi-static configuration of the LIDAR system 100 is shown, wherein the projection unit 102 includes a single light source 112. The term "bi-static configuration" broadly refers to a LIDAR system configuration in which the projected light leaving the LIDAR system and the reflected light entering the LIDAR system traverse substantially different optical paths. In some embodiments, the bi-static configuration of the LIDAR system 100 can include separating the optical paths by using completely different optical components, by using parallel but not completely separated optical components, or by using the same optical components for only a portion of the optical paths (the optical components can include, for example, windows, lenses, mirrors, beam splitters, etc.). Figure 2 In the example depicted in A, the bi-static configuration includes a configuration in which outgoing light and incident light pass through a single optical window 124, but the scanning unit 104 includes two light deflectors, a first light deflector 114A for the outgoing light and a second light deflector 114B for the incident light (the incident light in the LIDAR system includes emitted light reflected from objects in the scene and may also include ambient light arriving from other sources).
[0097] In this embodiment, all components of the LIDAR system 100 may be contained within a single housing 200, or may be distributed among multiple housings. As shown, the projection unit 102 is associated with a single light source 112, which includes a laser diode 202A (or one or more laser diodes coupled together) configured to emit light (projected light 204). In a non-limiting example, the light projected by the light source 112 may be a wavelength between about 800nm and 950nm, with an average power between about 50mW and about 500mW, with a peak power between about 50W and about 200W, and a pulse width between about 2ns and about 100ns. In addition, the light source 112 may optionally be associated with an optical component 202B for manipulating the light emitted by the laser diode 202A (e.g., for collimation, focusing, etc.). Note that other types of light sources 112 may be used, and the present disclosure is not limited to laser diodes. In addition, the light source 112 can emit its light in different formats, such as light pulses, frequency modulation, continuous wave (CW), quasi-CW, or any other form corresponding to the specific light source employed. The projection format and other parameters can be changed from time to time by the light source based on different factors (such as instructions from the processing unit 108). The projected light is projected toward the exit deflector 114A, which serves as a steering element for directing the projected light in the field of view 120. In this example, the scanning unit 104 also includes a pivotable return deflector 114B, which directs photons (reflected light 206) reflected back from the object 208 within the field of view 120 toward the sensor 116. The reflected light is detected by the sensor 116, and information about the object (e.g., the distance to the object 212) is determined by the processing unit 108.
[0098] In this figure, the LIDAR system 100 is connected to a host 210. Consistent with the present disclosure, the term "host" refers to any computing environment that can interface with the LIDAR system 100, which can be a vehicle system (e.g., part of the vehicle 110), a test system, a security system, a monitoring system, a traffic control system, a city modeling system, or any system that monitors its surroundings. Such a computing environment may include at least one processor and / or may be connected to the LIDAR system 100 via a cloud. In some embodiments, the host 210 may also include an interface to external devices, such as cameras and sensors configured to measure different characteristics of the host 210 (e.g., acceleration, steering wheel deflection, reverse drive, etc.). Consistent with the present disclosure, the LIDAR system 100 may be fixed to a stationary object associated with the host 210 (e.g., a building, a tripod) or a portable system associated with the host 210 (e.g., a portable computer, a movie camera). Consistent with the present disclosure, the LIDAR system 100 may be connected to the host 210 to provide the output of the LIDAR system 100 (e.g., a 3D model, a reflectivity image) to the host 210. Specifically, the host 210 can use the LIDAR system 100 to help detect and scan the environment of the host 210 or any other environment. In addition, the host 210 can integrate, synchronize, or otherwise use the output of the LIDAR system 100 with the output of other sensing systems (e.g., cameras, microphones, radar systems). In one example, the LIDAR system 100 can be used by a security system.
[0099] The LIDAR system 100 may also include a bus 212 (or other communication mechanism) that interconnects the subsystems and components used to communicate information within the LIDAR system 100. Optionally, the bus 212 (or another communication mechanism) may be used to interconnect the LIDAR system 100 with a host computer 210. Figure 2 In the example of FIG. 1A, the processing unit 108 includes two processors 118 to adjust the operation of the projection unit 102, the scanning unit 104, and the sensing unit 106 in a coordinated manner based at least in part on information received from internal feedback of the LIDAR system 100. In other words, the processing unit 108 can be configured to dynamically operate the LIDAR system 100 in a closed loop. A closed-loop system is characterized by having feedback from at least one element and updating one or more parameters based on the received feedback. In addition, the closed-loop system can receive feedback and update its own operation based at least in part on the feedback. A dynamic system or element is a system or element that can be updated during operation.
[0100] According to some embodiments, scanning the environment around the LIDAR system 100 may include illuminating the field of view 120 with light pulses. The light pulses may have parameters such as pulse duration, pulse angular dispersion, wavelength, instantaneous power, photon density at different distances from the light source 112, average power, pulse power intensity, pulse width, pulse repetition rate, pulse sequence, pulse duty cycle, wavelength, phase, polarization, etc. Scanning the environment around the LIDAR system 100 may also include detecting and characterizing various aspects of reflected light. The characteristics of the reflected light may include, for example, flight time (i.e., time from emission to detection), instantaneous power (e.g., power signature), average power over the entire return pulse, and photon distribution / signal over the return pulse period. By comparing the characteristics of the light pulses with the characteristics of the corresponding reflections, the distance and possible physical characteristics, such as the reflection intensity of the object 212, may be estimated. By repeating this process across multiple adjacent portions 122, an entire scan of the field of view 120 may be achieved in a predefined pattern (e.g., a grating, Lissajous, or other pattern). As discussed in more detail below, in some cases, LIDAR system 100 may direct light to only some portions 122 of field of view 120 during each scanning cycle. These portions may be adjacent to each other, but need not be.
[0101] In another embodiment, the LIDAR system 100 may include a network interface 214 for communicating with a host computer 210 (e.g., a vehicle controller). Communication between the LIDAR system 100 and the host computer 210 is represented by a dashed arrow. In one embodiment, the network interface 214 may include an integrated services digital network (ISDN) card, a cable modem, a satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, the network interface 214 may include a local area network (LAN) card to provide a data communication connection to a compatible LAN. In another embodiment, the network interface 214 may include an Ethernet port connected to a radio frequency receiver and transmitter and / or an optical (e.g., infrared) receiver and transmitter. The specific design and implementation of the network interface 214 depends on the communication network through which the LIDAR system 100 and the host computer 210 are intended to operate. For example, the network interface 214 may be used to provide the output of the LIDAR system 100, such as a 3D model, operating parameters of the LIDAR system 100, etc., to an external system. In other embodiments, the communication unit may be used to receive instructions from an external system, receive information about the inspected environment, receive information from another sensor, etc.
[0102] Figure 3An example of a monostatic configuration of a LIDAR system 100 including a plurality of projection units 102 is shown. The term "monostatic configuration" broadly refers to a LIDAR system configuration in which the projected light exiting the LIDAR system and the reflected light entering the LIDAR system pass through substantially similar optical paths. In one example, the outgoing light beam and the incident light beam can share at least one optical component through which both the outgoing light beam and the incident light beam pass. In another example, the outgoing light can pass through an optical window (not shown) and the incident light radiation can pass through the same optical window. The monostatic configuration can include a configuration in which the scanning unit 104 includes a single light deflector 114 that directs the projected light toward the field of view 120 and the reflected light toward the sensor 116. As shown, both the projected light 204 and the reflected light 206 strike the asymmetric deflector 216. The term "asymmetric deflector" refers to any optical device with two sides that is capable of deflecting a light beam striking it from one side in a direction different from the direction it deflects a light beam striking it from a second side. In one example, the asymmetric deflector does not deflect the projected light 204 and deflects the reflected light 206 toward the sensor 116. An example of an asymmetric deflector may include a polarizing beam splitter. In another example, the asymmetric 216 may include an optical isolator that allows light to pass in only one direction. An illustration of the asymmetric deflector 216 is shown in FIG. Figure 2 D. Consistent with the present disclosure, a monostatic configuration of LIDAR system 100 may include an asymmetric deflector to prevent reflected light from striking light source 112 and direct any reflected light toward sensor 116, thereby increasing detection sensitivity.
[0103] exist Figure 3 In an embodiment of the present invention, the LIDAR system 100 includes three projection units 102, each projection unit 102 having a single light source 112 aimed at a common light deflector 114. In one embodiment, multiple light sources 112 (including two or more light sources) can project light having substantially the same wavelength, and each light source 112 is generally associated with a different region of the field of view (represented in the figure as 120A, 120B, and 120C). This enables scanning of a wider field of view than can be achieved using light sources 112. In another embodiment, multiple light sources 102 can project light having different wavelengths, and all light sources 112 can be directed to the same portion (or overlapping portions) of the field of view 120.
[0104] Figure 4 An exemplary LIDAR system 100 including a beam splitter 1110 is shown. Figure 4As shown, the LIDAR system 100 may include a monolithic laser array 950 configured to emit one or more laser beams (e.g., 1102, 1104, 1106, 1108). Before the beams 1102, 1104, 1106, and / or 1108 are incident on a beam splitter 1110, the one or more laser beams may be collimated by one or more collimators 1112. The beam splitter 1110 may allow the laser beams 1102, 1104, 1106, and / or 1108 to pass through and be incident on deflectors 1121, 1123, which may be configured to direct the laser beams 1102, 1104, 1106, and / or 1108 toward a FOV 1170. Although in the embodiment of FIG. Figure 4 Only two deflectors 1121 , 1123 are shown, but it is contemplated that the LIDAR system 100 may include more than two deflectors 1121 , 1123 configured to direct one or more of the light beams 1102 , 1104 , 1106 and / or 1108 toward the FOV 1170 .
[0105] One or more objects in FOV 170 may reflect one or more of light beams 1102, 1104, 1106, and / or 1108. Figure 4 As shown, the reflected beams may be represented as laser beams 1152, 1154, 1156, and / or 1158. Although the reflected laser beams 1152, 1154, 1156, and / or 1158 are Figure 4 1154, 1156, and / or 1158 may be directed toward beam splitter 1110 by deflectors 1121, 1123, and / or another deflector. When beams 1152, 1154, 1156, and / or 1158 reach beam splitter 1110, beam splitter 1110 may be configured to direct reflected beams 1152, 1154, 1156, and / or 1158 received from FOV 1170 toward detector 1130 via lens 1122. Although Figure 4 Four beams are shown emitted by the monolithic laser array 950, but it is contemplated that the monolithic laser array 950 may emit any number of beams (eg, fewer or more than four).
[0106] In some embodiments, the beam splitter is configured to redirect each of the plurality of laser beams and pass through a plurality of reflected beams received from the field of view of the LIDAR system. As an example, Figure 5An exemplary LIDAR system 100 is shown, which may include a monolithic laser array 950, a collimator 1112, a beam splitter 1110, deflectors 1121, 1123, lenses and / or filters 1122, and a detector 1130. The monolithic laser array 950 may emit one or more laser beams 1102, 1104, 1106, and / or 1108, which may be collimated by one or more collimators 1112 before being incident on the beam splitter 1110.
[0107] The beam splitter 1110 may be configured to direct one or more of the laser beams 1102, 1104, 1106, and / or 1108 toward the deflectors 1121, 1123, which in turn may be configured to direct one or more of the laser beams 1102, 1104, 1106, and / or 1108 toward the FOV 1170. One or more objects in the FOV 1170 may reflect one or more of the laser beams 1102, 1104, 1106, and / or 1108. The reflected laser beams 1152, 1154, 1156, and / or 1158 may be directed by the deflectors 1121, 1123 to be incident on the beam splitter 1110. It is also contemplated that some or all of the reflected laser beams 1152 , 1154 , 1156 and / or 1158 may reach the beam splitter 1110 without being directed toward the beam splitter 1110 by the deflectors 1121 , 1123 .
[0108] like Figure 5 As shown in FIG. 1 , the beam splitter 1110 can be configured to allow reflected laser beams 1152, 1154, 1156, and / or 1158 to pass through the beam splitter 1110 toward the detector 1130. One or more lenses and / or filters 1122 can receive the reflected laser beams 1152, 1154, 1156, and / or 1158 and direct them toward the detector 1130. Although Figure 5 The monolithic laser array 950 is shown enabling four beams, but it is contemplated that the monolithic laser array 950 can emit any number of beams (eg, fewer or more than four). DETAILED DESCRIPTION
[0110] The accompanying drawings, which are incorporated into and constitute a part of the present disclosure, illustrate various disclosed embodiments. The following detailed description refers to the accompanying drawings. Where possible, the same reference numerals are used in the drawings and the following description to refer to the same or similar parts. Although several illustrative embodiments are described herein, modifications, adaptations, and other implementations are possible. For example, the components shown in the accompanying drawings may be replaced, added, or modified, and the illustrative methods described herein may be modified by replacing, reordering, removing, or adding steps to the disclosed methods. Therefore, the following detailed description is not limited to the disclosed embodiments and examples. On the contrary, the appropriate scope is defined by the attached content.
[0111] Improve pixel signal-to-noise ratio
[0112] There may be a method for determining an output of one or more pixels. The method may include collecting output data from the one or more pixels, determining an alignment metric indication, and changing at least one of: a corresponding alignment of the collected light and an IFOV for output data collection from the one or more pixels, and one or more parameters of the IFOV for output data collection.
[0113] A method for determining an output of a pixel may also be provided, the method may include: (a) receiving a pixel output signal during one or more learning periods, wherein the pixel includes a first plurality of sub-pixels; wherein a value of the pixel output signal is based on a value of at least one sub-pixel output signal; and (b) determining a number of pixel output signals to be output from the pixel and a contribution of the first plurality of sub-pixels to each pixel output signal based on one or more signal-to-noise ratio (SNR) criteria.
[0114] The value of a pixel output signal may be a function (sum, weighted sum, average, etc.) of one or more sub-pixel output signals. A sub-pixel may or may not contribute to a certain pixel output signal.
[0115] One or more SNR criteria may be selected from obtaining maximum SNR, providing maximum SNR under certain circumstances, or providing maximum SNR under certain misalignments.
[0116] The one or more SNR criteria may be associated with at least one parameter of time of day, lighting conditions, date, weather conditions, location, or one or more objects illuminated by the LIDAR system.
[0117] The aggregate length of one or more learning periods may be less than a second, less than a minute, greater than a minute, greater than an hour, less than an hour, less than a day, greater than a day, less than a week, greater than a week, less than a month, greater than a month, etc. A learning period may be the duration of a single acquisition.
[0118] The method may comprise configuring the pixel in dependence upon the determination - setting the number of pixel output signals and setting the contribution of the first plurality of sub-pixels to each pixel output signal.Different pixel output signals may be affected by output signals of different sub-pixels.
[0119] A technical solution for improving the SNR of a pixel including sub-pixels may be provided.
[0120] This improvement may take into account other considerations - such as optical module misalignment, captured context (context captured by the pixel), etc. This technical solution may provide a trade-off between SNR and the extent of the instantaneous field of view of the pixel.
[0121] A pixel may include a SiPM (silicon photomultiplier) or any other solid-state device including an array of avalanche photodiodes (APD, SPAD, etc.) on a common silicon substrate or any other device capable of measuring a characteristic (e.g., power, frequency) of an electromagnetic wave and generating an output (e.g., a digital signal) related to the measured characteristic. A subpixel may include a single element of the array (e.g., a single SPAD).
[0122] Assume that the pixel includes a first plurality (Nsp) of sub-pixels, and the pixel can output any number of pixel output signals - between one (a single pixel output) to a second plurality of pixel output signals.
[0123] Each pixel output signal may represent one or more sub-pixel output signals.
[0124] a. For example, each of the Nsp sub-pixels may have a pixel output signal to provide Nsp pixel output signals.
[0125] b. For yet another example, only a single pixel output signal may be provided, which may be a function (eg, a sum or weighted sum) of all sub-pixel output signals.
[0126] c. For yet another example, there may be at least one pixel output signal that is the sum of any combination of some sub-pixel output signals.
[0127] refer to FIG. 6A to FIG. 6C . Fig. 6A A pixel 202 comprising fifteen sub-pixels 201 ( 1 , 1 ) to 201 ( 3 , 5 ) arranged in five columns and three rows is shown; Figure 6B illustrates an illumination spot 209 formed by the collected light incident on the pixel 202; and Figure 6C Illustrated is an illumination point 208 that may be formed by reflection of light from some object (typically a small object) impinging on the pixel 202. A pixel may include any number of sub-pixels, any number of rows, and any number of columns.
[0128] The pixel 202 output signal may be equal to the sub-pixel output signal. The pixel output signal may be equal to the sum of all fifteen sub-pixel 201 output signals. The pixel output signal may be the sum of any selected number of sub-pixel 201 output signals, which in this example is between two sub-pixel output signals and fourteen sub-pixel output signals. In some embodiments, as described in further detail below, the pixel output may be an output array of its sub-pixels.
[0129] If a pixel outputs two or more pixel output signals, different pixel output signals may represent sums of different combinations of sub-pixel output signals.
[0130] The summation of the sub-pixel output signals can be done in the analog domain.
[0131] The summation of the sub-pixel output signals can be done in the digital domain.
[0132] One or more summations of the sub-pixel output signals may be done in the analog domain, and one or more other summations may be done in the digital domain.
[0133] A method may be provided to determine the number of pixel output signals (eg, between 1 and a second plurality) and the content of each output pixel output signal. Content - which sub-pixel output signals contribute to the pixel output signal.
[0134] The determination may be performed to satisfy one or more SNR criteria, e.g., to obtain a maximum SNR, to provide a maximum SNR in a particular scenario, to provide a maximum SNR under certain misalignments, etc. Any reference to a maximum value may apply, mutatis mutandis, to a submaximum SNR. The context may refer to, for example, at least one of a time of day, lighting conditions, date, weather conditions, location, one or more objects illuminated by the LIDAR system, etc.
[0135] The determination may be based on a signal generated by the pixel during one or more learning periods, the signal having a length that may vary between a fraction of a second, less than a minute, greater than a minute, less than an hour, greater than an hour, and the like.
[0136] The determination may be based on a relatively long (eg, more than a minute or an hour) learning period suitable for optimizing misalignments due to slowly changing environmental parameters and affecting system alignment over a longer time frame, such as ambient temperature effects.
[0137] The pixel may belong to an optical module of the LIDAR system. The determination may be performed solely by the optical module, may be determined by a computerized system other than the optical module, or may be determined by a collaboration between the optical module and other computerized systems.
[0138] Return to reference Fig. 6A , the pixel 202 is illustrated as including fifteen sub-pixels 201(1,1) to 201(3,5) arranged in five columns and three rows. A pixel may include any number of sub-pixels, any number of rows, and any number of columns. Multiple sub-pixels may be operated together to provide output data indicative of an instantaneous FOV (IFOV). More specifically, the projection unit 102, the scanning unit 104, and the sensing unit 106, as well as any optical elements used in the system, may preferably be aligned so that an illumination light beam emitted by the projection unit 102 toward an object and collected and directed by the scanning unit 104 to be detected by at least one pixel of the sensing unit 106 is aligned with at least one pixel of the sensing unit.
[0139] The alignment of the corresponding elements of the LIDAR system can be a mechanical alignment, which is associated with the position and orientation of the elements along a common optical path. Additionally or alternatively, the alignment can be provided by appropriate readout of multiple sub-pixels 201 from the pixel 202 that collects the reflected signal. In addition, any sub-pixel 201 used for readout rather than specifically collecting light reflected from an object in the scene effectively adds noise to the readout data. This is because the sub-pixel can generally collect ambient photons originating from the environment rather than reflected light 206.
[0140] Misalignment may have various causes including, for example, thermal fluctuations, system vibrations, scattering elements such as dust, and any other causes. Such misalignment may result in a situation where the signal readout from at least one pixel includes a certain number of sub-pixels that are not aligned with the illumination spot of the reflected light, and the sum of the reflected light may not be collected by the sub-pixels being read.
[0141] also, Figure 6B and Figure 6C Two cases are shown where only a portion of the full area of a pixel is illuminated by light reflected from an object (pass signal).
[0142] For example, Figure 6BA reflected light spot 209 is shown incident on a pixel to form a rectangular spot 209 that "covers" the center sub-pixels 201(1,2) to 201(1,4), 201(2,2) to 201(2,4), and 201(3,2) to 201(3,4), and only barely "covers" the outer sub-pixels 201(1,1), 201(1,5), 201(2,1), 201(2,5), 201(3,1), and 201(3,5). In this case, it may be beneficial to ignore the outer sub-pixels in the readout data because the outer sub-pixels 201(1,1), 201(1,5), 201(2,1), 201(2,5), 201(3,1) may contribute to noise but generally do not contribute to the collected signal.
[0143] In another example, Figure 6C A reflected light spot is shown incident on pixel 202 to form a rectangular spot 208 that diagonally covers the area of pixel 202. The light spot completely "covers" subpixel 201(3,1), covers most of subpixels 201(3,2), 201(3,3), 201(2,2), 201(2,3), 201(2,4), 201(2,5), and barely covers subpixels 201(3,4), 201(3,5), 201(1,4), and 201(1,5). Subpixels 201(1,1) to 201(1,3) are not covered at all.
[0144] In this case, it may be beneficial to ignore sub-pixels 201 ( 1 , 1 ) to 201 ( 1 , 3 ) and it may be beneficial to ignore sub-pixels 201 ( 3 , 4 ), 201 ( 3 , 5 ), 201 ( 1 , 4 ) and 201 ( 1 , 5 ).
[0145] For example, by aligning one or more optical elements or by selecting sub-pixels 201 that participate in reading out data from the pixel 202, the determination of the alignment of the illumination point on at least one pixel 202 can be accomplished using optimization to increase the signal-to-noise ratio. The optimization can be performed over time using a trial-and-error process to determine the best subset of sub-pixels to sum for each pixel. In some further embodiments, the optimization can utilize a predetermined alignment metric based on the relationship between the output signals of at least one pixel, the alignment metric being average data or instantaneous data over one or more scans.
[0146] To ensure that no signal is lost, the detector or pixel size can be designed to be larger than the expected pixel. This allows the entire reflected signal to impinge on the pixel, which can be summed. However, pixels that do not detect the target only contribute noise to the overall signal, which is why determining correlated pixels can improve detection. To increase the signal-to-noise ratio, selected portions of the pixel can be used to contribute to the output data, while other areas of the pixel may not be summed and may be ignored in the output data.
[0147] To this end, the LIDAR system of the present disclosure utilizes determining an alignment metric that indicates the alignment of an illumination spot 209 or 208 formed by the collected light incident on at least one pixel 202 with the area of the pixel 202 that is read to provide output data. Using the alignment metric, the present disclosure provides for changing the alignment of the readout of the at least one pixel 202 and the collected light to improve the signal-to-noise ratio (SNR) of the LIDAR system. The readout of at least one pixel and the alignment of the collected light can be performed by various techniques, including, for example, optimization of the alignment.
[0148] Optimization can include signals (i.e., reflected or received echoes) that provide a high confidence for the optimization, such as signals from targets within a specific distance range from the LIDAR system. For example, the distance range can be 100-200m or 150-250m from the LIDAR system. The optimization can ignore detections outside of this range. The system can exclude specific targets. For example, the system can exclude targets that saturate sub-pixels or targets with low signals because this may complicate the optimization. Alternatively, the optimization can include only signals with a specific area in the FOV. For example, if a region of interest (ROI) is defined, only signals received from targets in the ROI can be included.
[0149] As described above, the optimization can be directed to optimizing the alignment by changing the alignment of the scanning unit 104 or the alignment of the sensing unit 106. In some embodiments, the LIDAR system can utilize one or more optical elements located in the path of the collected light, such as prisms, lenses, reflectors, etc. In such embodiments, the optimization can operate to change the alignment of the collected light by changes in the orientation of the one or more optical elements used. In addition, in some embodiments, as described above, the optimization can utilize changes in the sub-pixels used in reading out the output data from at least one pixel of the sensing unit, and select one or more sub-pixels to be included or not included when providing the output signal.
[0150] For example, the optimization may utilize an initial set of sub-pixels used in setting the output data readout. Such an initial set may include all sub-pixels, a set of sub-pixels located at a central portion of a pixel area, or any other initial set determined based on system design. In different optimization steps, some sub-pixels may be deactivated so that their detection does not contribute to the sum of all sub-pixels. This may be done on a row / column basis. Figure 7 The seven steps 301-307 of column-by-column SNR evaluation are shown, and Figure 8 Five steps 311 - 315 of row-by-row evaluation of the SNR are shown.
[0151] Thus, in some embodiments utilizing SNR-based optimization, if the SNR increases when a row is excluded, then the row may be excluded from the summation. If the SNR decreases when the row is excluded, then the row should be included in the summation. If the SNR increases when a column is excluded, then the column may be excluded from the summation. If the SNR decreases when a column is excluded, then the column should be included in the summation.
[0152] Although a single row / column optimization is shown in this example, the optimization can evaluate any number of rows or columns, for example, 2, 3 or more rows / columns can be excluded / included to evaluate whether the SNR increases or decreases. Alternatively, non-array-shaped sub-pixel regions or single pixels can be used instead. Each step of the evaluation can be repeated any number of times to obtain a confidence level in the calculation. Once the row and column combination that produces the maximum SNR is determined based on the local optimum, the determined sub-pixels should be summed until another change is detected.
[0153] The SNR may be determined using an average reflectivity or an average confidence level over multiple frames that sample the scene over a duration where the scene is similar.
[0154] This process can be repeated continuously to monitor misalignment and continuously optimize the detection IFOV. However, since the optimization process of the IFOV may affect the integrity of the measurements detected while the optimization is being performed, the frequency of monitoring should be considered.
[0155] In addition, in some embodiments of the present disclosure, the present technology provides for periodically determining data about an alignment metric based on output data received from a sensing unit, and changing at least one of: an IFOV parameter, alignment of collected light reflected from a selected scene, and a readout of the at least one pixel of the sensing unit to improve the signal-to-noise ratio (SNR) of the system. As described above, this can be performed by an optimization process. In some further embodiments, the optimization process can utilize data about the distribution of light incident on at least one pixel.
[0156] refer to Fig. 9 , Fig.10 and Fig.11 , which illustrates the configuration of at least one pixel 221 and corresponding readout circuitry 230 according to some embodiments of the present disclosure. Fig. 9 , Fig.10 and Fig.11 Components 221 of an optical module including pixels 202, interface 203, readout circuitry 230, and local processor 231 are shown, and illustrate different embodiments of the present disclosure that may be implemented alone or in combination thereof.
[0157] Fig. 9 201A and 201B. Each of the first and second regions may also be formed by a plurality of sub-pixels 201(i, j), such as Fig. 9 As shown, or can be operated as a single photosensitive area. In addition, in some embodiments, the first area 201A and the second area 201B can be formed by a first pixel and a second pixel.
[0158] In this context, the LIDAR system is typically aligned to provide collected light incident on the sensing module 221 so that the reflected light spot is incident on at least the first pixel region 201A and the second pixel region 201B. In addition, the sensing module 221 can be connected to the readout circuit 230 through the interface 203 to provide output data, which includes data about the light collected by each of at least the first pixel (or pixel region) 201A and the second pixel (or pixel region) 201B. The readout circuit 230 and the local processor 231 apply certain processing to the output data collected by the sensing module and determine the relationship between the output signals of the first pixel 201A and the second pixel 201B (or pixel region). The local processor 231 or the processing unit 108 of the LIDAR system uses the relationship between the light intensity incident on different regions, different pixels or sub-pixels 201A and 201B of the sensing module 221 to determine the alignment metric of the light collection. For example, if the light intensity collected by the first pixel region 201A is higher than the light intensity collected by the second pixel region 201B, the alignment metric may indicate that the alignment is offset upward, and a downward alignment offset may be required. As indicated herein, the alignment offset may be provided by offsetting the IFOV of light collection performed by the sensing module 221 (e.g., by selecting sub-pixels of at least one pixel that participate in readout of the collected light, or by offsetting / aligning light collection as described herein).
[0159] Fig.10An additional configuration of a sensing module 221 and at least one pixel 202 is shown. In this configuration, the at least one pixel 202 includes a primary pixel 201 and an arrangement of one or more additional photodetectors 204(1) to 204(5) located next to the photo-sensing area of the pixel 201. The additional photodetectors may be placed on either side of the primary pixel 201 and may include a selected number of photodetectors.
[0160] In this configuration, the LIDAR system can be configured to align light collection if at least a portion of the illumination spot generated when the collected light is incident on the sensing module 221 extends beyond the photosensitive area of the primary pixel 201 so that some of the collected light can be incident on one or more of the additional light detectors 204(1) to 204(5). Thus, the readout circuit 230 receives output data from the primary pixel 201 and the additional light detectors 204(1) to 204(5) and provides the corresponding output data to the local processor 231. The local processor 231 operates to determine alignment by determining the intensity distribution of the light portion incident on the additional detectors 204(1) to 204(5). For example, if for the light collected at a selected relevant distance (e.g., 150m), the maximum light intensity is collected by the additional detector 204(3) and a specific light intensity is detected by the additional detectors 204(2) and 204(4), then a properly aligned IFOV can be determined. Detecting a change in the light distribution for the associated delay (associated target distance) of the collected light and the expected distribution of the additional detector can change the alignment metric, which indicates the need to align the offset to the corresponding direction. As indicated herein, the alignment offset can be provided by shifting the IFOV of the light collection performed by the sensing module 221 (e.g., by selecting sub-pixels of at least one pixel that participate in the readout of the collected light, or by shifting / aligning the light collection as described herein).
[0161] Fig.11 An additional exemplary configuration of a sensing module 221 and at least one pixel 202 thereof is shown. In this embodiment, at least one pixel 201 is formed by a sub-pixel array, exemplified as a 5×3 sub-pixel array 201(1,1) to 201(3,5). The readout circuit 230 is configured to collect readouts from each of the different sub-pixels 201(i,j) to enable separate processing of separate sub-pixel readouts.
[0162] In order to determine output data for at least one pixel 202, the readout circuit 230 or the local processor 231 may be operable to determine a spatial arrangement of sub-pixels that provide output data indicative of light incident thereon. The process may utilize an initial condition, namely, the collected light reflected from the object generally generates an illumination spot that is incident on a local sub-pixel cluster. This is because ambient photons may be incident on different sub-pixels without forming a particular cluster. Therefore, detection of a spatial cluster of sub-pixels that provide output data indicative of light incident thereon provides data about the collected light. In addition, in order to increase the signal-to-noise ratio and determine additional data about object parameters, the local processor 231 and / or the processing unit 108 may determine data about the collected light based on the sub-pixels of the cluster and ignore output data of sub-pixels outside the cluster.
[0163] This configuration allows the present technology to further provide sub-pixel accuracy and improve the detection of object parameters such as size, centroid position and / or reflectivity. Therefore, if an object is detected by a certain number of sub-pixels of one or more sub-pixels, the object parameters such as reflectivity, centroid position and object size can be corrected relative to the output data collected from the entire surface of at least one pixel. For example, detecting a small object that only reflects a spatial portion of the illumination beam can cause the reflected light to only be incident on the bottom portion of at least one pixel 202, for example, Figure 6C 208. Detection of the collected light using a readout from the entire pixel can indicate a larger target with low reflectivity. Identifying the collected light signal based on spatial clusters of sub-pixels that provide output data indicative of light incident thereon enables the boundaries of the target to be determined based on the relative reflectivity of each sub-pixel, making both target position and reflectivity measurements more accurate.
[0164] In this context, the interface 203 may receive up to Nsp signals indicating output signals output from Nsp subpixels, and send the plurality of output signals to a readout circuit 230, which in turn may provide one, some or all of the up to Nsp subpixel output signals to a local processor 231. The local processor may determine an output signal formed by the output signals of the selected number of subpixels. To this end, the local processor 231 may process data about the spatial distribution of subpixels providing non-zero output signals (indicating light incident thereon), and determine a spatial cluster of nearby subpixels providing output signals indicating light incident thereon. If such a cluster is not identified, the local processor 231 may generate output data indicating the absence of a signal. This is because noise is typically uniform and may result in photons incident on a subpixel without generating a spatial cluster. In the case of determining a spatial cluster, the output data may indicate the total light collected by the cluster, typically ignoring subpixels that are not part of the cluster. In addition, the output data may include data about the spatial arrangement of the cluster, thereby allowing the processing unit 108 to determine additional parameters about the corresponding object.
[0165] Thus, the local processor 231 may determine which sub-pixel outputs to read. In some embodiments, the local processor 231 may communicate to the interface 203 which sub-pixels are to be active, and may deactivate one or more irrelevant sub-pixels.
[0166] Fig.12 An additional exemplary configuration of components 222 of an optical module is shown, which includes pixels 202, an interface 203, a readout circuit and local adder 232, and a communication unit 234. The communication unit 234 communicates with another computerized unit 240.
[0167] This configuration can be based on the reference above Figures 9 to 11The interface 203 may be operated in any of the embodiments described, while utilizing another computerized unit 240 on a local processor. The interface 203 may receive up to Nsp sub-pixel output signals and send them to a readout circuit and local adder 232, which in turn may provide one or more sums of some or all of the up to Nsp sub-pixel output signals to a communication unit 234. The another computerized unit 240 (which may typically be at least one processor 118 or processing unit 108) may determine data regarding alignment metrics by determining data regarding the position and arrangement of sub-pixels that generate output signals indicative of light incident thereon. Thus, the another processing unit 240 may determine one or more IFOV parameters, alignment of the collected light, and / or selection of one or more pixel output signals to include when generating output data regarding light collection. For example, the another processing unit 240 may select which sub-pixel outputs to sum and which sub-pixel outputs to ignore.
[0168] Fig.13A and Fig. 13B Two exemplary operation methods according to some embodiments of the present disclosure are schematically shown. Fig.13A As shown, the method includes directing an illumination beam toward a scene 1310 and collecting reflected light from the scene by at least one pixel 1320. After light collection, the method includes reading output data 1330 from at least one pixel. In some embodiments, the output data may be a single value data indicating the intensity of light incident on the pixel. In some other embodiments, the output data may include two or more output data pieces indicating light incident on at least a first portion and a second portion of at least one pixel. In some other embodiments, the output data may include output data from one or more additional light detectors. In some further embodiments, the output data may include multiple data pieces, each data piece indicating output data of a sub-pixel of at least one pixel.
[0169] The method also includes determining an alignment metric based on the output data 1340. The alignment metric may indicate an overlap between an IFOV determined by a location according to which the output data is determined and an illumination spot formed by the collected light incident on at least one pixel. In this regard, the alignment metric may relate to a translation change, a size change, or a combination thereof between the IFOV and the illumination spot. Using the alignment metric, the method also includes changing one or more IFOV parameters 1350. In this regard, changing the IFOV parameters may include changing the alignment of light collection (or transmission toward a scene), changing the location of a readout region for reading out from at least one pixel, and / or changing the size or shape of a readout region for reading out from at least one pixel. As described above, adjusting the readout IFOV to the illumination point of the collected light can increase the SNR and may also allow determination of additional data about one or more objects in the scene.
[0170] The above disclosure describes several techniques for determining the alignment metric 1340. In some embodiments, the alignment metric can be determined after a selected scanning cycle, and one or more average parameters about the collected data are determined during the scanning cycle. For example, the alignment metric can be determined based on the average SNR of the object used to detect the selected parameter (e.g., a distance between 150m and 250m or between 150m and 300m). In some other examples described above, determining the alignment metric can be based on determining a relationship between readout data collected from at least a first pixel and a second (or more) pixel or pixel region. In some additional embodiments, determining the alignment metric can be based on determining a relationship between readout data collected from one or more additional photodetectors. These configurations enable point-by-point determination of the alignment metric and can simplify optimization by saving scanning time required for a scanning cycle.
[0171] Fig. 13BAn additional method according to the present disclosure is illustrated. In this embodiment, the method includes directing light toward a scene 1310 and collecting reflected light by at least one pixel 1320. Here, at least one pixel is formed by an array of multiple sub-pixels and is configured to provide output data, the output data including data about the light collected by each of the multiple sub-pixels. Therefore, reading the output data from at least one pixel 1330 provides a set of data slices (e.g., a binary data slice indicating whether a photon is incident on a sub-pixel). The method also includes processing the output data and determining an alignment metric 1340, including determining a spatial cluster 1342 of sub-pixels that provide output data indicating the light incident thereon. Using the data about the spatial cluster of sub-pixels, the method also includes changing an IFOV parameter 1350, which in this embodiment may include selecting sub-pixels associated with the determined spatial cluster to form an IFOV region 1352 for generating output data. This configuration enables the removal of pixel regions that do not collect light reflected from a scene, and may further enable the determination of additional object parameters 1360 with improved resolution. For example, the technology enables accurate detection of object reflectivity of small objects. Furthermore, the technique enables detection at sub-pixel resolution, allowing object height or width to be determined with increased resolution.
[0172] Fig.14 LIDAR systems according to some other embodiments of the present disclosure are shown. Fig.14 Involved Figure 1 The invention relates to an optical system for detecting an optical signal of a plurality of optical components, comprising a projection unit 102, a scanning unit 104, a sensing unit 106, and a processing unit 108, and further comprising an alignment unit 107, the alignment unit 107 comprising one or more elements 117, the one or more elements 117 being configured to change the alignment of the collected (or transmitted) light relative to at least one sensor 116 of the sensing unit 106. As described above, the changed alignment of the collected light can be provided by changing the orientation of the light deflector 114 or one or more optical elements used in light transmission or collection. In addition, changing the alignment of the collected light can include generating a position shift of one or more pixels of the at least one sensor 116 to align the IFOV for signal readout relative to the collected light.
[0173] An example of a local technical solution is shown below and is called "sense based on global optimization".
[0174] The input to the method may be a single pixel output signal.
[0175] Assume that this example aims to resolve optical module misalignment.
[0176] As a starting point, the method calculates the average SNR for a given selection of sub-pixels.
[0177] The average SNR may be calculated for different combinations of sub-pixels. Combinations may be obtained by removing some sub-pixels and / or adding sub-pixels. This may include finding a combination that improves the average SNR.
[0178] When a steady state is reached, sub-pixels with good Rx-Tx overlap may be retained and combinations thereof may be selected. Good Rx-Tx overlap is obtained when the receiver receives a large amount (e.g., a large portion) of energy reflected from the transmitted LIDAR spot incident on the object. The Rx-Tx overlap may represent the overlap between the IFOV of the receiver and the IFOV of the transmitter.
[0179] The averaging period of each combination may be long enough (eg, several frames to several minutes) to obtain a sufficient amount of information.
[0180] The method can use information over multiple frames to maximize the SNR, which can mean excluding certain pixels (in certain parts of the FOV, angles) or certain ranges, etc.
[0181] The method may include repeating, for each of a plurality of combinations: (i) using a readout that is a sum of all relevant sub-pixels, (ii) capturing several readings (eg, pixels, frames), and (iii) calculating the SNR.
[0182] A second example may include using multiple pixel output signals simultaneously - and finding the best combination of related pixels. The decision may be made per frame.
[0183] The second example allows for SNR optimization to be performed, which may enable corrections to be performed in angular position and reflectivity estimates.
[0184] Further optimizations can be done at each point in the FOV (but with more computational power).
[0185] For pixels, our approach can start with the maximum IFOV (the reduced IFOV from the first example is potentially used as a first step to reduce sub-pixel count and reduce complexity) and look at range detections with very high thresholds (e.g., false alarm rate much higher than desired but very good probability of detection). Looking at that suspect detection, we can look for the best combination of sub-pixels that will have the best SNR.
[0186] In the example above, the best combination would remove sub-pixels that do not have Rx-Tx overlap (e.g. Fig.11 outer sub-pixels of ) and sub-pixels with low Tx-target overlap (A2, A3, A4).
[0187] The method may include (i) locating a target within a pixel and thereby effectively increasing the system resolution. Additional attributes that may be added are a fill factor and the centroid of activated sub-pixels within the pixel (or other equivalent attributes that will address the location of the object within the pixel), and (ii) a corrected reflectivity estimate based on the fill factor. For partially covered pixels, the reflectivity estimate is always biased towards a lower result because the reported result is effectively multiplied by Tx to the target fill factor. If the fill factor is known, the method may correct the reflectivity estimate. Additionally, if the centroid of activated sub-pixels within the pixel is used to determine the position (as opposed to the centroid of the pixel), the location (i.e., angle and distance) of the target may have improved resolution. This may achieve a resolution greater than the resolution of each pixel, and objects with sub-pixel dimensions may be detected and their size / boundaries determined with sub-pixel accuracy.
[0188] The sensing unit may include a plurality of pixels, each of the plurality of pixels including a plurality of sub-pixels. If the sensing unit includes a plurality of pixels, the gradient between the signal outputs of the pixels may be used to determine whether the sub-pixels should be removed. For example, consider a 2-pixel sensing unit including pixel 1 and pixel 2. Pixel 1 and pixel 2 are of the same size and have the same number of sub-pixels. If a single laser spot reflected from a uniform target (e.g., at a uniform distance with uniform reflectivity, etc.) is incident on pixel 1 and pixel 2, each pixel may output a summed output signal. If the output signal from pixel 1 is different from the output signal from pixel 2, the gradient or ratio between the signals may be calculated. If the output signal from pixel 1 is lower than the output signal from pixel 2, it may be determined that some sub-pixels from pixel 1 may be removed.
[0189] In addition, if the sensing unit includes multiple spaced-apart pixels and multiple laser spots are incident on multiple pixels, a single global set of sub-pixels (i.e., IFOV) can be selected for each pixel based on the maximized SNR of the same subset for all pixels. This may be useful in scenarios where the pixels have shared resources related to signal multiplexing / multiplexing capabilities, for example, column selection, row selection, sub-pixel selection, or sub-pixel selection can be shared among all pixels. In addition, if IFOV (or sub-pixel) misalignment occurs due to misalignment of common optical components (e.g., LIDAR deflectors), the correction of each pixel should be similar if the pixels are aligned with each other. In this case, the determination of the selection of sub-pixels may not be based on the maximum SNR per pixel, but on some defined global optimization (e.g., optimization of the weakest pixel signal, optimization of the average SNR per pixel with constraints on non-uniformity, etc.).
[0190] The dynamic IFOV can be digitally controlled by controlling the "active area" of the pixel (i.e., the sub-pixel selected to contribute to the output signal). The dynamic IFOV can be controlled by optical and / or mechanical manipulation of the projection of the sensor to the scene (e.g., moving the folding mirror, or adjusting the Tx deflector (e.g., scanning mirror) relative to the RX deflector (e.g., scanning mirror) if they are 2 separate scanning mirrors).
[0191] The foregoing description has been presented for purposes of illustration. It is not exhaustive and is not limited to the precise forms or embodiments disclosed. Modifications and adjustments will be apparent to those skilled in the art in view of the specification and practice of the disclosed embodiments. In addition, although various aspects of the disclosed embodiments are described as being stored in a memory, those skilled in the art will appreciate that these aspects may also be stored on other types of computer-readable media, such as auxiliary storage devices, e.g., a hard disk or CD ROM, or other forms of RAM or ROM, USB media, DVD, Blu-ray or other optical drive media.
[0192] Computer programs based on the written description and disclosed methods are within the skill range of experienced developers. Various programs or program modules can be created using any technology known to those skilled in the art, or various programs or program modules can be designed in conjunction with existing software. For example, program parts or program modules can be designed in or by HTML including .Net frameworks, .Net Compact frameworks (and related languages, such as Visual Basic, C, etc.), Java, C++, Objective-C, HTML, HTML / AJAX combinations, XML, or Java applets.
[0193] In addition, although illustrative embodiments have been described herein, those skilled in the art will understand the scope of any and all embodiments with equivalent elements, modifications, omissions, combinations (e.g., across aspects of various embodiments), adaptations and / or changes based on this disclosure. The limitations in the claims will be interpreted broadly based on the language adopted in the claims and are not limited to the examples described in this specification or during the examination of this application. These examples should be interpreted as non-exclusive. In addition, the steps of the disclosed method can be modified in any way, including by reordering steps and / or inserting or deleting steps. Therefore, the description and examples are intended to be considered as illustrative only, and the true scope and spirit are indicated by the full scope of the attached claims and their equivalents.
Claims
1. A LIDAR system comprising at least one light source configured to scan a selected scene, a sensing unit comprising at least one pixel, and a processing unit, the at least one pixel being configured to generate output data indicative of light intensity collected by the at least one pixel; the processing unit being configured and operative to periodically determine data regarding an alignment metric based on the output data received from the sensing unit, and to change at least one of an IFOV parameter, an alignment of collected light reflected from the selected scene, and a readout of the at least one pixel of the sensing unit to improve a signal-to-noise ratio (SNR) of the system.
2. The LIDAR system according to claim 1 further includes a scanning unit, wherein the scanning unit includes at least one light deflector, and the at least one light deflector is positioned to receive the collected light reflected from one or more objects in the selected scene and direct the collected light to the sensing unit.
3. The LIDAR system according to claim 2, further comprising an alignment unit configured to align the path of the collected light to be incident on the IFOV of the at least one pixel, wherein the alignment unit is connected to a scanning unit, the scanning unit comprising at least one light deflector to align the path of the collected light by deflecting the at least one light deflector.
4. The LIDAR system of claim 1 , further comprising an alignment unit configured to align a path of the collected light to impinge on the IFOV of the at least one pixel.
5. The LIDAR system according to claim 4, wherein: The alignment unit is connected to the sensing unit to align a path of the collected light by changing a position of the at least one pixel.
6. The LIDAR system of claim 4, further comprising at least one optical element located in the path of the collected or transmitted light, and wherein, The alignment unit is connected to the at least one optical element to align a path of the collected light by a change in an orientation of the at least one optical element.
7. The LIDAR system of claim 1, wherein: The at least one pixel includes a plurality of sub-pixels and a readout circuit, and is configured to generate output data regarding light intensity collected by a selected number of sub-pixels among the plurality of sub-pixels, and wherein the processing unit generates operational instructions for selecting an arrangement and number of sub-pixels for generating the output data for changing at least one of the IFOV parameters and alignment.
8. The LIDAR system of claim 1, wherein: The periodically determining data regarding the alignment metric includes determining an average SNR over one or more scans of the scene based on a relationship between one or more signals associated with collected light reflected from one or more objects in the scene.
9. The LIDAR system of claim 1, wherein: The at least one pixel comprises at least a first readout region and a second readout region, wherein the at least first readout region and the second readout region provide readout data about light incident on at least the first region and the second region of the at least one pixel, and the periodic determination of data about the alignment metric comprises: determining a relationship between readouts from the at least first readout region and the second readout region.
10. The LIDAR system of claim 9, wherein: The at least one pixel is configured to provide a top region readout and a bottom region readout indicative of light incident on at least one of a top region and a bottom region or a right region and a left region of a region of the at least one pixel.
11. The LIDAR system of claim 1, wherein: The at least one pixel is formed by at least first and second pixel positions for detecting at least first and second portions of the illumination spot, and the periodically determining data regarding the alignment metric comprises determining a relationship between readouts from the at least two first and second pixels.
12. The LIDAR system of claim 1, wherein: The at least one pixel includes one or more additional light detectors, and wherein the sensing unit is aligned to direct the collected light to impinge on the at least one pixel and partially impinge on the one or more additional light detectors, and the periodic determination of data about the alignment metric includes: determining the intensity distribution of the light portion impinging on the one or more additional detectors.
13. The LIDAR system of claim 1, wherein: The at least one pixel comprises an arrangement of multiple sub-pixel sensors, and wherein the sensing unit is configured to provide a readout of the multiple sub-pixel sensors, and the periodic determination of data regarding the alignment metric comprises: processing the readout distribution of the multiple sub-pixels and determining signal data based on the spatial cluster readouts of the sub-pixels indicating data regarding the collected light.
14. The LIDAR system of claim 13, wherein: The processing unit is configured to receive input data indicating individual readouts of the plurality of sub-pixels and to process the input data to determine signal collection; the processing includes determining a sub-pixel spatial cluster readout indicating data about the collected light, and determining one or more parameters of the object based on the arrangement of the sub-pixel spatial clusters.
15. The LIDAR system of claim 14, wherein: The one or more parameters of the object include an object center of mass location, an object size along at least one axis, and an object reflectivity.
16. The LIDAR system of claim 1, wherein: The processing unit is configured and operative to periodically determine data regarding the alignment metric during typical ongoing operation.
17. A method for use in the operation of a LIDAR system, the method comprising: Receiving output data from at least one pixel, the output data indicating light reflected from a selected scene and incident on the at least one pixel, determining data regarding an alignment metric based on the output data from the at least one pixel, and calibrating light collection by the at least one pixel, the calibrating light collection comprising changing at least one of an IFOV parameter, alignment of collected light reflected from the selected scene, and readout of the at least one pixel of the sensing unit to improve a signal-to-noise ratio (SNR) of the system.
18. The method according to claim 17, wherein: The calibrating the light collection includes generating operating instructions for an alignment unit to align a path of the collected light to impinge on an active sensing area of the at least one pixel.
19. The method according to claim 17, wherein: The calibrating the light collection includes generating operating instructions for a scanning unit of the LIDAR to align a path of the collected light by deflecting the at least one light deflector.
20. The method according to claim 17, wherein: The calibrating the light collection includes changing the spatial alignment of the at least one pixel relative to the path of the collected light.
21. The method according to claim 17, wherein: The at least one pixel includes a plurality of sub-pixels and a readout circuit, and is configured to generate output data regarding light intensity collected by a selected number of sub-pixels among the plurality of sub-pixels, and wherein the changing the alignment of the collected light includes: selecting an arrangement of sub-pixels participating in generating the output data.
22. The method according to claim 17, wherein: Determining the alignment metric includes determining an average SNR over one or more scans of the scene based on a relationship between one or more signals associated with collected light reflected from one or more objects in the scene and collected noise.
23. The method according to claim 17, wherein: The at least one pixel includes at least a first readout region and a second readout region, and the at least first readout region and the second readout region provide readout data about light incident on the at least first region and the second region, and the determining of the alignment metric includes: determining the relationship between the readouts from the at least first readout region and the second readout region.
24. The method according to claim 23, wherein: The at least one pixel is configured to provide a top region readout and a bottom region readout indicative of light incident on a top half and a bottom half of an area of the at least one pixel.
25. The method of claim 17, wherein: The at least one pixel is formed by at least first and second pixel positions for detecting at least first and second portions of the illumination spot, and the periodically determining data regarding the alignment metric comprises determining a relationship between readouts from the at least two first and second pixels.
26. The method of claim 17, wherein: The at least one pixel includes one or more additional light detectors and is aligned to direct the collected light to be incident on the at least one pixel and the one or more additional light detectors, and determining the alignment metric includes: determining the intensity distribution of the light portion incident on the one or more additional detectors.
27. The method of claim 17, wherein: The at least one pixel comprises an arrangement of multiple sub-pixel sensors, and wherein the sensing unit is configured to provide a readout of the multiple sub-pixel sensors, and determining the alignment metric comprises: processing the readout distribution of the multiple sub-pixels and determining signal data based on the sub-pixel spatial cluster readouts indicating data about the collected light.
28. The method according to claim 27, wherein: The processing readout distribution includes: receiving input data indicating individual readouts of the plurality of sub-pixels, and processing the input data to determine signal collection by determining a sub-pixel spatial cluster readout indicating data about the collected light and determining one or more parameters of an object based on the arrangement of the sub-pixel spatial clusters.
29. The method according to claim 28, wherein: The one or more parameters of the object include an object center of mass location, an object size along at least one axis, and an object reflectivity.
30. A program storage device readable by a machine, the program storage device tangibly embodying a program of instructions executable by the machine to perform a method for use in the operation of a LIDAR system, the method comprising: Receiving output data from at least one pixel, the output data indicating light incident on the at least one pixel, determining data regarding an alignment metric based on the output data from the at least one pixel, and calibrating light collection by the at least one pixel, the calibrating light collection comprising changing at least one of an IFOV parameter, alignment of collected light reflected from a selected scene, and readout of the at least one pixel of the sensing unit to improve a signal-to-noise ratio (SNR) of the system.
31. A computer program product comprising a computer usable medium having computer readable program code embodied therein for use in the operation of a LIDAR system, the computer program product comprising computer readable instructions for: obtaining output data from at least one pixel; determining data regarding an alignment metric based on the output data; and calibrating light collection of the at least one pixel; in, The collection of calibration light includes at least one of changing IFOV parameters, alignment of the collected light reflected from the selected scene, and readout of the at least one pixel of the sensing unit to improve the signal-to-noise ratio (SNR) of the system.
32. A method for determining an output of a pixel, the method comprising: receiving a pixel output signal during one or more learning periods, wherein the pixel includes a first plurality of sub-pixels; wherein the value of the pixel output signal is based on the value of at least one sub-pixel output signal; as well as A number of pixel output signals to be output from the pixel and a contribution of each of the first plurality of sub-pixels to the pixel output signals are determined based on one or more signal-to-noise ratio (SNR) criteria.
33. The method of claim 32, wherein: The one or more SNR criteria are selected from obtaining a maximum SNR, providing a maximum SNR under certain circumstances, or providing a maximum SNR under certain misalignments.
34. The method of claim 32, wherein: The one or more SNR criteria are associated with at least one of time of day, lighting conditions, date, weather conditions, location, or one or more objects illuminated by the LIDAR system.
35. The method of claim 32, wherein: The one or more learning periods have a combined length of less than one second.
36. The method of claim 32, wherein: The aggregate length of the one or more learning periods is the duration of a single acquisition.
37. The method of claim 32, wherein: The one or more learning periods have a combined length of less than one minute.
38. The method of claim 32, wherein: The combined length of the one or more learning periods is greater than one hour.
39. The method of claim 32, wherein: The pixel output signal includes a plurality of pixel output signals from more than one pixel.
40. The method of claim 39, wherein: Each of the more than one pixels has the same number of sub-pixels.
41. The method of claim 40, wherein: The differences between the plurality of pixel output signals are used to determine the number of pixel output signals to be output.
Citation Information
Cited By
Systems and methods for variable-resolution refinement of Geiger mode lidar
US12710543B2
