System and method for human eye-safe lidar

Abstract

An electro-optical system can include a processor programmed to control a light source to enable variation of luminous flux within a scan of a field of view using light from the light source. The FOV can be divided into a plurality of segments, which can include a first set of non-contiguous segments, and each of the non-contiguous segments included in the first set can be separated from other non-contiguous segments in the first set by at least one segment. The scan of the FOV can include sequentially illuminating the non-contiguous segments, which can be done such that during illumination of a particular non-contiguous segment in the first set of non-contiguous segments, other segments in the plurality of segments are not illuminated, and such that other segments in the plurality of segments are not illuminated between the illumination of the non-contiguous segments in the first set of non-contiguous segments.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 876,198, filed July 19, 2019. The entire contents of the above application are incorporated herein by reference. Background Technology Technical Field

[0004] This disclosure generally relates to lidar (LIDAR) technology.

[0005] Background Information

[0006] With the advent of driver assistance systems and autonomous vehicles, automobiles need to be equipped with systems capable of reliably sensing and interpreting their surroundings, including identifying obstacles, hazards, objects, and other physical parameters that may affect vehicle navigation. To this end, a variety of different technologies have been proposed, including radar operating alone or in a redundant manner, LiDAR, and camera-based systems.

[0007] One consideration for driver assistance systems and autonomous vehicles is the system's ability to determine its surroundings under various conditions, including rain, fog, darkness, bright light, and snow. Light detection and ranging systems (LiDAR, also known as laser radar) are an example of technologies that work well under diverse conditions. This technology measures the distance to an object by illuminating it with light and using sensors to measure the reflected pulses. Lasers are one example of light sources that can be used in LiDAR systems. As with any sensing system, for LiDAR-based sensing systems to be fully adopted by the automotive industry, the system must provide reliable data to enable the detection of distant objects. However, the maximum illumination power of current LiDAR systems is limited by the need to ensure eye safety (i.e., to prevent them from damaging the human eye, which can occur when the projected light is absorbed by the cornea and lens of the eye, causing thermal damage to the retina).

[0008] The systems and methods disclosed herein are aimed at improving the performance of lidar systems while complying with eye safety regulations. Summary of the Invention

[0009] In one embodiment, an electro-optical system may include at least one processor programmed to control at least one light source such that luminous flux can vary within a scan of a field of view using light from the at least one light source. The field of view may be divided into multiple segments. The multiple segments may include a first set of discontinuous segments, and each of the discontinuous segments included in the first set may be separated from other discontinuous segments in the first set by at least one segment. Scanning the field of view may include sequentially illuminating the discontinuous segments included in the first set of discontinuous segments. The sequential illumination of the discontinuous segments included in the first set of discontinuous segments may be performed such that during the illumination of a particular discontinuous segment in the first set of discontinuous segments, other segments of the multiple segments are not illuminated, and that other segments of the multiple segments are not illuminated between the illuminations of the discontinuous segments in the first set of discontinuous segments.

[0010] In one embodiment, a method for controlling an electro-optical system may include controlling at least one light source such that the luminous flux can vary within a scan of a field of view using light from the at least one light source. A plurality of segments may include a first set of discontinuous segments, and each of the discontinuous segments included in the first set may be separated from other discontinuous segments in the first set by at least one segment. Scanning the field of view may include sequentially illuminating the discontinuous segments included in the first set of discontinuous segments. The sequential illumination of the discontinuous segments included in the first set of discontinuous segments may be performed such that during the illumination of a particular discontinuous segment in the first set of discontinuous segments, other segments of the plurality of segments are not illuminated, and that other segments of the plurality of segments are not illuminated between the illuminations of the discontinuous segments in the first set of discontinuous segments.

[0011] In one embodiment, an electro-optical system may include at least one processor programmed to control at least one light source such that luminous flux can vary within a scan of a field of view using light from the at least one light source. The field of view may include a first portion and a second portion different from the first portion. The first portion may include a first sub-section and a second sub-section different from the first sub-section, and the second portion may include a third sub-section and a fourth sub-section different from the third sub-section. Scanning the field of view may include illuminating the first sub-section, second sub-section, third sub-section, and fourth sub-section in the following order: illuminating the first sub-section but not illuminating the second, third, and fourth sub-sections; illuminating the third sub-section but not illuminating the first, second, and fourth sub-sections; illuminating the second sub-section but not illuminating the first, third, and fourth sub-sections; illuminating the fourth sub-section but not illuminating the first, second, and third sub-sections. The illumination level delivered to each of the first, second, third, and fourth sub-sections is below a threshold. The total illumination level delivered to each of the first and second sub-sections exceeds a threshold.

[0012] In one embodiment, an electro-optical system can include at least one processor programmed to control at least one light source to enable a variation in luminous flux within a scan of a field of view using light from the at least one light source. The field of view can include a plurality of non-contiguous segments. Each of the plurality of non-contiguous segments can be non-contiguous and non-overlapping from one another. The scan of the field of view can include illuminating a first segment of the plurality of non-contiguous segments without illuminating any other portion of the field of view, and after illuminating the first segment of the plurality of non-contiguous segments and before illuminating any other portion of the field of view, illuminating a second segment of the plurality of non-contiguous segments without illuminating any other portion of the field of view.

[0013] The foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the claims. BRIEF DESCRIPTION OF DRAWINGS

[0014] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate various disclosed embodiments. In the drawings:

[0015] FIG. 1A is a diagram illustrating an exemplary lidar system consistent with the disclosed embodiments.

[0016] FIG. 1B is an image showing exemplary output of a single scan cycle of a lidar system mounted on a vehicle consistent with the disclosed embodiments.

[0017] FIG. 1C is another image showing a representation of a point cloud model determined from output of a lidar system consistent with the disclosed embodiments.

[0018] FIG. 2A , FIG. 2B , FIG. 2C , FIG. 2D , FIG. 2E , FIG. 2F and FIG. 2G are diagrams illustrating different configurations of a projection unit according to some embodiments of the present disclosure.

[0019] FIG. 3A , FIG. 3B , FIG. 3C and FIG. 3D are diagrams illustrating different configurations of a scanning unit according to some embodiments of the present disclosure.

[0020] FIG. 4A , FIG. 4B , FIG. 4C , FIG. 4D and FIG. 4E are diagrams illustrating different configurations of a sensing unit according to some embodiments of the present disclosure.

[0021] FIG. 5A Four example diagrams are included that illustrate emission patterns for a single frame time for a single portion of the field of view.

[0022] FIG. 5B Three example diagrams are included that illustrate emission schemes for a single frame time for the entire field of view.

[0023] FIG. 5C is a diagram illustrating actual light emission and received reflections of the emission toward a projection during a single frame time for the entire field of view.

[0024] FIG. 6A , FIG. 6B and FIG. 6C is a diagram illustrating a first example implementation consistent with some embodiments of the present disclosure.

[0025] FIG. 6D is a diagram illustrating a second example implementation consistent with some embodiments of the present disclosure.

[0026] FIG. 7 is a diagram illustrating an example lidar system consistent with disclosed embodiments.

[0027] FIG. 8 is a diagram illustrating a portion of an example field of view of a lidar system consistent with disclosed embodiments.

[0028] FIG. 9 is a diagram illustrating a portion of an example field of view of a lidar system consistent with disclosed embodiments.

[0029] FIG. 10 is a flowchart diagram illustrating an example process for detecting objects in an environment of a lidar system consistent with disclosed embodiments.

[0030] FIG. 11 is a flowchart diagram illustrating an example process for detecting objects in an environment of a lidar system consistent with disclosed embodiments.

[0031] FIG. 12 is a flowchart diagram illustrating an example process for detecting objects in an environment of a lidar system consistent with disclosed embodiments. DETAILED DESCRIPTION

[0032] The following detailed description references the drawings, wherein like numerals indicate the same or similar elements and features. As used in the following detailed description and claims, the term “exemplary” shall mean, according to one

[0033] Terminology Definitions

[0034] The disclosed embodiments can relate to optical systems. As used herein, the term “optical system” broadly includes any system for generating, detecting, and / or manipulating light. As examples only, an optical system can include one or more optical components for generating, detecting, and / or manipulating light. For example, a light source, a lens, a mirror, a prism, a beam splitter, a collimator, a polarizing optic, an optical modulator, an optical switch, an optical amplifier, an optical detector, an optical sensor, an optical fiber component, a semiconductor optical component, although not every one is necessary, each of which can be part of an optical system. In addition to one or more optical components, an optical system can include other non-optical components, such as electronic components, mechanical components, chemical reaction components, and semiconductor components. The non-optical components can cooperate with the optical components of the optical system. For example, an optical system can include at least one processor for analyzing detected light.

[0035] Consistent with the present disclosure, the optical system can be a lidar system. As used herein, the term“lidar system” broadly includes any system that can determine a parameter value indicative of a distance between a pair of tangible objects based on reflected light. In one embodiment, the lidar system can determine a distance between a pair of tangible objects based on reflections of light emitted by the lidar system. As used herein, the term“determine a distance” broadly includes generating an output indicative of a distance between a pair of tangible objects. The determined distance can represent a physical dimension between a pair of tangible objects. As an example only, the determined distance can include a line of flight between the lidar system and another tangible object in a field of view of the lidar system. In another embodiment, the lidar system can determine a relative velocity between a pair of tangible objects based on reflections of light emitted by the lidar system. Examples of outputs indicative of a distance between a pair of tangible objects include: a quantity of a standard unit of length (e.g., a number of meters, a number of inches, a number of kilometers, a number of millimeters) between the tangible objects, a quantity of any unit of length (e.g., a number of lengths of the lidar system), a ratio of the distance to another length (e.g., a ratio to a length of an object detected in a field of view of the lidar system), an amount of time (e.g., given in a standard unit, any unit, or a ratio, such as a time taken for light to travel between the tangible objects), one or more positions (e.g., specified using an agreed-upon coordinate system, specified relative to a known position), and the like.

[0036] A lidar system can determine distances between pairs of tangible objects based on reflected light. In one embodiment, a lidar system can process a sensor's detection results that yield time information indicative of a time period between emission of a light signal and a time at which the light signal is detected by the sensor. This time period is sometimes referred to as the "time of flight" of the light signal. In one example, the light signal can be a short pulse, the rise and / or fall times of which can be detected upon receipt. Using known information about the speed of light in the relevant medium (typically air), information about the time of flight of the light signal can be processed to provide a distance traveled by the light signal between emission and detection. In another embodiment, a lidar system can determine distances based on frequency phase shifts (or multi-frequency phase shifts). In particular, a lidar system can process information indicative of one or more modulated phase shifts of a light signal (e.g., by solving a number of simultaneous equations to give a final measurement). For example, a transmitted optical signal can be modulated with one or more constant frequencies. At least one phase shift of the modulation between the transmitted signal and a detected reflection can be indicative of a distance traveled by light between emission and detection. The modulation can be applied to a continuous wave light signal, a quasi-continuous wave light signal, or another type of transmitted light signal. It is noted that a lidar system can use additional information to determine distances, e.g., position information (e.g., relative position) between a location of projection of a signal, a location of detection (especially if far apart from one another), etc.

[0037] In some embodiments, a lidar system can be used to detect a plurality of objects in an environment of the lidar system. The term "detect an object in an environment of the lidar system" broadly includes generating information indicative of an object that reflects light toward a detector associated with the lidar system. If the lidar system detects more than one object, the generated information pertaining to different objects can be interrelated, e.g., a car driving on a road, a bird sitting on a tree, a man touching a bicycle, a truck moving toward a building. The dimensions of the environment in which a lidar system detects objects can vary from implementation to implementation. For example, a lidar system can be used to detect a plurality of objects in an environment of a vehicle on which the lidar system is installed, up to a horizontal distance of 100 m (or 200 m, 300 m, etc.) and up to a vertical distance of 10 m (or 25 m, 50 m, etc.). In another example, a lidar system can be used to detect a plurality of objects in an environment of a vehicle or within a predefined horizontal range (e.g., 25°, 50°, 100°, 180°, etc.) and up to a predefined vertical elevation (e.g., ±10°, ±20°, +40° - 20°, ±90°, or 0° - 90°).

[0038] As used herein, the term“detecting an object” can broadly refer to determining the presence of an object (e.g., the object can be present in a certain direction relative to the lidar system and / or another reference location, or the object can be present in a certain volume of space). Additionally or alternatively, the term“detecting an object” can refer to determining a distance between the object and another location (e.g., a location of the lidar system, a location on the ground, or a location of another object). Additionally or alternatively, the term“detecting an object” can refer to identifying the object (e.g., classifying the type of object, such as a car, a plant, a tree, a road; resolving a specific object (e.g., the Washington Monument); determining a license plate number; determining a composition of the object (e.g., solid, liquid, transparent, translucent); determining kinematic parameters of the object (e.g., whether it is moving, its velocity, its direction of movement, an expansion of the object). Additionally or alternatively, the term“detecting an object” can refer to generating a point cloud map, where each of one or more points of the point cloud map corresponds to a location in the object or on a face thereof. In one embodiment, a data resolution associated with a point cloud map representation of a field of view can be associated with 0.1° x 0.1° or 0.3° x 0.3° of the field of view.

[0039] Consistent with the present disclosure, the term“object” broadly includes a finite composition of matter that can reflect light from at least a portion thereof. For example, an object can be at least partially solid (e.g., a car, a tree); at least partially liquid (e.g., a puddle on a road, rain); at least partially gaseous (e.g., smoke, a cloud); composed of a plurality of distinct particles (e.g., a dust storm, fog, a spray); and can be one or more orders of magnitude in size, such as about 1 millimeter (mm), about 5 mm, about 10 mm, about 50 mm, about 100 mm, about 500 mm, about 1 meter (m), about 5 m, about 10 m, about 50 m, about 100 m, and so on. Smaller or larger objects can also be detected, as well as any size between those examples. It is noted that for various reasons, a lidar system can detect only a portion of an object. For example, in some cases, light can only reflect off of some sides of an object (e.g., only the side facing the lidar system will be detected); in other cases, light can only be projected on a portion of an object (e.g., a laser beam projected onto a road or a building); in other cases, an object can be partially obstructed by another object between the lidar system and the detected object; in other cases, a sensor of the lidar can only detect light reflected from a portion of an object, e.g., because ambient light or other interference interferes with detection of some portions of the object.

[0040] Consistent with the present disclosure, a lidar system can be configured to detect objects by scanning the environment of the lidar system. The term“scanning the environment of the lidar system” broadly includes illuminating a field of view or a portion of a field of view of the lidar system. In one example, scanning the environment of the lidar system can be achieved by moving or pivoting a light deflector to cause light to be deflected in different directions toward different portions of the field of view. In another example, scanning the environment of the lidar system can be achieved by changing the positioning (i.e., location and / or orientation) of the sensor relative to the field of view. In another example, scanning the environment of the lidar system can be achieved by changing the positioning (i.e., location and / or orientation) of the light source relative to the field of view. In yet another example, scanning the environment of the lidar system can be achieved by changing the location of at least one light source and at least one sensor to move rigidly relative to the field of view (i.e., the relative distance and orientation of the at least one sensor to the at least one light source is maintained).

[0041] As used herein, the term“field of view of the lidar system” can broadly include the range of observable environment of the lidar system in which objects can be detected. It is noted that the field of view (FOV) of the lidar system can be influenced by various conditions, such as but not limited to: the orientation of the lidar system (e.g., the direction of the optical axis of the lidar system); the location of the lidar system relative to the environment (e.g., the distance above the ground and adjacent terrain and obstacles); the operational parameters of the lidar system (e.g., the emission power, computational settings, defined operational angles), and the like. The field of view of the lidar system can be defined, for example, by a solid angle (e.g., using the angle, where and 0 is an angle defined in a vertical plane, for example, relative to the axis of symmetry of the lidar system and / or its FOV). In one example, the field of view can also be defined within a certain range (e.g., up to 200 m).

[0042] Similarly, the term“instantaneous field of view” can broadly include the range of the observable environment in which the lidar system can detect objects at any given moment. For example, for a scanning lidar system, the instantaneous field of view is narrower than the overall FOV of the lidar system, and it can be moved within the FOV of the lidar system in order to enable detection in other portions of the FOV of the lidar system. The movement of the instantaneous field of view within the FOV of the lidar system can be achieved by moving the light deflector of the lidar system (or external to the lidar system) in order to deflect the light beam in different directions to and / or from the lidar system. In one embodiment, the lidar system can be configured to scan a scene in the environment in which the lidar system is operating. As used herein, the term“scene” can broadly include some or all of the objects within the field of view of the lidar system, in their relative positions and in their current state, for the duration of the operation of the lidar system. For example, the scene can include ground elements (e.g., ground surface, road, grass, sidewalk, pavement markings), sky, man-made objects (e.g., vehicles, buildings, signs), vegetation, people, animals, light projecting elements (e.g., flashlights, sun, other lidar systems), etc.

[0043] The disclosed embodiments can involve obtaining information used in generating a reconstructed three-dimensional model. Examples of types of reconstructed three-dimensional models that can be used include point cloud models and polygonal meshes (e.g., triangular meshes). The terms“point cloud” and“point cloud model” are well known in the art and should be interpreted to include a collection of data points that are spatially located in some coordinate system (i.e., have an identifiable position in the space described by the respective coordinate system). The term“point cloud point” refers to a point in space (which can be dimensionless, or infinitesimally small, e.g., 1 cm 3), and its position can be described by a set of coordinates (e.g., (X, Y, Z), (r, f, Q)) using a point cloud model. By way of example only, a point cloud model can store additional information for some or all of its points (e.g., color information for points generated from camera images). Likewise, any other type of reconstructed three-dimensional model can store additional information for some or all of its objects. Similarly, the terms “polygon mesh” and “triangle mesh” are well known in the art and should be interpreted to include a collection of vertices, edges, and faces that define the shape of one or more 3D objects, such as polyhedral objects. These faces can include one or more of the following: triangles (triangle meshes), quadrilaterals, or other simple convex polygons, as this can simplify rendering. These faces can also include more general concave polygons or polygons with holes. Polygon meshes can be represented using different techniques, such as: vertex- vertex meshes, face-vertex meshes, winged-edge meshes, and rendered dynamic meshes. Different parts of a polygon mesh (e.g., vertices, faces, edges) or directly and / or relative to each other are spatially located in some coordinate system (i.e., have identifiable positions in the space described by the respective coordinate system). The generation of a reconstructed three-dimensional model can be achieved using any standard, specialized, and / or novel photogrammetry techniques, many of which are known in the art. It is noted that a lidar system can generate other types of environment models.

[0044] Consistent with the disclosed embodiments, a lidar system can include at least one projection unit having a light source configured to project light. As used herein, the term “light source” broadly refers to any device configured to emit light. In one embodiment, the light source can be a laser, such as a solid-state laser, a laser diode, a high-power laser, or an alternative light source, such as a light emitting diode (LED) based light source. Further, as illustrated throughout the figures, the light source 112 can emit light in different formats, such as light pulses, continuous wave (CW), quasi-CW, and the like. For example, one type of light source that can be used is a vertical cavity surface emitting laser (VCSEL). Another type of light source that can be used is an external cavity diode laser (ECDL). In some examples, the light source can include a laser diode configured to emit light having a wavelength between approximately 650 nm and 1150 nm. Alternatively, the light source can include a laser diode configured to emit light having a wavelength between approximately 800 nm and approximately 1000 nm, between approximately 850 nm and approximately 950 nm, or between approximately 1300 nm and approximately 1600 nm. Unless otherwise stated, the term “approximately” with respect to a numerical value is defined as a variation of up to 5% from the stated value. Reference is made to the following figures for additional details regarding the projection unit and the at least one light source. FIGS. 2A-2C Additional details are described regarding the projection unit and the at least one light source.

[0045] Consistent with the disclosed embodiments, a lidar system may include at least one scanning unit having at least one light deflector configured to deflect light from a light source in order to scan a field of view. The term "light deflector" broadly includes any mechanism or module configured to deflect light off its original path; for example, mirrors, prisms, controllable lenses, mechanical mirrors, mechanically scanned polygons, active diffraction (e.g., controllable LCDs), Risley prisms, non-mechanical electric beam steering (such as that manufactured by Vscent), polarization gratings (such as those provided by Boulder Non-Linear Systems), optical phased arrays (OPAs), and so on. In one embodiment, a light deflector may include multiple optical components, such as at least one reflective element (e.g., a mirror), at least one refractive element (e.g., a prism, a lens), and so on. In one example, the light deflector may be movable to deflect light to different degrees (e.g., discrete degrees, or over a continuous span of degrees). The optical deflector can optionally be controlled in different ways (e.g., deflecting to a degree α, changing the deflection angle Δα, moving a component of the optical deflector by M millimeters, changing the speed of the deflection angle change). Furthermore, the optical deflector can optionally be operated to change the deflection angle within a single plane (e.g., the θ coordinate). The optical deflector can optionally be operated to operate in two non-parallel planes (e.g., θ and θ). The deflection angle can be changed within the coordinate system. Alternatively or additionally, the optical deflector may optionally be operable to change the deflection angle between predetermined settings (e.g., along a predefined scan path) or otherwise. Regarding the use of optical deflectors in LiDAR systems, it should be noted that optical deflectors can be used in the outbound direction (also known as the transmission direction or TX) to deflect light from a light source to at least a portion of the field of view. However, optical deflectors can also be used in the inbound direction (also known as the receiving direction or RX) to deflect light from at least a portion of the field of view to one or more optical sensors. See below for reference. FIGS. 3A-3C Additional details are described regarding the scanning unit and at least one optical deflector.

[0046] The disclosed embodiments can involve pivoting a light deflector in order to scan a field of view. As used herein, the term "pivot" broadly includes rotation of an object (especially a solid object) about one or more axes of rotation while substantially maintaining the center of rotation fixed. In one embodiment, pivoting of a light deflector can include rotation of the light deflector about a fixed axis (e.g., a shaft), but need not necessarily be so. For example, in some MEMS mirror implementations, a MEMS mirror can be moved by actuating a plurality of benders connected to the mirror, which can experience some spatial translation in addition to rotation. Such a mirror can nevertheless be designed to rotate about a substantially fixed axis, and thus is considered to be pivoted in accordance with the present disclosure. In other embodiments, some types of light deflector (e.g., non-mechanical electro-optical beam steering, OPA) do not require any moving parts or internal movement in order to change the deflection angle of the deflected light. It is noted that any discussion regarding moving or pivoting a light deflector also applies, mutatis mutandis, to controlling a light deflector such that it changes the deflection behavior of the light deflector. For example, controlling a light deflector can cause a change in the deflection angle of a light beam arriving from at least one direction.

[0047] The disclosed embodiments can involve receiving reflections associated with a portion of a field of view corresponding to a single instantaneous position of a light deflector. As used herein, the term“instantaneous position of a light deflector” (also referred to as“state of a light deflector”) broadly refers to a place or position in space that at least one controlled component of a light deflector is located in at an instantaneous point in time or over a short span of time. In one embodiment, the instantaneous position of a light deflector can be measured relative to a frame of reference. The frame of reference can relate to at least one fixed point in the lidar system. Alternatively, for example, the frame of reference can relate to at least one fixed point in the scene. In some embodiments, the instantaneous position of a light deflector can include some movement, typically to within a limited degree of the maximum degree of change during a scan relative to the field of view, of one or more components of the light deflector (e.g., mirrors, prisms). For example, a scan of an entire field of view of a lidar system can include changing the deflection of light over a span of 30°, and the instantaneous position of at least one light deflector can include an angular shift of the light deflector within 0.05°. In other embodiments, the term“instantaneous position of a light deflector” can refer to the position of the light deflector during acquisition of light that is processed to provide data for a single point of a point cloud (or another type of 3D model) generated by the lidar system. In some embodiments, the instantaneous position of a light deflector can correspond to a fixed position or orientation in which the deflector is paused for a brief period of time during illumination of a particular sub-region of the lidar field of view. In other cases, the instantaneous position of a light deflector can correspond to some position / orientation along a scanned range of positions / orientations of the light deflector through which the light deflector passes as part of a continuous or semi-continuous scan of the lidar field of view. In some embodiments, a light deflector can be moved such that the light deflector is in a plurality of different instantaneous positions during a scan cycle of a lidar FOV. In other words, the deflector can be moved through a series of different instantaneous positions / orientations during the time period over which a scan cycle occurs, and the deflector can arrive at each different instantaneous position / orientation at different times during the scan cycle.

[0048] Consistent with the disclosed embodiments, a lidar system can include at least one sensing unit having at least one sensor configured to detect reflections from objects in a field of view. The term "sensor" broadly includes any device, element, or system capable of measuring a characteristic of an electromagnetic wave (e.g., power, frequency, phase, pulse timing, pulse duration) and generating an output related to the measured characteristic. In some embodiments, the at least one sensor can include a plurality of detectors made up of a plurality of detection elements. The at least one sensor can include one or more types of photosensors. It is noted that the at least one sensor can include multiple sensors of the same type, which can differ in other characteristics (e.g., sensitivity, size). Other types of sensors can also be used. A combination of several types of sensors can be used for different reasons, such as to improve detection over a range span (especially in the near range); to improve the dynamic range of the sensor; to improve the time response of the sensor; and to improve detection under varying environmental conditions (e.g., atmospheric temperature, rain, etc.). In one embodiment, the at least one sensor includes a SiPM (silicon photomultiplier), which is a solid-state single-photon sensitive device built from an array of avalanche photodiodes (APDs), single-photon avalanche diodes (SPADs), used as detection elements on a common silicon substrate. In one example, the typical distance between SPADs can be between about 10 pm and about 50 pm, with each SPAD can have a recovery time between about 20 ns and about 100 ns. Similar photomultipliers from other non-silicon materials can also be used. While SiPM devices work in a digital / switching mode, SiPM is an analog device because all microcells can be read in parallel, enabling it to generate signals in a dynamic range from a single photon to thousands of photons detected by different SPADs. It is noted that the outputs from different types of sensors (e.g., SPAD, APD, SiPM, PIN diode, photodetector) can be combined together into a single output that can be processed by a processor of the lidar system. Reference is made below to FIGS. 4A-4C Additional details are described regarding the sensing unit and the at least one sensor.

[0049] Consistent with the disclosed embodiments, a lidar system can include at least one processor configured to, or in communication with, perform different functions. The at least one processor can constitute any physical device having circuitry that performs logical operations on one or more inputs. For example, the at least one processor can include one or more integrated circuits (ICs), including application-specific integrated circuits (ASICs), microchips, microcontrollers, microprocessors, all or part of a central processing unit (CPU), graphics processing units (GPUs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), or other circuitry adapted to execute instructions or perform logical operations. The instructions executed by the at least one processor can be, for example, preloaded into a memory integrated with or embedded in the controller, or can be stored in a separate memory. The memory can include random access memory (RAM), read-only memory (ROM), hard disks, optical disks, magnetic media, flash memory, other permanent, fixed, or volatile memory, or any other mechanism capable of storing instructions. In some embodiments, the memory is configured to store information representative of data about objects in an environment of the lidar system. In some embodiments, the at least one processor can include more than one processor. Each processor can have a similar configuration, or the processors can have different configurations that are electrically connected or disconnected from one another. For example, the processors can be separate circuits or integrated in a single circuit. When more than one processor is used, the processors can be configured to operate independently or cooperatively. The processors can be coupled electrically, magnetically, optically, acoustically, mechanically, or by other means that allow them to interact. Additional details regarding processing units and at least one processor are described below with reference to FIGS. 5A-5C Additional details regarding processing units and at least one processor are described below with reference to

[0050] System Overview

[0051] FIG. 1AA lidar system 100 is illustrated that includes a projection unit 102, a scanning unit 104, a sensing unit 106, and a processing unit 108. The lidar system 100 can be mountable on a vehicle 110. Consistent with embodiments of the present disclosure, the projection unit 102 can include at least one light source 112, the scanning unit 104 can include at least one light deflector 114, the sensing unit 106 can include at least one sensor 116, and the processing unit 108 can include at least one processor 118. In one embodiment, the at least one processor 118 can be configured to coordinate operation of the at least one light source 112 with movement of the at least one light deflector 114 in order to scan a field of view 120. During a scan cycle, each instantaneous position of the at least one light deflector 114 can be associated with a particular portion 122 of the field of view 120. Further, the lidar system 100 can include at least one optional optical window 124 for directing light projected toward the field of view 120 and / or receiving light reflected from objects in the field of view 120. The optional optical window 124 can serve different purposes, such as collimation of the projected light and focusing of the reflected light. In one embodiment, the optional optical window 124 can be an opening, a flat window, a lens, or any other type of optical window.

[0052] Consistent with the present disclosure, the lidar system 100 can be used in autonomous or semi-autonomous road vehicles (e.g., cars, buses, trucks, vans, and any other land vehicles). Autonomous road vehicles having the lidar system 100 can scan their environment and drive to a destination without human input. Similarly, the lidar system 100 can also be used in autonomous / semi-autonomous aerial vehicles (e.g., UAVs, drones, quadcopters, and any other aerial vehicles or devices); or autonomous or semi-autonomous watercraft (e.g., boats, ships, submarines, or any other watercraft). Autonomous aerial and watercraft vehicles having the lidar system 100 can scan their environment and navigate to a destination autonomously or using a remote operator. According to one embodiment, the vehicle 110 (road vehicle, aerial vehicle, or watercraft) can use the lidar system 100 to help detect and scan the environment in which the vehicle 110 is operating.

[0053] It should be noted that the lidar system 100 or any of its components can be used with any of the example embodiments and methods disclosed herein. Further, although some aspects of the lidar system 100 are described with respect to an example vehicle-based lidar platform, the lidar system 100, any of its components, or any of the processes described herein can be applicable to other platform types of lidar systems.

[0054] In some embodiments, the lidar system 100 can include one or more scanning units 104 to scan the environment around the vehicle 110. The lidar system 100 can be attached or mounted to any portion of the vehicle 110. The sensing unit 106 can receive reflections from the environment surrounding the vehicle 110 and transmit a reflected signal indicative of light reflected from objects in the field of view 120 to the processing unit 108. Consistent with the present disclosure, the scanning unit 104 can be mounted to or incorporated into a bumper, fender, side panel, spoiler, roof, headlamp assembly, tail lamp assembly, rearview mirror assembly, hood, trunk, or any other suitable portion of the vehicle 110 capable of housing at least a portion of the lidar system. In some cases, the lidar system 100 can capture a complete surround view of the environment of the vehicle 110. Thus, the lidar system 100 can have a 360-degree horizontal field of view. In one example, as shown in FIG. 1, the lidar system 100 can include a single scanning unit 104 mounted on the roof of the vehicle 110. Alternatively, the lidar system 100 can include multiple scanning units (e.g., two, three, four, or more scanning units 104), each having a field of view such that the overall horizontal field of view is covered by a 360-degree scan around the vehicle 110. Those skilled in the art will recognize that the lidar system 100 can include any number of scanning units 104 arranged in any manner, each having a field of view of 80° to 120° or less, depending on the number of units employed. Moreover, a 360-degree horizontal field of view can also be obtained by mounting multiple lidar systems 100 on the vehicle 110, each having a single scanning unit 104. Note, however, that the one or more lidar systems 100 need not provide a complete 360° field of view, and narrower fields of view can be useful in some cases. For example, the vehicle 110 can require a first lidar system 100 having a 75° field of view looking forward of the vehicle, and possibly a second lidar system 100 having a similar FOV (optionally with a lower detection range) looking rearward. Note also that different vertical field of view angles can also be implemented. FIG. 1A In some embodiments, the lidar system 100 can include one or more scanning units 104 to scan the environment around the vehicle 110. The lidar system 100 can be attached or mounted to any portion of the vehicle 110. The sensing unit 106 can receive reflections from the environment surrounding the vehicle 110 and transmit a reflected signal indicative of light reflected from objects in the field of view 120 to the processing unit 108. Consistent with the present disclosure, the scanning unit 104 can be mounted to or incorporated into a bumper, fender, side panel, spoiler, roof, headlamp assembly, tail lamp assembly, rearview mirror assembly, hood, trunk, or any other suitable portion of the vehicle 110 capable of housing at least a portion of the lidar system. In some cases, the lidar system 100 can capture a complete surround view of the environment of the vehicle 110. Thus, the lidar system 100 can have a 360-degree horizontal field of view. In one example, as shown in FIG. 1, the lidar system 100 can include a single scanning unit 104 mounted on the roof of the vehicle 110. Alternatively, the lidar system 100 can include multiple scanning units (e.g., two, three, four, or more scanning units 104), each having a field of view such that the overall horizontal field of view is covered by a 360-degree scan around the vehicle 110. Those skilled in the art will recognize that the lidar system 100 can include any number of scanning units 104 arranged in any manner, each having a field of view of 80° to 120° or less, depending on the number of units employed. Moreover, a 360-degree horizontal field of view can also be obtained by mounting multiple lidar systems 100 on the vehicle 110, each having a single scanning unit 104. Note, however, that the one or more lidar systems 100 need not provide a complete 360° field of view, and narrower fields of view can be useful in some cases. For example, the vehicle 110 can require a first lidar system 100 having a 75° field of view looking forward of the vehicle, and possibly a second lidar system 100 having a similar FOV (optionally with a lower detection range) looking rearward. Note also that different vertical field of view angles can also be implemented.

[0055] FIG. 1Bis an image showing exemplary output from a single scan cycle of a lidar system 100 mounted on a vehicle 110, consistent with the disclosed embodiments. In this example, the scanning unit 104 is incorporated into the right headlamp assembly of the vehicle 110. Each gray dot in the image corresponds to a location in the environment surrounding the vehicle 110 determined from reflections detected by the sensing unit 106. In addition to the location, each gray dot can also be associated with different types of information, e.g., intensity (e.g., how much light returned from that location), reflectivity, proximity to other points, and so on. In one embodiment, the lidar system 100 can generate a plurality of point cloud data entries from detected reflections of a plurality of scan cycles of the field of view to enable, for example, determination of a point cloud model of the environment surrounding the vehicle 110.

[0056] FIG. 1C is an image showing a representation of a point cloud model determined from output of the lidar system 100. Consistent with the disclosed embodiments, a surround view image can be generated from the point cloud model by processing the generated point cloud data entries of the environment surrounding the vehicle 110. In one embodiment, the point cloud model can be provided to a feature extraction module that processes the point cloud information to identify a plurality of features. Each feature can include data about different aspects of the point cloud and / or objects in the environment surrounding the vehicle 110 (e.g., cars, trees, people, and roads). The features can have the same resolution as the point cloud model (i.e., have the same number of data points, optionally arranged into a 2D array of similar size), or can have different resolutions. The features can be stored in any kind of data structure (e.g., raster, vector, 2D array, ID array). Furthermore, virtual features such as a representation of the vehicle 110, a boundary line, or a bounding box separating regions or objects in the image (e.g., as depicted in FIG. 1B is an image showing a representation of a point cloud model determined from output of the lidar system 100. Consistent with the disclosed embodiments, a surround view image can be generated from the point cloud model by processing the generated point cloud data entries of the environment surrounding the vehicle 110. In one embodiment, the point cloud model can be provided to a feature extraction module that processes the point cloud information to identify a plurality of features. Each feature can include data about different aspects of the point cloud and / or objects in the environment surrounding the vehicle 110 (e.g., cars, trees, people, and roads). The features can have the same resolution as the point cloud model (i.e., have the same number of data points, optionally arranged into a 2D array of similar size), or can have different resolutions. The features can be stored in any kind of data structure (e.g., raster, vector, 2D array, ID array). Furthermore, virtual features such as a representation of the vehicle 110, a boundary line, or a bounding box separating regions or objects in the image (e.g., as depicted in

[0057] Projection Unit

[0058] FIGS. 2A-2G various configurations of the projection unit 102 and their roles in the lidar system 100 are depicted. In particular, FIG. 2A is a schematic diagram illustrating a projection unit 102 with a single light source; FIG. 2B is a schematic diagram illustrating multiple projection units 102 with multiple light sources aimed at a common light deflector 114; FIG. 2C is a schematic diagram illustrating a projection unit 102 with a primary light source and an auxiliary light source 112; FIG. 2Dis a schematic diagram illustrating an asymmetric deflector used in some configurations of the projection unit 102; FIG. 2E is a schematic diagram illustrating a first configuration of a non-scanning lidar system; FIG. 2F is a schematic diagram illustrating a second configuration of a non-scanning lidar system; and FIG. 2G is a schematic diagram of a lidar system that scans in the outbound direction but does not scan in the inbound direction. Those skilled in the art will recognize that the depicted configurations of the projection unit 102 can have many variations and modifications.

[0059] FIG. 2A An example of a bi-static configuration of the lidar system 100 is illustrated, in which the projection unit 102 includes a single light source 112. The term "bi-static configuration" broadly refers to a lidar system configuration in which the outgoing light from the lidar system and the reflected light into the lidar system travel through substantially different optical paths. In some embodiments, the bi-static configuration of the lidar system 100 can include separating the optical paths by using entirely different optical components, by using parallel but not entirely separate optical components, or by using the same optical components for only part of the optical paths (the optical components can include, for example, windows, lenses, mirrors, beam splitters, etc.). In some embodiments, the bi-static configuration of the lidar system 100 can include separating the optical paths by using the same optical components for both the outbound and inbound light, but with different optical paths for the outbound and inbound light. FIG. 2A In the depicted example, the bi-static configuration includes a configuration in which the outbound light and the inbound light travel through a single optical window 124, but the scanning unit 104 includes two light deflectors, a first light deflector 114A for the outbound light and a second light deflector 114B for the inbound light (the inbound light in the lidar system includes the emitted light reflected from objects in the scene, and can also include ambient light arriving from other sources). In the depicted example, the first light deflector 114A is a mirror and the second light deflector 114B is a prism, but those skilled in the art will recognize that many other configurations are possible. FIG. 2E and FIG. 2G In the depicted example, the bi-static configuration includes a configuration in which the outbound light travels through a first optical window 124A and the inbound light travels through a second optical window 124B. In all of the example configurations described above, the inbound and outbound optical paths are different from each other.

[0060] In this embodiment, all components of the lidar system 100 can be contained within a single housing 200, or can be divided among multiple housings. As shown, the projection unit 102 is associated with a single light source 112 that includes a laser diode 202A (or one or more laser diodes coupled together) configured to emit light (projection light 204). In one non-limiting example, the light projected by the light source 112 can be at a wavelength of between about 800 nm and 950 nm, with an average power of between about 50 mW and about 500 mW, with a peak power of between about 50 W and about 200 W, and a pulse width of between about 2 ns and about 100 ns. Additionally, the light source 112 can optionally be associated with an optical assembly 202B for manipulating the light emitted by the laser diode 202A (e.g., for collimation, focusing, etc.). It is noted that other types of light sources 112 can be used, and the present disclosure is not limited to laser diodes. Furthermore, the light source 112 can emit light in different formats, such as light pulses, frequency modulation, continuous wave (CW), quasi-CW, or any other form corresponding to the particular light source employed. The projection format and other parameters can be changed by the light source from time to time based on different factors, such as instructions from the processing unit 108. The projection light is projected toward an out-transmission deflector 114A, which serves as a turning element for directing the projection light in the field of view 120. In this example, the scanning unit 104 also includes a pivotable return deflector 114B that directs photons (reflected light 206) reflected back from an object 208 within the field of view 120 toward the sensor 116. The reflected light is detected by the sensor 116, and information about the object (e.g., distance to the object 212) is determined by the processing unit 108.

[0061] In this figure, the lidar system 100 is connected to a host 210. In accordance with the present disclosure, the term "host" refers to any computing environment that can interface with the lidar system 100, which can be a vehicle system (e.g., part of the vehicle 110), a testing system, a safety system, a surveillance system, a traffic control system, a city modeling system, or any system that monitors its surroundings. Such a computing environment can include at least one processor and / or can be connected to the lidar system 100 via the cloud. In some embodiments, the host 210 can also include an interface to external devices, such as cameras and sensors configured to measure different characteristics of the host 210 (e.g., acceleration, steering wheel deflection, reverse driving, etc.). In accordance with the present disclosure, the lidar system 100 can be fixed to a stationary object associated with the host 210 (e.g., a building, a tripod) or to a portable system associated with the host 210 (e.g., a laptop computer, a movie camera). In accordance with the present disclosure, the lidar system 100 can be connected to the host 210 to provide outputs of the lidar system 100 (e.g., 3D models, reflectivity images) to the host 210. In particular, the host 210 can use the lidar system 100 to help detect and scan the environment of the host 210 or any other environment. Moreover, the host 210 can integrate, synchronize, or otherwise use the outputs of the lidar system 100 with the outputs of other sensing systems (e.g., cameras, microphones, radar systems). In one example, the lidar system 100 can be used by a safety system.

[0062] The lidar system 100 can also include a bus 212 (or other communication mechanism) interconnecting the subsystems and components of the lidar system 100 for communicating information between the subsystems and components. Optionally, the bus 212 (or another communication mechanism) can be used to interconnect the lidar system 100 with the host 210. In FIG. 2A In one example, the processing unit 108 includes two processors 118 to adjust the operation of the projection unit 102, the scanning unit 104, and the sensing unit 106 in a coordinated manner based at least in part on information received from internal feedback of the lidar system 100. In other words, the processing unit 108 can be configured to dynamically operate the lidar system 100 in a closed loop. A closed loop system is characterized by having feedback from at least one element and updating one or more parameters based on the received feedback. Moreover, a closed loop system can receive feedback and update its own operation based at least in part on the feedback. A dynamic system or element is a system or element that can be updated during operation.

[0063] According to some embodiments, scanning the environment around the lidar system 100 can include illuminating the field of view 120 with light pulses. The light pulses can have parameters such as: pulse duration, pulse angular dispersion, wavelength, instantaneous power, photon density at different distances from the light source 112, average power, pulse power intensity, pulse width, pulse repetition rate, pulse sequence, pulse duty cycle, wavelength, phase, polarization, etc. The environment around the lidar system 100 can also include detecting and characterizing various aspects of the reflected light. Characteristics of the reflected light can include, for example: time of flight (i.e., time from emission until detection), instantaneous power (e.g., power signature), average power over the entire returned pulse, and photon distribution / signal over the returned pulse period. By comparing the characteristics of the light pulses with the characteristics of the corresponding reflections, the distance of the object 212 can be estimated, as well as possibly physical characteristics such as reflectivity. By repeating this process over multiple adjacent portions 122 in a predefined pattern (e.g., raster, Lissajous, or other pattern), the entire scan of the field of view 120 can be achieved. As discussed in more detail below, in some cases the lidar system 100 can direct light to only some of the portions 122 in the field of view 120 at each scan cycle. These portions can be adjacent to each other, but not necessarily so.

[0064] In another embodiment, the lidar system 100 can include a network interface 214 for communicating with the host 210 (e.g., a vehicle controller). The communication between the lidar system 100 and the host 210 is represented by the dashed arrow. In one embodiment, the network interface 214 can include an integrated service digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, the network interface 214 can include a local area network (LAN) card to provide a data communication connection to a compatible LAN. In another embodiment, the network interface 214 can include an Ethernet port connecting to a radio frequency receiver and transmitter and / or an optical (e.g., infrared) receiver and transmitter. The specific design and implementation of the network interface 214 depends on the communication network(s) on which the lidar system 100 and the host 210 are to operate. For example, the network interface 214 can be used to provide outputs of the lidar system 100, such as 3D models, operating parameters of the lidar system 100, etc., to an external system. In other embodiments, the communication unit can be used to receive instructions from an external system, receive information about the environment being inspected, receive information from another sensor, etc.

[0065] FIG. 2BAn example of a monostatic configuration of a lidar system 100 including multiple projection units 102 is illustrated. The term "monostatic configuration" broadly refers to a lidar system configuration in which the projection light that exits the lidar system and the reflected light that enters the lidar system travel through substantially similar optical paths. In one example, the outgoing beam and the incoming beam can share at least one optical component through which both the outgoing beam and the incoming beam travel. In another example, the outgoing optical radiation can travel through an optical window (not shown) and the incoming optical radiation can travel through the same optical window. The monostatic configuration can include a configuration in which the scanning unit 104 includes a single optical deflector 114 that directs the projection light toward the field of view 120 and directs the reflected light toward the sensor 116. As shown, both the projection light 204 and the reflected light 206 hit the asymmetric deflector 216. The term "asymmetric deflector" refers to any optical device having two sides that is capable of deflecting a beam of light hitting it from a second side in a different direction than it deflects a beam of light hitting it from a first side. In one example, the asymmetric deflector does not deflect the projection light 204, but deflects the reflected light 206 toward the sensor 116. One example of an asymmetric deflector can include a polarizing beam splitter. In another example, the asymmetric 216 can include an optical isolator that allows light to pass in only one direction. An illustration of an asymmetric deflector 216 is illustrated in FIG. 2D Consistent with the present disclosure, the monostatic configuration of the lidar system 100 can include an asymmetric deflector to prevent the reflected light from hitting the light source 112 and direct all of the reflected light toward the sensor 116, thereby increasing the detection sensitivity.

[0066] In the embodiment of FIG. 2B the lidar system 100 includes three projection units 102, each having a single light source 112 aimed at a common optical deflector 114. In one embodiment, the multiple light sources 112 (including two or more light sources) can project light having substantially the same wavelength, and each light source 112 is generally associated with a different region of the field of view (indicated as 120A, 120B, and 120C in the figure). This enables a wider field of view to be scanned than can be achieved with a single light source 112. In another embodiment, the multiple light sources 112 can project light having different wavelengths, and all of the light sources 112 can be directed to the same portion (or overlapping portions) of the field of view 120.

[0067] FIG. 2CAn example of a lidar system 100 is illustrated in which the projection unit 102 includes a primary light source 112A and an auxiliary light source 112B. The primary light source 112A can project light having a longer wavelength than is sensitive to the human eye in order to optimize SNR and detection range. For example, the primary light source 112A can project light having a wavelength between approximately 750 nm and 1100 nm. In contrast, the auxiliary light source 112B can project light having a wavelength that is visible to the human eye. For example, the auxiliary light source 112B can project light having a wavelength between approximately 400 nm and 700 nm. In one embodiment, the auxiliary light source 112B can project light along substantially the same optical path as the light projected by the primary light source 112A. The two light sources can be time synchronized and can project light emissions together or in an interleaved pattern. An interleaved pattern means that the light sources are not active at the same time, which can mitigate mutual interference. Those skilled in the art will readily see that other combinations of wavelength ranges and activation schedules can also be implemented.

[0068] Consistent with some embodiments, the auxiliary light source 112B can cause the human eye to blink if it is too close to the lidar optical output port. This can ensure an eye safety mechanism that is not feasible with typical laser sources that utilize the near infrared spectrum. In another embodiment, the auxiliary light source 112B can be used for calibration and reliability at a service point in a manner somewhat similar to headlamp calibration with special reflectors / patterns at a height above the ground relative to the vehicle 110. An operator at the service point can check the calibration of the lidar by simple visual inspection of the scan pattern on a feature target, such as a test pattern board at a specified distance from the lidar system 100. In addition, the auxiliary light source 112B can provide a means for operational confidence that the lidar is working for the end user. For example, the system can be configured to allow a person to place a hand in front of the light deflector 114 to test its operation.

[0069] The auxiliary light source 112B can also have an invisible element that can act as a backup system in the event of a failure of the primary light source 112A. This feature is useful for a fail-safe device with a higher functional safety rating. Assuming that the auxiliary light source 112B can be visible, and also for reasons of cost and complexity, the auxiliary light source 112B can be associated with less power than the primary light source 112A. Thus, in the event of a failure of the primary light source 112A, the system functionality will fall back to the functional and capability set of the auxiliary light source 112B. Although the capabilities of the auxiliary light source 112B can be inferior to those of the primary light source 112A, the lidar system 100 system can be designed in a manner that enables the vehicle 110 to safely reach its destination.

[0070] FIG. 2DAn asymmetric deflector 216, which can be part of the lidar system 100, is illustrated. In the illustrated example, the asymmetric deflector 216 includes a reflective surface 218 (such as a mirror) and a one-way deflector 220. Although not necessarily so, the asymmetric deflector 216 can optionally be a transceiver configured deflector. The asymmetric deflector 216 can be used in a transceiver configuration of the lidar system 100 in order to allow a common optical path for transmitting and receiving light via at least one deflector 114, as illustrated in FIG. 2B and FIG. 2C However, a typical asymmetric deflector, such as a beamsplitter, is characterized by energy loss, especially in the receive path, which can be more sensitive to power loss than the transmit path.

[0071] As depicted in FIG. 2D , the lidar system 100 can include an asymmetric deflector 216 in the transmit path that includes a one-way deflector 220 for separating between the transmit light signal and the receive light signal. Optionally, the one-way deflector 220 can be substantially transparent to the transmit light and substantially reflective to the receive light. The transmit light is generated by the projection unit 102 and can travel through the one-way deflector 220 to the scanning unit 104, which deflects it toward the optical exit. The receive light reaches the at least one deflection element 114 through the optical entrance, which deflects the reflected signal into a separate path away from the light source and toward the sensing unit 106. Optionally, the asymmetric deflector 216 can be combined with a polarized light source 112 that is linearly polarized with the same polarization axis as the one-way deflector 220. Notably, the cross-section of the outgoing beam is much smaller than the cross-section of the reflected signal. Thus, the lidar system 100 can include one or more optical components (e.g., lenses, collimators) for focusing or otherwise manipulating the emitted polarized beam to the dimensions of the asymmetric deflector 216. In one embodiment, the one-way deflector 220 can be a polarization beamsplitter that is nearly transparent to the polarized beam.

[0072] Consistent with some embodiments, the lidar system 100 can also include optics 222 (e.g., a quarter-wave plate retarder) for modifying the polarization of the emitted light. For example, the optics 222 can modify the linear polarization of the emitted beam to circular polarization. Light reflected from the field of view back to the system 100 will pass through the deflector 114 back to the optics 222, which experiences circular polarization with an inverted handedness relative to the transmit light. The optics 222 will then convert the received inverted handedness polarized light to linear polarization that is not on the same axis as the linear polarization of the polarization beamsplitter 216. As noted above, the receive light-patch is larger than the transmit light-patch due to the optical dispersion of the beam through the distance to the target.

[0073] Some of the received light will impinge on the one-way deflector 220, which will reflect the light with some power loss toward the sensing unit 106. However, another portion of the received light spot will fall on the reflective surface 218 surrounding the one-way deflector 220 (e.g., the polarizing beamsplitter slit). The reflective surface 218 will reflect the light with substantially zero power loss toward the sensing unit 106. The one-way deflector 220 will reflect the light that will eventually reach the detector, which consists of various polarization axes and directions. Optionally, the sensing unit 106 can include a sensor 116 that is polarization-agnostic and is primarily sensitive to the amount of impinging photons in a certain wavelength range.

[0074] It is noted that the proposed asymmetric deflector 216 provides superior performance when compared to a simple mirror with a through-hole. In the mirror with a hole, all reflected light that reaches the hole is lost to the detector. However, in the deflector 216, the one-way deflector 220 deflects a large portion (e.g., about 50%) of such light toward the corresponding sensor 116. In a lidar system, the number of photons that reach the lidar from a remote distance is very limited, and therefore, improvement in the photon capture rate is important.

[0075] According to some embodiments, an apparatus for beamsplitting and turning is described. A polarized light beam can be emitted from a light source having a first polarization. The emitted light beam can be directed to pass through a polarizing beamsplitter assembly. The polarizing beamsplitter assembly includes a one-way slit on a first side and a mirror on an opposite side. The one-way slit enables the polarized emitted light beam to travel toward a quarter-wave plate / wave retarder, which changes the emitted signal from a polarized signal to a linear signal (or vice versa) so that a subsequently reflected light beam cannot travel through the one-way slit.

[0076] FIG. 2EAn example of a transceiver collocated configuration of the lidar system 100 without the scanning unit 104 is illustrated. To illuminate the entire field of view (or substantially the entire field of view) without the deflector 114, the projection unit 102 can optionally include an array of light sources (e.g., 112A-112F). In one embodiment, the array of light sources can include a linear array of light sources controlled by the processor 118. For example, the processor 118 can cause the linear array of light sources to sequentially project collimated laser beams toward a first optional optical window 124A. The first optional optical window 124A can include a diffuser lens to diffuse the projected light and sequentially form a wide horizontal and narrow vertical beam of light. Optionally, some or all of the at least one light source 112 of the system 100 can project light simultaneously. For example, the processor 118 can cause the array of light sources to project beams of light from multiple non-adjacent light sources 112 simultaneously. In the depicted example, the light source 112A, the light source 112D, and the light source 112F project laser beams toward the first optional optical window 124A simultaneously, thereby illuminating the field of view with three narrow vertical beams of light. The beam of light from the fourth light source 112D can reach an object in the field of view. The light reflected from the object can be captured by the second optical window 124B and can be redirected to the sensor 116. FIG. 2E The depicted configuration is considered a transceiver collocated configuration because the optical paths of the projected light and the reflected light are substantially different. It is noted that the projection unit 102 can also include multiple light sources 112 arranged in a non-linear configuration, such as a two-dimensional array, a hexagonal tiling, or any other manner.

[0077] FIG. 2F An example of a transceiver collocated configuration of the lidar system 100 without the scanning unit 104 is illustrated. The configuration of the lidar system 100 in this figure is similar to FIG. 2E The depicted example embodiment, to illuminate the entire field of view without the deflector 114, the projection unit 102 can include an array of light sources (e.g., 112A-112F). However, unlike FIG. 2E Instead, this configuration of the lidar system 100 can include a single optical window 124 for both the projected light and for the reflected light. Using an asymmetric deflector 216, the reflected light can be redirected to the sensor 116. FIG. 2E The depicted configuration is considered a transceiver collocated configuration because the optical paths of the projected light and the reflected light are substantially similar to each other. In the context of the optical paths of the projected light and the reflected light, the term “substantially similar” means that the overlap between the two optical paths can be greater than 80%, greater than 85%, greater than 90%, or greater than 95%.

[0078] FIG. 2G An example of a transceiver collocated configuration of the lidar system 100 is illustrated. The configuration of the lidar system 100 in this figure is similar to FIG. 2AThe depicted configurations. For example, both configurations include a scanning unit 104 to direct the projected light in the outgoing direction toward the field of view. However, in contrast to embodiments of the FIG. 2A In this configuration, the scanning unit 104 does not redirect the reflected light in the incoming direction. Instead, the reflected light passes through the second optical window 124B and into the sensor 116. FIG. 2G The depicted configurations are considered transceiver configurations because the optical paths of the projected light and the reflected light are substantially different from each other. In the context of the optical paths of the projected light and the reflected light, the term “substantially different” means that the overlap between the two optical paths can be less than 10%, less than 5%, less than 1%, or less than 0.25%.

[0079] Scanning Unit

[0080] FIGS. 3A-3D Various configurations of the scanning unit 104 and their roles in the lidar system 100 are depicted. In particular, FIG. 3A is a diagram illustrating a scanning unit 104 with a MEMS mirror (e.g., shaped as a square), FIG. 3B is a diagram illustrating another scanning unit 104 with a MEMS mirror (e.g., shaped as a circle), FIG. 3C is a diagram illustrating a scanning unit 104 with a reflector array for a transceiver configuration lidar system, and FIG. 3D is a diagram illustrating an example lidar system 100 mechanically scanning the environment around the lidar system 100. Those skilled in the art will recognize that the configurations of the scanning unit 104 depicted are merely exemplary and that many variations and modifications are possible within the scope of the present disclosure.

[0081] FIG. 3AAn example scan unit 104 is illustrated with a single-axis square MEMS mirror 300. In this example, the MEMS mirror 300 is used as at least one deflector 114. As shown, the scan unit 104 can include one or more actuators 302 (specifically, 302A and 302B). In one embodiment, the actuators 302 can be made of a semiconductor (e.g., silicon) and include a piezoelectric layer (e.g., PZT, lead zirconate titanate, aluminum nitride), a semiconductor layer, and a base layer that change their dimensions in response to an electrical signal applied by an actuation controller. In one embodiment, the physical properties of the actuator 302 can determine the mechanical stress experienced by the actuator 302 when current passes through it. When the piezoelectric material is activated, it exerts a force on the actuator 302 and causes it to bend. In one embodiment, the resistivity of the one or more actuators 302 (Ractive) can be measured in the activated state when the mirror 300 is deflected at a certain angular position and compared to the resistivity in the resting state (Rrest). Feedback including Ractive can provide information to determine the actual mirror deflection angle compared to the intended angle and, if needed, correct the mirror 300 deflection. The difference between Rrest and Ractive can be correlated into an angular deflection value by the mirror drive that can be used to close the loop. This embodiment can be used to dynamically track the actual mirror position and can optimize the response, amplitude, deflection efficiency, and frequency of linear mode and resonant mode MEMS mirror schemes. This embodiment is described in more detail below with reference to FIGS. 4A-4C. FIGS. 3A-3D In more detail.

[0082] During scanning, current (represented as a dashed line in the figure) can flow from contact 304A to contact 304B (through actuator 302A, spring 306A, mirror 300, spring 306B, and actuator 302B). Isolation gaps in the semiconductor frame 308, such as isolation gap 310, can make actuators 302A and 302B two separate islands that are electrically connected through springs 306 and frame 308. The current flow or any associated electrical parameters (voltage, current frequency, capacitance, relative permittivity, etc.) can be monitored by associated position feedback. In the case of a mechanical failure, where one of the components is damaged, the current flow through the structure will change and alter its functional calibration values. In extreme cases (e.g., when a spring breaks), the current will stop completely by means of the faulty element due to the circuit break in the electrical chain.

[0083] FIG. 3BAnother example scan unit 104 is illustrated with a biaxial circular MEMS mirror 300. In this example, the MEMS mirror 300 is used as at least one deflector 114. In one embodiment, the MEMS mirror 300 can have a diameter between about 1 mm and about 5 mm. As shown, the scan unit 104 can include four actuators 302 (302A, 302B, 302C, and 302D), each of which can be at a different length. In the illustrated example, current (represented as dashed lines in the figure) flows from contact 304A to contact 304D, but in other cases, current can flow from contact 304A to contact 304B, from contact 304A to contact 304C, from contact 304B to contact 304C, from contact 304B to contact 304D, or from contact 304C to contact 304D. Consistent with some embodiments, the biaxial MEMS mirror can be configured to deflect light in a horizontal direction and a vertical direction. For example, the deflection angle of the biaxial MEMS mirror can be between about 0° and 30° in the vertical direction and between about 0° and 50° in the horizontal direction. Those skilled in the art will recognize that the configuration of the depicted mirror 300 can have many variations and modifications. In one example, at least the deflector 114 can have a biaxial square mirror or a single axis circular mirror. Examples of circular and square mirrors are shown in FIG. 3A and FIG. 3B Depending on system specifications, any shape can be employed. In one embodiment, the actuators 302 can be incorporated as integral parts of at least the deflector 114, such that the motive force to move the MEMS mirror 300 is applied directly to it. Further, the MEMS mirror 300 can be connected to a frame 308 by one or more rigid support elements. In another embodiment, at least the deflector 114 can include an electrostatic or electromagnetic MEMS mirror.

[0084] As described above, a transceiver integrated scanning laser radar system utilizes at least a portion of the same optical path for emitting the projected light 204 and for receiving the reflected light 206. The beam in the outgoing path can be collimated and focused into a narrow beam, while the reflection in the return path is spread into a larger spot due to dispersion. In one embodiment, the scan unit 104 can have a large reflection area in the return path and an asymmetric deflector 216 that redirects the reflection (i.e., the reflected light 206) to the sensor 116. In one embodiment, the scan unit 104 can include a MEMS mirror with a large reflection area and negligible impact on field of view and frame rate performance. Additional details regarding the asymmetric deflector 216 are provided below with reference to FIG. 2D

[0085] In some embodiments (e.g., as FIG. 3C ​As illustrated in the example of FIG. 1, the scanning unit 104 can include a deflector array (e.g., a reflector array) with small light deflectors (e.g., mirrors). In one embodiment, implementing the light deflector 114 as a group of smaller individual light deflectors working in synchrony can allow the light deflector 114 to perform with a high scan rate at large deflection angles. In terms of the effective area, the deflector array can essentially act as a large light deflector (e.g., a large mirror). This deflector array can be operated using a shared steering component configuration, which allows the sensor 116 to collect reflected photons from substantially the same portion of the field of view 120 that is concurrently illuminated by the light source 112. The term "concurrently" means that two selected functions occur during a coincident or overlapping period of time, whether one begins and ends within the duration of the other or the latter begins before the former is completed.

[0086] FIG. 3C An example of the scanning unit 104 is illustrated, in which the reflector array 312 has small mirrors. In this embodiment, the reflector array 312 functions as the at least one deflector 114. The reflector array 312 can include a plurality of reflector units 314 configured to pivot (individually or together) and direct light pulses toward the field of view 120. For example, the reflector array 312 can be part of the outbound path of light projected from the light source 112. In particular, the reflector array 312 can direct the projected light 204 toward a portion of the field of view 120. The reflector array 312 can also be part of the return path for light reflected from the surface of an object located within the illuminated portion of the field of view 120. In particular, the reflector array 312 can direct the reflected light 206 toward the sensor 116 or toward the asymmetric deflector 216. In one example, the reflector array 312 can have an area between about 75 to about 150 mm2, where each reflector unit 314 can have a width of about 10 pm and the support structure can be less than 100 pm. 2

[0087] According to some embodiments, the reflector array 312 can include one or more subgroups of steerable deflectors. Each subgroup of electrically steerable deflectors can include one or more deflector units, such as the reflector units 314. For example, each steerable deflector unit 314 can include at least one of a MEMS mirror, a reflective surface component, and an electromechanical actuator. In one embodiment, each reflector unit 314 can be individually controlled by an individual processor (not shown) such that it can be tilted toward a specific angle along each of one or two separate axes. Alternatively, the reflector array 312 can be associated with a common controller (e.g., the processor 118) configured to synchronously manage the movement of the reflector units 314 such that at least a portion of them will concurrently pivot and point in approximately the same direction.​

[0088] Further, the at least one processor 118 can select at least one reflector unit 314 for the outbound path (hereinafter referred to as “TX mirror”) and a set of reflector units 314 for the return path (hereinafter referred to as “RX mirror”). In keeping with the present disclosure, increasing the number of TX mirrors can increase reflected photon beam spread. Additionally, decreasing the number of RX mirrors can narrow the receive field and compensate for ambient light conditions (such as clouds, rain, fog, extreme heat, and other environmental conditions) and improve signal-to-noise ratio. Also, as indicated above, the transmitted light beams are generally narrower than the reflected spot blocks, and thus can be deflected entirely by a small portion of the deflection array. Also, it is possible to block light reflected from the portion of the deflection array used for transmission (e.g., TX mirror) from reaching the sensor 116, thereby reducing the effects of internal reflections of the lidar system 100 on system operation. Further, the at least one processor 118 can cause one or more reflector units 314 to pivot to overcome mechanical damage and drift due to, for example, heat and gain effects. In an example, one or more reflector units 314 can move differently (frequency, rate, speed, etc.) than expected, and their movement can be compensated for by appropriately electrically controlling the deflector.

[0089] FIG. 3D An example lidar system 100 is illustrated that scans an environment of the mechanical scanning lidar system 100. In this example, the lidar system 100 can include a motor or other mechanism to rotate the housing 200 about an axis of the lidar system 100. Alternatively, the motor (or other mechanism) can mechanically rotate a rigid structure of the lidar system 100 on which the one or more light sources 112 and the one or more sensors 116 are mounted, thereby scanning the environment. As described above, the projection unit 102 can include at least one light source 112 configured to project a light emission. The projected light emission can travel along the outbound path toward the field of view 120. In particular, as the projected light 204 travels toward the optional optical window 124, the projected light emission can be reflected by the deflector 114A through the exit aperture 314. The reflected light emission can travel along the return path from the object 208 toward the sensing unit 106. For example, as the reflected light 206 travels toward the sensing unit 106, the reflected light 206 can be reflected by the deflector 114B. Those skilled in the art will recognize that a lidar system having a rotating mechanism for rotating one or more light sources or one or more sensors in unison can use such unison rotation in place of (or in addition to) turning the internal light deflector.

[0090] In embodiments where the scanning of the field of view 120 is mechanical, the projected light emission can be directed to an exit aperture 314 that is part of a wall 316 separating the projection unit 102 from other parts of the lidar system 100. In some examples, the wall 316 can be formed of a transparent material (e.g., glass) coated with a reflective material to form a deflector 114B. In this example, the exit aperture 314 can correspond to a portion of the wall 316 that is not coated with the reflective material. Additionally or alternatively, the exit aperture 314 can comprise a hole or cutout in the wall 316. The reflected light 206 can be reflected by the deflector 114B and directed toward an entrance aperture 318 of the sensing unit 106. In some examples, the entrance aperture 318 can comprise a filter window configured to allow wavelengths within a certain wavelength range to enter the sensing unit 106 and attenuate other wavelengths. The reflection from the object 208 in the field of view 120 can be reflected by the deflector 114B and hit the sensor 116. By comparing several characteristics of the reflected light 206 and the projected light 204, at least one aspect of the object 208 can be determined. For example, by comparing the time at which the light source 112 emits the projected light 204 and the time at which the sensor 116 receives the reflected light 206, a distance between the object 208 and the lidar system 100 can be determined. In some examples, other aspects of the object 208 (such as shape, color, material, etc.) can also be determined.

[0091] In some examples, the lidar system 100 (or a portion thereof, including at least one light source 112 and at least one sensor 116) can be rotated about at least one axis to determine a three-dimensional map of the surrounding environment of the lidar system 100. For example, the lidar system 100 can be rotated about a substantially vertical axis (as illustrated by arrow 320) in order to scan the field of view 120. Although FIG. 3D Although the lidar system 100 is illustrated as rotating clockwise about the axis (as illustrated by arrow 320), additionally or alternatively, the lidar system 100 can be rotated in a counterclockwise direction. In some examples, the lidar system 100 can be rotated 360 degrees about the vertical axis. In other examples, the lidar system 100 can be rotated back and forth along a region smaller than 360 degrees of the lidar system 100. For example, the lidar system 100 can be mounted on a platform that swings back and forth about an axis without making a complete rotation.

[0092] Sensing Unit

[0093] FIGS. 4A-4E Various configurations of the sensing unit 106 are depicted and their roles in the lidar system 100. In particular, FIG. 4A is a diagram illustrating an example sensing unit 106 with a detector array, FIG. 4Bis a diagram illustrating a transceiver hybrid scan using a two-dimensional sensor, FIG. 4C is a diagram illustrating an example of a two-dimensional sensor 116, FIG. 4D is a diagram illustrating a lens array associated with the sensor 116, and FIG. 4E includes three diagrams illustrating lens structures. Those skilled in the art will recognize that the configuration of the depicted sensing unit 106 is merely exemplary and can have many alternative variations and modifications consistent with the principles of the present disclosure.

[0094] FIG. 4A An example of a sensing unit 106 having a detector array 400 is illustrated. In this example, the at least one sensor 116 includes the detector array 400. The lidar system 100 is configured to detect objects (e.g., bicycle 208A and cloud 208) in the field of view 120 located at different distances (which can be several meters or more) from the lidar system 100. The objects 208 can be solid objects (e.g., road, tree, car, person), liquid objects (e.g., fog, water, atmospheric particles), or another type of object (e.g., dust or powdered illuminated objects). When the photons emitted from the light source 112 hit the objects 208, they either reflect, refract, or are absorbed. Typically, as shown, only a portion of the photons reflected from the objects 208 enter the optional optical window 124. Because a 1 ns difference in travel time results from a distance change of every about 15 cm (because the photons travel to and from the objects 208 at the speed of light), the time difference between the travel times of different photons hitting different objects can be detectable by a time-of-flight sensor with a fast enough response.

[0095] The sensor 116 includes a plurality of detection elements 402 for detecting photons of the photon pulse reflected back from the field of view 120. The detection elements can all be included in a detector array 400, which can have a rectangular arrangement (e.g., as shown) or any other arrangement. The detection elements 402 can operate concurrently or partially concurrently with each other. In particular, each detection element 402 can emit detection information for each sampling duration (e.g., every 1 nanosecond). In one example, the detector array 400 can be a SiPM (silicon photomultiplier), which is a solid-state single-photon sensitive device built on a common silicon substrate from an array of single-photon avalanche diodes (SPADs, used as detection elements 402). Similar photomultiplier tubes from other non-silicon materials can also be used. While SiPM devices work in a digital / switching mode, SiPMs are analog devices because all the microcells are read in parallel, enabling the generation of signals in a dynamic range from a single photon to thousands of photons detected by different SPADs. As mentioned above, more than one type of sensor (e.g., SiPM and APD) can be implemented. Potentially, the sensing unit 106 can include at least one APD integrated into a SiPM array and / or at least one APD detector located next to a SiPM on a separate or common silicon substrate.

[0096] In one embodiment, the detection elements 402 can be grouped into a plurality of regions 404. These regions are geometric locations or environments within the sensor 116 (e.g., within the detector array 400) and can be shaped into different shapes (e.g., rectangles, squares, rings, etc., as shown, or any other shape). While not all individual detectors included within the geometric bounds of a region 404 necessarily belong to that region, in most cases they will not belong to other regions 404 that cover other bounds of the sensor 310, unless some overlap is desired in the seams between regions. As FIG. 4A As illustrated in FIG. 4, the regions can be non-overlapping regions 404, but alternatively they can overlap. Each region can be associated with a region output circuit 406 associated with that region. The region output circuit 406 can provide a region output signal for the corresponding set of detection elements 402. For example, the region output circuit 406 can be a summing circuit, but other forms that combine the outputs of individual detectors into a unit output (whether scalar, vector, or any other format) can be employed. Optionally, each region 404 is a single SiPM, but need not be, and the regions can be sub-portions of a single SiPM, groups of several SiPMs, or even combinations of different types of detectors.

[0097] In the illustrated example, the processing unit 108 is located in a separate housing 200B of the host unit 210 (internal or external) (e.g., within vehicle 110), and the sensing unit 106 may include a dedicated processor 408 for analyzing reflected light. Alternatively, the processing unit 108 may be used to analyze reflected light 206. It should be noted that the lidar system 100 may be implemented with multiple housings in ways other than the illustrated example. For example, the light deflector 114 may be located in a different housing than the projection unit 102 and / or the sensing module 106. In one embodiment, the lidar system 100 may include multiple housings connected to each other in different ways, such as: wired connections, wireless connections (e.g., RF connections), fiber optic cables, and any combination thereof.

[0098] In one embodiment, analyzing the reflected light 206 may include determining the time of flight of the reflected light 206 based on the output of individual detectors in different regions. Optionally, the processor 408 may be configured to determine the time of flight of the reflected light 206 based on multiple regions of the output signal. In addition to the time of flight, the processing unit 108 may also analyze the reflected light 206 to determine the average power over the entire return pulse and may determine the photon distribution / signal (“pulse shape”) during the return pulse period. In the illustrated example, the output of any detection element 402 may not be directly transmitted to the processor 408, but may be combined (e.g., summed) with the signals from other detectors in region 404 before being transmitted to the processor 408. However, this is merely an example and the circuitry of sensor 116 may transmit information from detection element 402 to processor 408 via other routes (not via region output circuitry 406).

[0099] FIG. 4B This diagram illustrates a lidar system 100 configured to scan its environment using a two-dimensional sensor 116. FIG. 4B In the example, sensor 116 is a matrix of 4x6 detectors 410 (also referred to as “pixels”). In one embodiment, the pixel size can be approximately 1×1 mm. Sensor 116 is two-dimensional in that it has more than one set (e.g., rows, columns) of detectors 410 along two non-parallel axes (e.g., orthogonal axes, as illustrated in the example). The number of detectors 410 in sensor 116 can vary between different implementations, for example, depending on the desired resolution, signal-to-noise ratio (SNR), desired detection distance, etc. For example, sensor 116 can have any value between 5 and 5000 pixels. In another example (not shown in the figure), sensor 116 can also be a one-dimensional matrix (e.g., 1x8 pixels).

[0100] It is noted that each detector 410 can include multiple detection elements 402, such as avalanche photodiodes (APDs), single-photon avalanche diodes (SPADs), a combination of avalanche photodiodes (APDs) and single-photon avalanche diodes (SPADs), or a detection element that measures both the time-of-flight from a laser pulse transmission event to a reception event and the intensity of the received photons. For example, each detector 410 can include any value between 20 and 5000 SPADs. The outputs of the detection elements 402 in each detector 410 can be summed, averaged, or otherwise combined to provide a unified pixel output.

[0101] In the illustrated example, the sensing unit 106 can include a two-dimensional sensor 116 (or multiple two-dimensional sensors 116) with a field of view that is smaller than the field of view 120 of the lidar system 100. In this discussion, the field of view 120 (which can be the entire field of view scanned by the lidar system 100 without any movement, rotation, or roll of the lidar system 100) is denoted as a “first FOV 412,” while the smaller FOV of the sensor 116 is denoted as a “second FOV 412” (interchangeably referred to as an “instantaneous field of view”). Depending on the specific use of the lidar system 100, the coverage area of the second FOV 414 relative to the first FOV 412 can vary and can be, for example, between 0.5% and 50%. In one example, the second FOV 412 can be elongated in the vertical dimension by between about 0.05° and 1°. Even if the lidar system 100 includes more than one two-dimensional sensor 116, the combined field of view of the array of sensors can still be smaller than the first FOV 412, for example, by at least 5 times, at least 10 times, at least 20 times, or at least 50 times.

[0102] To cover the first FOV 412, the scanning unit 106 can direct photons arriving from different parts of the environment to the sensor 116 at different times. In the illustrated transceiver configuration, the scanning unit 106 can direct reflected light 206 to the sensor 116 along with the projected light 204 toward the field of view 120 and when the at least one light deflector 114 is in an instantaneous position. In general, at each instant during the scan of the first FOV 412, the beam emitted by the lidar system 100 covers a portion of the environment (in angular opening) that is larger than the second FOV 414 and includes the portion of the environment from which the scanning unit 104 and the sensor 116 collect light.

[0103] FIG. 4Cis a diagram illustrating an example of a two-dimensional sensor 116. In this embodiment, the sensor 116 is a matrix of 8X5 detectors 410, and each detector 410 includes a plurality of detection elements 402. In one example, detector 410A, located in the second row (denoted as “R2”) and third column (denoted as “C3”) of the sensor 116, includes a matrix of 4X3 detection elements 402. In another example, detector 410B, located in the fourth row (denoted as “R4”) and sixth column (denoted as “C6”) of the sensor 116, includes a matrix of 3X3 detection elements 402. Thus, the number of detection elements 402 in each detector 410 can be constant, or can vary, and different detectors 410 in the common array can have different numbers of detection elements 402. The outputs of all detection elements 402 in each detector 410 can be summed, averaged, or otherwise combined, to provide a single pixel output value. It is noted that although the detectors 410 in the example are arranged in a rectangular matrix (straight rows and columns), other arrangements can be used, for example, a circular arrangement or a honeycomb arrangement. FIG. 4C

[0104] According to some embodiments, the measurements from each detector 410 can enable determination of a time of flight from a light pulse emission event to a reception event and an intensity of received photons. The reception event can be a result of reflection of the light pulse from an object 208. The time of flight can be a timestamp value representing a distance of the reflecting object to the optional optical window 124. The time of flight value can be implemented by a photon detection and counting method, such as time-correlated single photon counting (TCSPC), an analog method for photon detection, such as signal integration and qualification (via an analog-to-digital converter or a plain comparator), or other methods.

[0105] In some embodiments and with reference to FIG. 4B ​During a scan cycle, each instantaneous position of at least one optical deflector 114 can be associated with a specific portion 122 of the field of view 120. The sensor 116 is designed to allow correlation between reflected light from a single portion of the field of view 120 and multiple detectors 410. Therefore, the scan resolution of the lidar system can be represented by multiplying the number of instantaneous positions (per scan cycle) by the number of detectors 410 in the sensor 116. Information from each detector 410 (i.e., each pixel) represents the basic data elements from which the captured field of view in three-dimensional space is constructed. This can include, for example, basic elements of a point cloud representation with spatial location and associated reflection intensity values. In one embodiment, reflections from a single portion of the field of view 120 detected by multiple detectors 410 can be returned from different objects located within that single portion of the field of view 120. For example, a single portion of the field of view 120 can be larger than 50x50 cm in the far field, which can easily include two, three, or more objects that partially overlap each other.

[0106] FIG. 4D This is a cross-sectional view of a portion of a sensor 116 according to an example of the currently disclosed subject matter. The illustrated portion of sensor 116 includes a portion of a detector array 400 comprising four detection elements 402 (e.g., four SPADs, four APDs). Detector array 400 may be a photodetector sensor implemented in complementary metal-oxide-semiconductor (CMOS). Each detection element 402 has a sensitive region located within the substrate environment. Although not necessarily, sensor 116 can be used in a transceiver co-location lidar system with a narrow field of view (e.g., because scanning unit 104 scans different portions of the field of view at different times). The narrow field of view for the incident beam (if implemented) eliminates the problem of defocus imaging. FIG. 4D As illustrated, sensor 116 may include multiple lenses 422 (e.g., microlenses), each lens 422 directing incident light toward a different detection element 402 (e.g., toward the active region of detection element 402), which is useful when defocusing is not an issue. Lenses 422 can be used to increase the optical fill factor and sensitivity of detector array 400 because most of the light reaching sensor 116 can be deflected toward the active region of detection element 402.

[0107] like FIG. 4DAs illustrated, the detector array 400 may include several layers embedded in a silicon substrate by various methods (e.g., implantation) to create sensitive regions, contact elements with metal layers, and isolation elements (e.g., shallow trench implantation (STI), guard rings, optical trenches, etc.). The sensitive regions may be volumetric elements in a CMOS detector that enable the optical conversion of incident photons into current when a sufficient voltage bias is applied to the device. In the case of an APD / SPAD, the sensitive regions will be a combination of electric fields that pull electrons generated by photon absorption towards a multiplication region where photon-induced electrons are amplified, resulting in a breakdown avalanche of multiplied electrons.

[0108] Detectors illuminated on the front (e.g., such as) FIG. 4D The image shown has an input optical port on the same side as the metal layer residing on top of the semiconductor (silicon). The metal layer is needed to enable electrical connections between each individual photodetector element (e.g., anode and cathode) and various other elements (such as bias voltages, quench / ballast elements, and other photodetectors in the common array). The optical port through which photons illuminate the sensitive area of ​​the detector is formed by channels through the metal layer. It should be noted that light passing through this channel from some directions can be transmitted through one or more metal layers (e.g., metal layer ML6, as shown). FIG. 4D The leftmost detector element 402 (illustrated in the diagram) is blocked. This blockage reduces the overall optical light absorption efficiency of the detector.

[0109] FIG. 4E The illustration shows three detection elements 402 according to an example of the currently disclosed subject matter, each detection element having an associated lens 422. FIG. 4E The diagram illustrates a lens configuration that can be implemented in association with one or more of the detection elements 402 of sensor 116, represented as 402(1), 402(2), and 402(3). It should be noted that combinations of these lens configurations can also be implemented.

[0110] In the lens configuration illustrated with respect to detection element 402(1), the focal point of the associated lens 422 can be located above the semiconductor surface. Optionally, the openings in different metal layers of the detection element can have different sizes aligned with the focusing cone generated by the associated lens 422. Such a structure can improve the signal-to-noise ratio and resolution of the array 400 as a whole device. Large metal layers can be important for power delivery and grounding shielding. This approach can be useful, for example, for transceiver lidar designs with narrow field of view, where the incident beam consists of parallel rays and the imaging focus has no effect on the detected signal.

[0111] In the lens configuration illustrated with respect to the left detection element 402(1), the photon detection efficiency of the detection element 402 can be improved by identifying a sweet spot. Specifically, a photodetector implemented in CMOS can have a sweet spot in a sensitive volume region where the probability of a photon generating an avalanche effect is highest. Thus, the focal point of the lens 422 can be located within the sensitive volume region at the sweet spot location, as demonstrated by the detection element 402(1). The lens shape and distance from the focal point can take into account the refractive indices of all elements along the path from the lens to the sensitive sweet spot location buried in the semiconductor material.

[0112] In the lens configuration illustrated with respect to the right detection element, a diffuser and a reflective element can be used to improve the photon absorption efficiency in the semiconductor material. Specifically, near-IR wavelengths require a significantly long path of silicon material in order to achieve a high probability of absorption of a photon that has traveled through. In a typical lens configuration, a photon can pass through the sensitive region and can not be absorbed into a detectable electron. For a CMOS device manufactured with a typical foundry process, the long absorption path that improves the probability of a photon generating an electron changes the size of the sensitive region toward a less practical dimension (e.g., tens of μιη). FIG. 4E The rightmost detection element demonstrates a technique for handling incident photons. The associated lens 422 focuses the incident light onto a diffuser element 424. In one embodiment, the light sensor 116 can also include a diffuser in a gap located away from an outer surface of at least some of the detectors. For example, the diffuser 424 can turn the light beam laterally (e.g., as perpendicular as possible) toward the sensitive region and a reflective optical trench 426. The diffuser is located at, above, or below the focal point. In this embodiment, the incident light can be focused at a specific location where the diffuser element is located. Optionally, the detector element 422 is designed to optically avoid inactive regions where photon-induced electrons can be lost and reduce the effective detection efficiency. The reflective optical trench 426 (or other forms of optical reflective structures) bounces the photons back and forth over the sensitive region, increasing the likelihood of detection. Ideally, the photons will be trapped indefinitely in a cavity composed of the sensitive region and the reflective trench until the photons are absorbed and generate electron / hole pairs. FIG. 4E

[0113] ​Consistent with the present disclosure, long paths are created to have the illuminating photons absorbed and contribute to higher probability of detection. Optical trenches can also be implemented in the detection elements 422 for reducing the crosstalk effect of spurious photons generated during avalanche that can leak to other detectors and cause false detection events. According to some embodiments, the photodetector array can be optimized so as to take advantage of higher received signal yield, which means that the same amount of received signal is received and less signal is lost to internal degradation of the signal. The photodetector array can be improved by: (a) moving the focal point to a location above the semiconductor surface, optionally by designing the metal layer above the substrate appropriately; (b) by steering the focal point to the most responsive / sensitive area of the substrate (or "sweet spot") and (c) adding a diffuser above the substrate to steer the signal toward the "sweet spot" and / or adding reflective material to the trench so that the deflected signal is reflected back to the "sweet spot".

[0114] While in some lens configurations, the lenses 422 can be placed such that their focal points are above the center of the corresponding detection elements 402, it is noted that this need not be the case. In other lens configurations, the focal points of the lenses 422 are shifted relative to the position of the center of the corresponding detection elements 402 based on the distance of the respective detection elements 402 from the center of the detection array 400. This can be useful in relatively large detection arrays 400, where the detector elements further from the center receive light at increasingly off-axis angles. Moving the position of the focal point (e.g., toward the center of the detection array 400) allows for correction of the angle of incidence. In particular, moving the position of the focal point (e.g., toward the center of the detection array 400) allows for correction of the angle of incidence while using substantially the same lenses 422 for all detection elements, which are placed at the same angle relative to the surface of the detector.

[0115] When using a relatively small sensor 116 that only covers a small portion of the field of view, it can be useful to add an array of lenses 422 to the array of detection elements 402, because in this case the reflected signals from the scene arrive at the detector array 400 from essentially the same angle, and thus it is easy to focus all the light onto the individual detectors. It is also noted that in one embodiment, the lenses 422 can be used in the lidar system 100 to facilitate increasing the overall detection probability of the entire array 400 (preventing photons from being "wasted" in dead zones between detectors / sub-detectors), at the expense of spatial uniqueness. This embodiment is in contrast to prior art embodiments such as CMOS RGB cameras, which prioritize spatial uniqueness (i.e., do not allow light propagating in the direction of detection element A to be directed by the lens toward detection element B, i.e., "escape" to another detection element of the array). Optionally, the sensor 116 includes an array of lenses 422, each lens being associated with a corresponding detection element 402, while at least one of the lenses 422 deflects light propagating to a first detection element 402 toward a second detection element 402 (whereby it can increase the overall detection probability of the entire array).

[0116] In particular, consistent with some embodiments of the present disclosure, the light sensor 116 can include an array of light detectors (e.g., the detector array 400), each light detector (e.g., the detector 410) being configured to cause a flow of electrical current when light passes through an outer surface of the respective detector. Further, the light sensor 116 can include at least one microlens configured to direct light toward the array of light detectors, the at least one microlens having a focal point. The light sensor 116 can also include at least one layer of electrically conductive material interposed between the at least one microlens and the array of light detectors and having a gap therein to allow light to pass from the at least one microlens to the array, the at least one layer being sized to maintain a space between the at least one microlens and the array such that the focal point (e.g., the focal point can be a plane) is located in the gap at a location spaced apart from a detection surface of the array of light detectors.

[0117] In related embodiments, each detector can include a plurality of single-photon avalanche diodes (SPADs) or a plurality of avalanche photodiodes (APDs). The electrically conductive material can be a multi-layer metal pinch, and at least one layer of electrically conductive material can be electrically connected to a detector in the array. In one example, the at least one layer of electrically conductive material includes a plurality of layers. Further, the gap can be shaped to converge from the at least one microlens toward a focal point, and diverge from the area of the focal point toward the array. In other embodiments, the light sensor 116 can also include at least one reflector adjacent to each photodetector. In one embodiment, the plurality of microlenses can be arranged in a lens array, and the plurality of detectors can be arranged in a detector array. In another embodiment, the plurality of microlenses can include a single lens configured to project light to a plurality of detectors in the array.

[0118] By way of non-limiting example reference FIG. 2E 、 FIG. 2F and FIG. 2G It is noted that one or more sensors 116 of the system 100 can receive light from the scan deflector 114, or directly from the FOV in the absence of scanning. Even if light from the entire FOV arrives at the at least one sensor 116 simultaneously, in some implementations, the one or more sensors 116 can sample only a portion of the FOV at any given time for detection output. For example, if the illumination of the projection unit 102 illuminates different portions of the FOV at different times (whether using the deflector 114 and / or by activating different light sources 112 at different times), light can arrive at all pixels or sensors 116 of the sensing unit 106, and only the pixels / sensors that are expected to detect lidar illumination can be actively collecting data for detection output. In this way, the remaining pixels / sensors do not unnecessarily collect ambient noise. With respect to scanning— in the outbound direction or in the inbound direction— it is noted that substantially different scan scales can be implemented. For example, in some implementations, the scan region can cover 1% or 0.1% of the FOV, while in other implementations, the scan region can cover 10% or 25% of the FOV. Of course, all other relative portions of the FOV values can also be implemented.

[0119] Processing Unit

[0120] FIGS. 5A-5C Different functions of the processing unit 108 are depicted in accordance with some embodiments of the present disclosure. In particular, FIG. 5A is a diagram illustrating a pattern of emission in a single frame time for a single portion of the field of view, FIG. 5B is a diagram illustrating an emission scheme in a single frame time for the entire field of view, and FIG. 5C is a diagram illustrating actual light emission projected toward the field of view during a single scan cycle.

[0121] FIG. 5A Four examples of emission patterns for a single portion 122 of the field of view 120 associated with an instantaneous position of the at least one light deflector 114 over a single frame time are illustrated. Consistent with embodiments of the present disclosure, the processing unit 108 can control (or coordinate operation of) the at least one light source 112 and the light deflector 114 in a manner that enables the luminous flux to vary as the field of view 120 is scanned. Consistent with other embodiments, the processing unit 108 can control only the at least one light source 112, and the light deflector 114 can move or pivot in a fixed, predefined pattern.

[0122] FIG. 5A Figures A-D in the diagram depict the power of light emitted toward the single portion 122 of the field of view 120 over time. In Figure A, the processor 118 can control operation of the light source 112 in a manner that enables an initial light emission to be projected toward the portion 122 of the field of view 120 during scanning of the field of view 120. When the projection unit 102 includes a pulsating light source, the initial light emission can include one or more initial pulses (also referred to as “pilot pulses”). The processing unit 108 can receive pilot information from the sensor 116 regarding reflections associated with the initial light emission. In one embodiment, the pilot information can be represented as a single signal based on the output of one or more detectors (e.g., one or more SPADs, one or more APDs, one or more SiPMs, etc.), or as multiple signals based on the output of multiple detectors. In one example, the pilot information can include analog and / or digital information. In another example, the pilot information can include a single value and / or multiple values (e.g., for different times and / or portions of a segment).

[0123] Based on the information regarding reflections associated with the initial light emission, the processing unit 108 can be configured to determine a type of subsequent light emission to project toward the portion 122 of the field of view 120. The subsequent light emission determined for a particular portion of the field of view 120 can occur during the same scan cycle (i.e., in the same frame) or in a subsequent scan cycle (i.e., in a subsequent frame).

[0124] In FIG. B, the processor 118 can control operation of the light source 112 in a manner such that light pulses of different intensities are projected toward individual portions 122 of the field of view 120 during a scan of the field of view 120. In one embodiment, the lidar system 100 can be operable to generate one or more different types of depth maps, such as any one or more of the following types: a point cloud model, a polygon mesh, a depth image (holding depth information for each pixel of an image or 2D array), or any other type of 3D model of a scene. The sequence of depth maps can be a time sequence, with different depth maps generated at different times. Each depth map of the sequence associated with a scan period (interchangeably referred to as a “frame”) can be generated over the duration of a corresponding subsequent frame time. In one example, a typical frame time can last less than a second. In some embodiments, the lidar system 100 can have a fixed frame rate (e.g., 10 frames per second, 25 frames per second, 50 frames per second), or the frame rate can be dynamic. In other embodiments, the frame times for different frames can not be the same across the sequence. For example, the lidar system 100 can implement a rate of 10 frames per second that includes generating a first depth map in 100 milliseconds (average), a second frame in 92 milliseconds, and a third frame at 142 milliseconds, and so on.

[0125] In FIG. C, the processor 118 can control operation of the light source 112 in a manner such that light pulses associated with different durations are projected toward individual portions 122 of the field of view 120 during a scan of the field of view 120. In one embodiment, the lidar system 100 can be operable to generate a different number of pulses in each frame. The number of pulses can vary between 0 and 32 pulses (e.g., 1, 5, 12, 28, or more pulses), and can be based on information derived from previous emissions. The time between light pulses can depend on the desired detection range, and can be between 500 ns and 5000 ns. In one example, the processing unit 108 can receive information from the sensor 116 regarding reflections associated with each light pulse. Based on that information (or the absence of that information), the processing unit 108 can determine whether additional light pulses are needed. It is noted that the durations of the processing times and emission times in FIGS. A-D are not to scale. In particular, the processing times can be substantially longer than the emission times. In FIG. D, the projection unit 102 can include a continuous wave light source. In one embodiment, the initial light emission can include a time period during which light is emitted, and the subsequent emission can be a continuation of the initial emission, or there can be discontinuities. In one embodiment, the intensity of the continuous emission can vary over time.

[0126] Consistent with some embodiments of the present disclosure, the emission pattern can be determined on a per portion of the field of view 120. In other words, the processor 118 can control the emission of light to allow for differentiation of illumination of different portions of the field of view 120. In one example, the processor 118 can determine the emission pattern for a single portion 122 of the field of view 120 based on detection of reflected light from the same scan cycle (e.g., an initial emission), which makes the lidar system 100 extremely dynamic. In another example, the processor 118 can determine the emission pattern for a single portion 122 of the field of view 120 based on detection of reflected light from a previous scan cycle. The difference in the pattern of the subsequent emission can result from different values of the light source parameters determined for the subsequent emission, such as any of the following:

[0127] a. total energy of the subsequent emission.

[0128] b. energy profile of the subsequent emission.

[0129] c. number of light pulse repetitions per frame.

[0130] d. light modulation characteristics, such as duration, rate, peak, average power, and pulse shape.

[0131] e. wave properties of the subsequent emission, such as polarization, wavelength, and the like.

[0132] Consistent with the present disclosure, the differentiation of the subsequent emission can be used for different purposes. In one example, it is possible to limit the emission power level in portions of the field of view 120 for which safety is a consideration, while emitting higher power levels for other portions of the field of view 120 (thereby improving signal-to-noise ratio and detection range). This is relevant to eye safety, but can also be relevant to skin safety, safety of optical systems, safety of sensitive materials, and the like. In another example, based on the results of the detection from the same frame or a previous frame, it is possible to direct more energy toward portions of the field of view 120 for which the energy will be more useful (e.g., regions of interest, targets at greater distances, low-reflectivity targets, and the like), while limiting the illumination energy to other portions of the field of view 120. It is noted that the processing unit 108 can process the detection signals from a single instantaneous field of view multiple times within a single scan frame time; for example, the subsequent emission can be determined after each pulse emission or after multiple pulse emissions.

[0133] FIG. 5BThree examples of emission schemes for a single frame time of the field of view 120 are illustrated. Consistent with embodiments of the present disclosure, the obtained information can be used, at least on the processing unit 108, to dynamically adjust the operating mode of the lidar system 100 and / or to determine parameter values for specific components of the lidar system 100. The obtained information can be determined from processing data captured in the field of view 120 or received (directly or indirectly) from the host 210. The processing unit 108 can use the obtained information to determine a scanning scheme for scanning different portions of the field of view 120. The obtained information can include current light conditions, current weather conditions, a current driving environment of the host vehicle, a current location of the host vehicle, a current trajectory of the host vehicle, a current terrain of the road surrounding the host vehicle, or any other conditions or objects detectable by light reflection. In some embodiments, the determined scanning scheme can include at least one of: (a) designating portions within the field of view 120 as to be actively scanned as part of a scanning cycle, (b) a projection plan for the projection unit 102 defining light emission profiles at different portions of the field of view 120; (c) a deflection plan for the scanning unit 104 defining, for example, deflection directions, frequencies, and designating idle elements within the reflector array; and (d) a detection plan for the sensing unit 106 defining detector sensitivity or responsivity patterns.

[0134] Further, the processing unit 108 can determine the scanning scheme at least in part by obtaining an identification of at least one region of interest within the field of view 120 and at least one region of non-interest within the field of view 120. In some embodiments, the processing unit 108 can determine the scanning scheme at least in part by obtaining an identification of at least one high region of interest within the field of view 120 and at least one lower region of interest within the field of view 120. For example, the identification of at least one region of interest within the field of view 120 can be determined, e.g., from processing data captured in the field of view 120, based on data from another sensor (e.g., a camera, a GPS), received (directly or indirectly) from the host 210, or any combination thereof. In some embodiments, the identification of at least one region of interest can include an identification of a portion, area, sector, pixel, or object within the field of view 120 that is important for monitoring. Examples of areas that can be identified as regions of interest can include a crosswalk, a moving object, a person, a nearby vehicle, or any other environmental condition or object that can aid in vehicle navigation. Examples of areas that can be identified as regions of non-interest (or lower interest) can be static (non-moving) distant buildings, a skyline, a horizon, and areas above objects in the field of view. Once the identification of at least one region of interest within the field of view 120 is obtained, the processing unit 108 can determine a scanning scheme or change an existing scanning scheme. To further determine or change the light source parameters (as described above), the processing unit 108 can allocate detector resources based on the identification of the at least one region of interest. In one example, to reduce noise, the processing unit 108 can activate detectors 410 expected to be at regions of interest and disable detectors 410 expected to be at regions of non-interest. In another example, the processing unit 108 can change detector sensitivity, e.g., increase sensor sensitivity for long-range detection of low-reflectivity power.

[0135] FIG. 5BFigures A-C in the middle depict examples of different scanning schemes for scanning the field of view 120. Each square in the field of view 120 represents a different portion 122 associated with an instantaneous position of at least one light deflector 114. The legend 500 details the levels of light flux represented by the fill patterns of the squares. Figure A depicts a first scanning scheme in which all portions have the same importance / priority and are assigned a default light flux. The first scanning scheme can be used in a startup phase or periodically interleaved with another scanning scheme to monitor the entire field of view for unexpected / new objects. In one example, the light source parameters in the first scanning scheme can be configured to generate light pulses at a constant amplitude. Figure B depicts a second scanning scheme in which a portion of the field of view 120 is assigned a high light flux, while the rest of the field of view 120 is assigned a default light flux and a low light flux. The least interesting portion of the field of view 120 can be assigned a low light flux. Figure C depicts a third scanning scheme in which a compact vehicle and a bus are identified in the field of view 120 (see the outline map). In this scanning scheme, the edges of the vehicle and the bus can be tracked at a high power, and the central mass of the vehicle and the bus can be assigned less light flux (or no light flux). This light flux assignment enables more optical budget to be concentrated on the edges of the identified objects, while less optical budget is concentrated on their less important centers.

[0136] FIG. 5C Figure illustrates light emission toward the field of view 120 during a single scanning cycle. In the depicted example, the field of view 120 is represented by an 8X9 matrix, where each of the 72 cells corresponds to a separate portion 122 associated with a different instantaneous position of at least one light deflector 114. In this example scanning cycle, each portion includes one or more white dots, which represent the number of light pulses projected toward that portion, and some portions include black dots, which represent reflected light from that portion detected by the sensor 116. As shown, the field of view 120 is divided into three regions: Region I on the right side of the field of view 120, Region II in the middle of the field of view 120, and Region III on the left side of the field of view 120. In this example scanning cycle, Region I is initially assigned a single light pulse for each portion; Region II, which was previously identified as a region of interest, is initially assigned three light pulses for each portion; and Region III is initially assigned two light pulses for each portion. Also as shown, the scanning of the field of view 120 reveals four objects 208: two freeform objects in the near field (e.g., between 5 and 50 meters), a rounded square object in the middle field (e.g., between 50 and 150 meters), and a triangular object in the far field (e.g., between 150 and 500 meters). Although the scanning cycle is depicted as a single cycle, the scanning of the field of view 120 can be performed in multiple cycles, where each cycle can be a different scanning scheme or a different set of light pulses. FIG. 5CThe discussion uses the number of pulses as an example of light flux allocation, but note that light flux allocation to different parts of the field of view can also be implemented in other ways, such as: pulse duration, pulse angular dispersion, wavelength, photon density at different distances from the light source 112, average power, pulse power intensity, pulse width, pulse repetition rate, pulse sequence, pulse duty cycle, wavelength, phase, polarization, and so on. In FIG. 5C The illustration of light emission as a single scan cycle in FIG. 1 1 1 demonstrates different capabilities of the lidar system 100. In a first embodiment, the processor 1 18 is configured to detect a first object (e.g., a rounded square object) at a first distance using two light pulses, and to detect a second object (e.g., a triangular object) at a second distance greater than the first distance using three light pulses. In a second embodiment, the processor 1 18 is configured to allocate more light to the parts of the field of view that identify regions of interest. In particular, in this example, Region II is identified as a region of interest, and thus it is allocated three light pulses, while the rest of the field of view 120 is allocated two or fewer light pulses. In a third embodiment, the processor 1 18 is configured to control the light source 1 12 in such a way that only a single light pulse is projected toward FIG. 5C B1, B2, and C1 in FIG. 1 1 1 B, although they are part of Region III, which was initially allocated two light pulses per part. This occurs because the processing unit 108 detects an object in the near field based on the first light pulse. Allocation of less than the maximum pulse amount can also be the result of other considerations. For example, detection of an object at a first distance (e.g., a near field object) in at least some regions can result in a reduction of the total amount of light emitted to this part of the field of view 120.

[0137] Additional details and examples regarding different components of the lidar system 100 and their associated functionality are included in U.S. Patent Application No. 15 / 391,916, filed by the Applicant on December 28, 2016; U.S. Patent Application No. 15 / 393,749, filed by the Applicant on December 29, 2016; U.S. Patent Application No. 15 / 393,285, filed by the Applicant on December 29, 2016; and U.S. Patent Application No. 15 / 393,593, filed by the Applicant on December 29, 2016, which are incorporated by reference herein in their entireties.

[0138] Exemplary Implementation: Vehicle

[0139] FIGS. 6A-6CAn implementation of the lidar system 100 in a vehicle (e.g., vehicle 110) is illustrated. Any of the aspects of the lidar system 100 described above or below can be incorporated into the vehicle 110 to provide a range-sensing vehicle. In particular, in this example, the lidar system 100 integrates multiple scan units 104 and potentially multiple projection units 102 in a single vehicle. In one embodiment, a vehicle can utilize, for example, such a lidar system to improve power, range, and accuracy in the overlap region and beyond, as well as redundancy in sensitive portions of the FOV (e.g., the forward direction of movement of the vehicle). As FIG. 6A As shown in the middle, the vehicle 110 can include a first processor 118A to control scanning of the field of view 120A, a second processor 118 to control scanning of the field of view 120B, and a third processor 118C to control synchronization of scanning of the two fields of view. In one example, the processor 118C can be a vehicle controller and can have a shared interface between the first processor 118A and the second processor 118. The shared interface can enable exchange of data at an intermediate processing level and enable synchronization of scanning of the combined fields of view to form an overlap in time and / or space. In one embodiment, the data exchanged using the shared interface can be: (a) time of flight of received signals associated with pixels in and / or near the overlap field of view; (b) laser steering position state; (c) detection state of objects in the field of view.

[0140] FIG. 6B An overlap region 600 between the field of view 120A and the field of view 120B is illustrated. In the depicted example, the overlap region is associated with 24 portions 122 from the field of view 120A and 24 portions 122 from the field of view 120B. Assuming the overlap region is defined and known by the processors 118A and 118, then each processor can be designed to limit the amount of light emitted in the overlap region 600 to comply with eye safety limits across multiple light sources, or for other reasons such as maintaining an optical budget. Further, the processors 118A and 118 can avoid interference between light emitted by the two light sources through loose synchronization between the scan units 104A and 104B and / or through control of laser transmit timing and / or detection circuit enable timing.

[0141] FIG. 6CThe overlap region 600 between the fields of view 120A and 120B can be used to increase the detection range of the vehicle 110 is illustrated. Two or more light sources 112 that project their nominal light emission into the overlap region can be leveraged to increase the effective detection range, consistent with the present disclosure. The term “detection range” can include an approximate distance from the vehicle 110 at which the lidar system 100 can clearly detect an object. In one embodiment, the maximum detection range of the lidar system 100 is approximately 300 meters, approximately 400 meters, or approximately 500 meters. For example, for a detection range of 200 meters, the lidar system 100 can detect an object that is 200 meters (or less) from the vehicle 110 more than 95%, more than 99%, more than 99.5% of the time. Even if the reflectivity of the object can be less than 50% (e.g., less than 20%, less than 10%, or less than 5%). Furthermore, the lidar system 100 can have a false positive rate of less than 1%. In one embodiment, the SNR can be improved and thus the service range and / or quality of objects located in the overlap region can be increased with light from two light sources that are collocated in time and space. The processor 118C can extract high level information from the reflected light in the fields of view 120A and 120B. The term “extract information” can include any processing to identify information associated with objects, individuals, locations, events, and the like in captured image data by any means known to one of ordinary skill in the art. Furthermore, the processors 118A and 118 can share high level information, such as objects (road dividers, background, pedestrians, vehicles, and the like) and motion vectors to enable each processor to alert the surrounding area that it is about to become a region of interest. For example, a moving object in the field of view 120A can be determined to soon enter the field of view 120B.

[0142] Exemplary Implementation: Monitoring System

[0143] FIG. 6D An implementation of the lidar system 100 in a surveillance system is illustrated. As mentioned above, the lidar system 100 can be fixed to a stationary object 650, which can include a motor or other mechanism to rotate the housing of the lidar system 100 to obtain a wider field of view. Alternatively, the surveillance system can include multiple lidar units. In the depicted example, the surveillance system can use a single rotatable lidar system 100 to obtain 3D data representing the field of view 120 and process the 3D data to detect people 652, vehicles 654, changes in the environment, or any other form of safety-relevant data. FIG. 6D

[0144] ​Consistent with some embodiments of the present disclosure, 3D data can be analyzed to monitor retail business processes. In one embodiment, 3D data can be used in retail business processes involving physical security (e.g., detecting: intrusions into a retail facility, vandalism within or around a retail facility, unauthorized access to secure areas, and suspicious behavior around cars in a parking lot). In another embodiment, 3D data can be used in public safety (e.g., detecting: people slipping and falling on store property, hazardous liquid spills or obstructions on store floors, attacks or abductions in a store parking lot, obstructions of fire exits, and crowding in or outside a store area). In another embodiment, 3D data can be used for business intelligence data gathering (e.g., tracking people through a store area to determine, for example, how many people pass through, where they stop, how long they stop, what their shopping habits look like compared to their purchasing habits).

[0145] Consistent with other embodiments of the present disclosure, 3D data can be analyzed and used for traffic enforcement. In particular, 3D data can be used to identify vehicles traveling in excess of a legal speed limit or some other road legal requirement. In one example, a lidar system 100 can be used to detect vehicles crossing stop lines or designated pull-over locations while a red traffic signal light is displayed. In another example, a lidar system 100 can be used to identify vehicles traveling in lanes reserved for public transportation. In yet another example, a lidar system 100 can be used to identify vehicles making turns at intersections where specific turns are prohibited at red lights.

[0146] Eye-safe laser radar system

[0147] Eye safety requirements in lidar systems and other electro-optical systems can limit the amount of illumination that a system can emit per unit of time. A maximum permissible exposure (MPE) can be defined for different light sources, which depends on various factors such as the wavelength of the light source. The MPE defines the highest power or energy density (in W / cm2or J / cm2) that is considered safe. In some cases, the MPE can depend on the total time of exposure. The systems and methods described herein enable relatively high levels of illumination to be emitted for lidar detection while still maintaining eye safety.

[0148] FIG. 7 is a diagram illustrating an example lidar system 700 consistent with some embodiments of the present disclosure. As shown, the lidar system 700 can include a light emission component 702, a sensing unit 710, and a processing unit 714. FIG. 7

[0149] ​The light emission assembly 702 can be configured to emit light emissions into a field of view of the lidar system 700 based on instructions received from the processing unit 714. The light emission assembly 702 can include a light source 704 and optics 708. The light source 704 can be configured to emit light. The processing unit 714 can be programmed to cause the light emission assembly to scan the field of view multiple times during a frame and construct a point cloud based on reflections received from the scans during the frame. In some embodiments, the processing unit 714 can be programmed to cause the light emission assembly to scan the field of view more than 2, 3, 5, 10, 20, 50, or 100, 200, more than 1,000, or any intervening number of times during a frame.

[0150] In some embodiments, the light source 704 can include one or more light sources of one or more types described elsewhere in the present disclosure (e.g., lasers, LEDs, vertical cavity surface emitting lasers (VCSELs), pixel arrays, etc.). In some embodiments, the light source 704 can include two or more light sources configured to emit light emissions. For example, the light source 704 can include a first light source and a second light source. The first light source can be configured to emit a first light emission, and the second light source can be configured to emit a second light emission. The first light emission can be different from the second light emission. For example, the first light emission can have a different wavelength, intensity, power level, etc., or a combination thereof, than the second light emission.

[0151] The processing unit 714 can be programmed to control one or more components of the lidar system 700. For example, the processing unit 714 can be configured to control the light emission assembly 702 to emit light emissions into a field of view (or one or more segments thereof) of the lidar system 700. In some embodiments, the processing unit 714 can include a processor 716 configured to perform the functions of the processing unit 714 described in the present disclosure. The processor 716 can be similar to the processor 118 described elsewhere in the present disclosure. For example, the processor 716 can be programmed to control at least one light source to enable a variation in luminous flux within a scan of a field of view using light from the at least one light source. As another example, the processor 716 can be operable to determine whether an object is located in a field of view of a lidar system based on reflection signals of light from an environment of the lidar system received by at least one sensor.

[0152] The processing unit 714 can be programmed to control the light sources to sequentially illuminate non-contiguous segments included in a first set of non-contiguous segments of the field of view of the lidar system. During illumination of a particular non-contiguous segment of the first set of non-contiguous segments, other segments of the plurality of segments can not be illuminated. Additionally, other segments of the plurality of segments can not be illuminated between illumination of non-contiguous segments of the first set of non-contiguous segments. Each illumination directed to a non-contiguous segment of the first set of non-contiguous segments can not exceed a predetermined threshold. In some embodiments, the predetermined threshold can be an illumination level associated with a standard maximum permissible exposure (MPE), which can be the highest power or energy density (in W / cm2or J / cm2) of light directed to a human that is considered safe, i.e., the likelihood of causing damage can be negligible. For example, the standard MPE can satisfy the requirements of Class 1 eye safety (e.g., according to the International Electrotechnical Commission (IEC) standard 60825-1).

[0153] In some embodiments, the illumination level of each segment of the field of view can be limited within an eye safety threshold (e.g., equal to or below the standard MPE), and the illumination level of a portion of the field of view (which can include two or more segments adjacent to each other) can exceed the eye safety threshold for continuous illumination. As used herein, the term “continuous illumination” refers to illumination of all segments of a portion of the FOV before moving to other segments of the FOV (e.g., within a single frame of lidar detection). Optionally, the processing unit 714 can apply a scanning pattern to the at least one light source, where the segments of all portions of the FOV can be scanned in an intermittent manner such that each temporary illumination level of the light source for continuous illumination (e.g., within the same instance position of the scanning deflector) can be limited within the eye safety threshold (e.g., below the standard MPE), while the illumination level of some or all portions exceeds the eye safety threshold for continuous illumination. Optionally, the processing unit 714 can control such illumination scheme according to Class 1 eye safety (e.g., according to the International Electrotechnical Commission (IEC) standard 60825-1).

[0154] In some embodiments, the lidar system 700 can include a fast scanning mirror (e.g., a ID scanning mirror or a 2D scanning mirror). The processing unit 714 controls the scanning mirror such that the mirror can scan the entire FOV (served by that mirror) multiple times (e.g., more than 5 times, more than 10 times, more than 20 times, more than 50 times, more than 200 times, more than 1,000 times, or any intermediate number of times, etc.) per frame. Optionally, the processing unit 714 can control synchronization between the fast scanning mirror and the at least one light source such that only a small portion of the FOV served in each scan of the mirror can be illuminated and scanned for each scan period within that frame.

[0155] The sensing unit 710 can include a sensor 712 configured to detect reflections from the field of view of the lidar system 700. The sensor 712 can include any device, element, or system capable of measuring a characteristic (e.g., power, frequency, phase, pulse timing, pulse duration) of an electromagnetic wave and generating an output related to the measured characteristic. In some embodiments, the at least one sensor can include a plurality of detectors made up of a plurality of detection elements. The sensor 712 can include one or more types of light sensors. It is noted that the at least one sensor can include a plurality of sensors of the same type, which can differ in other characteristics (e.g., sensitivity, size, etc.). Other types of sensors can also be used. A combination of several types of sensors can be used for different reasons, such as to improve detection over a range span, especially in the near range; to improve the dynamic range of the sensor; to improve the time response of the sensor; and to improve detection under varying environmental conditions (e.g., atmospheric temperature, rain, etc.). In one embodiment, the at least one sensor can include a SiPM (silicon photomultiplier), which is a solid-state single-photon sensitive device built from an array of avalanche photodiodes (APDs), single-photon avalanche diodes (SPADs), functioning as detection elements on a common silicon substrate. In one example, the typical distance between SPADs can be between about 7 pm and about 50 pm, and each SPAD can have a recovery time between about 20 ns and about 70 ns. Similar photomultipliers from other non-silicon materials can also be used. While SiPM devices work in a digital / switching mode, SiPM is an analog device because all microcells can be read in parallel, enabling it to generate signals in a dynamic range from a single photon to thousands of photons detected by different SPADs. It is noted that the outputs from different types of sensors (e.g., SPAD, APD, SiPM, PIN diode, photodetector) can be combined together to form a single output that can be processed by a processor of the lidar system. Reference is made below to FIGS. 4A-4C Additional details are described regarding the sensing unit and the at least one sensor. Optionally, the lidar system 700 can include a scanning unit for directing flash illumination to different portions of the field of view at different times. In this case, the determination of the spatial light modulation can be performed for each portion of the field of view (e.g., if eye safety is an issue), but not necessarily so (e.g., if sensor pixel failure is an issue). In some embodiments, the sensor 712 can include a detector array, which can include a focal plane detector array.

[0156] In some embodiments, the lidar system 700 (or light emitting assembly 702) may include a light deflector (not shown) configured to deflect light from at least one light source into the field of view. The light deflector may include a microelectromechanical system (MEMS) mirror, a rotating prism, an optical phased array controller, a vertical-cavity surface-emitting laser (VCSEL) array controller, a scanning mirror, or a combination thereof.

[0157] In some embodiments, the light emitting component 702 may include a spatial light modulator (not shown) configured to modulate the luminous flux to vary within a scan of the field of view. For example, the spatial light modulator may be configured to block (and / or suppress) light emission from one or more light sources. The spatial light modulator may include one or more spatial filters that selectively filter (and / or block) light emission (or a portion thereof) emitted from light source 704. Optics 708 may be configured to direct light emission from the spatial light modulator into the field of view. For example, the spatial light modulator may allow light emission from light source 704 to be transmitted into field of view 720. In some embodiments, the spatial light modulator may modulate light emission from light source 704 in a non-binary manner. For example, the spatial light modulator may suppress a portion of the light emission (e.g., the intensity of the light emission is reduced by the spatial light modulator), and the suppressed light emission may be emitted into the corresponding portion of the field of view.

[0158] FIG. 8 This diagram illustrates a portion of an exemplary field of view 800 of a lidar system 700 consistent with the disclosed embodiments. The lidar system 700 can be configured to control at least one light source such that the luminous flux can vary within a scan of the field of view 720 using light from the at least one light source. The field of view 800 may be... FIG. 7 This is a portion of the field of view 720 shown. (As shown) FIG. 8 As shown, the field of view 800 may include 120 FOV pixels arranged in a 15x8 array. Laser points 810, corresponding to light pulses emitted by the light source of the lidar system, may appear in segments of a size corresponding to 4x1 pixels in the lidar system's sensor. Laser points 810 may scan the field of view 800 within two rows of 15x4 FOV pixels.

[0159] The field of view 800 can be divided into multiple segments. In some embodiments, each segment may have a size sufficient to cover the size of a unit light beam (e.g., laser point 810) guided into the field of view. In some embodiments, one or more human eyes may appear within the field of view of the lidar system. For example, as... FIG. 8As shown, a human eye 850, including a pupil 851 and an iris 852, can appear in the field of view 800. The human eye 850 can include an angular size in a given short-range distance such that the pupil 851 corresponds to 9 (3x3) FOV pixels. Scanning the beam from left to right (or right to left) in the example shown can result in the pupil 851 being exposed to three consecutive illumination sequences (e.g., one or more light pulses for each column of 1x4 FOV pixels, or a continuous emission for each column). Since eye damage is cumulative over time, it can be appropriate to limit the MPE of the system to a given standard MPE.

[0160] To avoid causing damage to the human eye 850 (and / or the pupil 851) by directing too much optical power to the human eye 850, the lidar system can illuminate light into segments of the field of view in a non-continuous manner. For example, the field of view 800 can be divided into a plurality of segments by, for example, the processing unit 714. The field of view 800 can include a first set of non-continuous segments. Each of the non-continuous segments included in the first set can be separated by at least one segment from other non-continuous segments in the first set. For example, the first set of non-continuous segments can include segment 821 and segment 822. Segment 821 can be separated by three segments from segment 822. The processing unit 714 can be programmed to control the light source to illuminate the non-continuous segments included in the first set of non-continuous segments sequentially. Other segments in the plurality of segments can not be illuminated during illumination of a particular non-continuous segment in the first set of non-continuous segments. In some embodiments, other segments in the plurality of segments can not be illuminated between illumination of the non-continuous segments in the first set of non-continuous segments. For example, the processing unit 714 can be programmed to control the light source to illuminate segment 821 without illuminating other segments in the plurality of segments. The processing unit 714 can be programmed to control the light source to subsequently illuminate segment 822, and other segments in the plurality of segments (including segments between segment 821 and segment 822, such as segments 831, 841, 832, 842) can not be illuminated between illumination of segment 821 and segment 822. Each illumination directed to a non-continuous segment in the first set of non-continuous segments can not exceed a predetermined threshold. In some embodiments, the predetermined threshold can be an illumination level associated with a standard maximum permissible exposure (MPE), which can be the highest power or energy density (in W / cm2or J / cm2) of light directed to a human that is considered safe, i.e., the likelihood of causing damage can be negligible. For example, the standard maximum permissible exposure (MPE) can satisfy the requirements of Class 1 eye safety (e.g., according to the International Electrotechnical Commission (IEC) standard 60825-1).

[0161] In some embodiments, the plurality of segments of the field of view 800 can include a second set of non-continuous segments different from the first set of non-continuous segments. For example, as shown in FIG. 8B, the field of view 800 can include a second set of non-continuous segments including segment 831 and segment 832. Segment 831 can be separated by at least one segment from segment 832. The processing unit 714 can be programmed to control the light source to illuminate the non-continuous segments included in the second set of non-continuous segments sequentially. Other segments in the plurality of segments can not be illuminated during illumination of a particular non-continuous segment in the second set of non-continuous segments. In some embodiments, other segments in the plurality of segments can not be illuminated between illumination of the non-continuous segments in the second set of non-continuous segments. For example, the processing unit 714 can be programmed to control the light source to illuminate segment 831 without illuminating other segments in the plurality of segments. The processing unit 714 can be programmed to control the light source to subsequently illuminate segment 832, and other segments in the plurality of segments (including segments between segment 831 and segment 832, such as segments 821, 841, 822, 842) can not be illuminated between illumination of segment 831 and segment 832. Each illumination directed to a non-continuous segment in the second set of non-continuous segments can not exceed the predetermined threshold. FIG. 8As shown, the field of view 800 can include a second set of non-contiguous segments, including segment 831 and segment 832. Each of the non-contiguous segments included in the second set is separated from other non-contiguous segments in the second set by at least one segment. For example, segment 831 and 832 can be separated by three segments. The processing unit 714 can also be programmed to control the light source to sequentially illuminate the non-contiguous segments included in the second set of non-contiguous segments after the sequential illumination of the first set of non-contiguous segments. For example, after the sequential illumination of the first set of non-contiguous segments (e.g., segment 821 and segment 822), the processing unit 714 can be programmed to control the light source to sequentially illuminate segment 831 segment 832. During the illumination of a particular non-contiguous segment in the second set of non-contiguous segments, other segments in the plurality of segments can not be illuminated. In some embodiments, other segments in the plurality of segments can not be illuminated between the illumination of the non-contiguous segments in the second set of non-contiguous segments. For example, the processing unit 714 can be programmed to control the light source to illuminate segment 831 without illuminating other segments in the plurality of segments. The processing unit 714 can be programmed to control the light source to subsequently illuminate segment 832, and other segments in the plurality of segments (including 821, 841, 822, 842) can not be illuminated between the illumination of segment 831 and segment 832. Each illumination directed to a non-contiguous segment in the second set of non-contiguous segments can not exceed a predetermined threshold. The predetermined threshold can be an illumination level associated with a standard maximum permissible exposure (MPE), such as an illumination level that satisfies a Class 1 eye safety requirement (e.g., according to International Electrotechnical Commission (IEC) standard 60825-1).

[0162] In some embodiments, while illumination of a single segment (e.g., during a scan period or during a frame) can not exceed the predetermined threshold, the total illumination of the illumination of a particular segment and the illumination of a segment adjacent to the particular segment (e.g., during a scan period or during a frame) can exceed the predetermined threshold. Alternatively, the total illumination of the illumination of three (or more) adjacent segments (e.g., during a scan period or during a frame) can exceed the predetermined threshold, while the total illumination of any subset of the illumination of the three (or more) adjacent segments (e.g., during a scan period or during a frame) can not exceed the predetermined threshold. For example, segment 821 and segment 831 can be adjacent to each other. Neither the illumination of segment 821 nor the illumination of segment 831 can exceed the predetermined threshold during a scan period, while the total illumination of the illumination of segment 821 and the illumination of segment 831 can exceed the predetermined threshold.

[0163] In some embodiments, a non-continuous segment included in the second group of non-continuous segments may include a segment adjacent to a first non-continuous segment included in the first group of non-continuous segments. For example, a non-continuous segment included in the second group of non-continuous segments may include segment 831 adjacent to segment 821 (which may be one of the non-continuous segments included in the first group of non-continuous segments), such as... FIG. 8 As shown. As another example, the non-contiguous segments included in the second set of non-contiguous segments may include segment 832 adjacent to segment 822 (which may be one of the non-contiguous segments included in the first set of non-contiguous segments). In some embodiments, the illumination directed to the first non-contiguous segment included in the first set of non-contiguous segments and the segment adjacent to the first non-contiguous segment included in the first set of non-contiguous segments may be less than the illumination level associated with a predetermined threshold. For example, during a scan cycle, the illumination of both segment 821 and segment 831 may not exceed the predetermined threshold, while the total illumination of segment 821 and segment 831 may exceed the predetermined threshold. In some embodiments, the total illumination directed to the first non-contiguous segment included in the first set of non-contiguous segments and the segment adjacent to the first non-contiguous segment included in the first set of non-contiguous segments may be greater than the illumination level associated with a predetermined threshold. For example, the total illumination of segment 821 and segment 831 during a scan cycle may exceed the predetermined threshold.

[0164] In some embodiments, sequentially illuminating the non-contiguous segments included in the first set of non-contiguous segments can include sequentially illuminating the non-contiguous segments included in the first set of non-contiguous segments in each of a plurality of scans. For example, the processing unit 714 can be programmed to control the light source 704 to sequentially illuminate the segment 821 and the segment 822 in a plurality of scans (e.g., a first scan cycle, a second scan cycle, etc.). The sensing unit 710 can be configured to receive reflections of light from the environment from the light source in each scan. The processing unit 714 can be programmed to construct a point cloud output 822 including the segment 821 and the segment 822 based in part on the reflections summed from the plurality of scans of the non-contiguous segments included in the first set of non-contiguous segments. In some embodiments, as described above, the plurality of segments of the field of view 800 can include a second set of non-contiguous segments different from the first set of non-contiguous segments. The processing unit 714 can also be programmed to control the light source 704 to sequentially illuminate the non-contiguous segments included in the second set of non-contiguous segments (e.g., the segment 831, the segment 832) in each of the plurality of scans. The sensing unit 710 can be configured to receive reflections of light from the environment from the light source in each scan. The processing unit 714 can be programmed to construct a point cloud output 832 including the segment 821, the segment 822, the segment 831, and the segment 832 based in part on the reflections summed from the plurality of scans of the non-contiguous segments included in the first set of non-contiguous segments and the reflections summed from the plurality of scans of the non-contiguous segments included in the second set of non-contiguous segments.

[0165] In some embodiments, a first non-contiguous segment included in the first set of non-contiguous segments and a segment adjacent to the first non-contiguous segment included in the first set of non-contiguous segments have a same size. For example, as shown in FIG. 8A, the segment 821 (one of the non-contiguous segments included in the first set of non-contiguous segments) is adjacent to the segment 831 (one of the non-contiguous segments included in the second set of non-contiguous segments). The segment 821 and the segment 831 can have a same size. Alternatively, a first non-contiguous segment included in the first set of non-contiguous segments and a segment adjacent to the first non-contiguous segment included in the first set of non-contiguous segments have different sizes. Additionally, in some embodiments, a first non-contiguous segment included in the first set of non-contiguous segments and a segment adjacent to the first non-contiguous segment included in the first set of non-contiguous segments can have a same shape or different shapes. FIG. 8

[0166] ​In some embodiments, a first non-contiguous segment of the non-contiguous segments included in the first group of non-contiguous segments can be illuminated during a first scan period, and a segment adjacent to the first non-contiguous segment of the non-contiguous segments included in the first group of non-contiguous segments can be illuminated during a second scan period. For example, as shown in FIG. 8, segment 821 (one of the non-contiguous segments included in the first group of non-contiguous segments) can be illuminated during a first scan period, and segment 831 (one of the non-contiguous segments included in the second group of non-contiguous segments) can be illuminated during a second scan period. In some embodiments, segment 821 can not be illuminated during the second scan period. Alternatively or additionally, segment 831 can not be illuminated during the first scan period. FIG. 9

[0167] In some embodiments, the non-contiguous segments included in the first group of non-contiguous segments can be illuminated during a first scan period, and the non-contiguous segments included in the second group of non-contiguous segments can be illuminated during a second scan period. For example, segment 821 and segment 822 (the non-contiguous segments included in the first group of non-contiguous segments) can be illuminated during a first scan period. Segment 831 and segment 832 (the non-contiguous segments included in the second group of non-contiguous segments) can be illuminated during a second scan period. In some embodiments, segment 821 and segment 822 can not be illuminated during the second scan period. Alternatively or additionally, segment 831 and segment 832 can not be illuminated during the first scan period.

[0168] In some embodiments, processing unit 714 can be programmed to control light source 704 to illuminate at least one of the non-contiguous segments included in the first group of non-contiguous segments during a plurality of scan periods in a frame. For example, processing unit 714 can be programmed to control light source 704 to illuminate segment 821 (one of the non-contiguous segments included in the first group of non-contiguous segments) during a plurality of scan periods in a frame. The illumination directed to segment 821 during each of the plurality of scan periods can be less than an illumination level associated with a predetermined threshold. In some embodiments, a total illumination of the illumination directed to at least one of the non-contiguous segments included in the first group of non-contiguous segments during each of the plurality of scan periods is greater than the illumination level associated with the predetermined threshold. For example, a frame can include five scan periods, and the illumination directed to segment 821 during each of the five scan periods during the frame can be less than the illumination level associated with the predetermined threshold, and the total illumination of the illumination directed to segment 821 during each of the five scan periods can be greater than the illumination level associated with the predetermined threshold.

[0169] FIG. 7 ​This diagram illustrates a portion of an exemplary field of view 900 of a lidar system 700 consistent with the disclosed embodiments. The lidar system 700 can be configured to control at least one light source such that the luminous flux can vary within a scan of the field of view 720 using light from the at least one light source. The field of view 900 may be... FIG. 9 This is a portion of the field of view 720 shown. (As shown) FIG. 9 As shown, the field of view 900 may include 120 FOV pixels arranged in a 15x8 array. Laser points 910, corresponding to light pulses emitted by the light source of the lidar system, may appear in segments that may correspond to a 4x1 pixel size in the lidar system's sensor. Laser points 910 may scan the field of view 800 in two rows of 15x4 FOV pixels. In some embodiments, one or more human eyes may appear within the lidar system's field of view. For example, as... FIG. 10 As shown, a human eye 950, including a pupil 951 and an iris 952, can appear in a field of view 900. To avoid damage to the human eye 950 by directing excessive light power to the human eye 950 and pupil 951, the lidar system can illuminate the field of view in a discontinuous manner. For example, the field of view 900 can be divided into multiple parts, each of which can include multiple segments. For example, the field of view 900 can include a first part 920 and a second part 930. The first part can include a first segment and a second segment different from the first segment, and the second part can include a third segment and a fourth segment different from the third segment. For example, the first part 920 can include segments 921 and 922 (and 923 in some embodiments). The second part 930 can include segments 931 and 932 (and 933 in some embodiments).

[0170] The processing unit 714 can be programmed to control the at least one light source (e.g., the plurality of light sources 704) in a manner that enables the luminous flux to vary within a scan of the field of view using light from the at least one light source. In some embodiments, the scan of the field of view can include illuminating the first section, the second section, the third section, and the fourth section in the following order: (1) illuminating the first section but not the second section, the third section, and the fourth section; (2) illuminating the third section but not the first section, the second section, and the fourth section; (3) illuminating the second section but not the first section, the third section, and the fourth section; and (4) illuminating the fourth section but not the first section, the second section, and the third section. For example, the processing unit 714 can be programmed to control the light sources 704 to illuminate the four sections in the following order: (1) illuminating the section 921 but not the section 922, the section 931, and the section 932; (2) illuminating the section 931 but not the section 921, the section 922, and the section 932; (3) illuminating the section 922 but not the section 921, the section 931, and the section 932; (4) illuminating the section 932 but not the section 921, the section 922, and the section 931.

[0171] In some embodiments, the illumination level of the illumination delivered to each of the first section, the second section, the third section, and the fourth section (e.g., during a scan period or during a frame) is below a threshold. In some embodiments, the threshold can be an illumination level associated with a standard maximum permissible exposure (MPE), which can be the highest power or energy density (in W / cm2or J / cm2) of light directed to a human that is considered safe, i.e., the likelihood of causing damage can be negligible. For example, the standard MPE can satisfy the requirements of Class 1 eye safety (e.g., according to the International Electrotechnical Commission (IEC) standard 60825-1). In some embodiments, the total illumination level of the illumination delivered to each of the first section and the second section (e.g., during a scan period or during a frame) exceeds the threshold.

[0172] In some embodiments, the processing unit 714 can be programmed to control the light source 704 to sequentially illuminate the first subsection of the first portion 920 and the third subsection of the second portion 930 (but not other subsections of the first portion 920 and the second portion 930) in each of the multiple scans. For example, the processing unit 714 can be programmed to control the light source 704 to sequentially illuminate the subsection 921 and the subsection 931 in the multiple scans (e.g., a first scan cycle, a second scan cycle, etc.). The sensing unit 710 can be configured to receive reflections of light from the environment of the light source in each scan. The processing unit 714 can be programmed to construct a point cloud output 931 based in part on the reflections summed from the multiple scans of the non-contiguous subsections included in the first set of non-contiguous subsections, including the subsection 921 and the subsection 822. In some embodiments, the processing unit 714 can also be programmed to control the light source 704 to sequentially illuminate the second subsection of the first portion 920 and the fourth subsection of the second portion 930 (but not other subsections of the first portion 920 and the second portion 930) in each of the multiple scans. The sensing unit 710 can be configured to receive reflections of light from the environment of the light source in each scan. The processing unit 714 can be programmed to construct a point cloud output based in part on the reflections summed from the multiple scans of the first subsection, the second subsection, the third subsection, and the fourth subsection.

[0173] In some embodiments, the first portion 920 and the second portion 930 can have the same size and / or shape. Alternatively or additionally, the first portion 920 and the second portion 930 can have different sizes and / or shapes. Alternatively or additionally, the first subsection and the second subsection of the first portion 920 can have the same size and / or shape. Alternatively or additionally, the first subsection and the second subsection of the first portion 920 can have different sizes and / or shapes. Alternatively or additionally, the first subsection of the first portion 920 and the third subsection of the second portion 930 can have the same size and / or shape. Alternatively or additionally, the first subsection of the first portion 920 and the third subsection of the second portion 930 can have different sizes and / or shapes.

[0174] In some embodiments, the first subsection (e.g., the subsection 921) of the first portion 920 and the third subsection (e.g., the subsection 931) of the second portion 930 can be illuminated during a first scan cycle, and the second subsection (e.g., the subsection 922) of the first portion 920 and the fourth subsection (e.g., the subsection 932) of the second portion 930 can be illuminated during a second scan cycle. In some embodiments, the subsection 921 and the subsection 931 can not be illuminated during the second scan cycle. Alternatively or additionally, the subsection 922 and the subsection 932 can not be illuminated during the first scan cycle.

[0175] In some embodiments, the processing unit 714 can be programmed to control the light source 704 to illuminate at least one subsection of a portion of the field of view during a plurality of scan cycles in a frame. For example, the processing unit 714 can be programmed to control the light source 704 to illuminate the subsection 921 during a plurality of scan cycles in a frame. The illumination directed to the subsection 921 during each of the plurality of scan cycles can be less than an illumination level associated with the predetermined threshold. In some embodiments, the total illumination of the illumination directed to the subsection 921 during each of the plurality of scan cycles can be greater than the illumination level associated with the predetermined threshold. For example, the frame can include five scan cycles, and the illumination directed to the subsection 921 during each of the five scan cycles during the frame can be less than the illumination level associated with the predetermined threshold, and the total illumination of the illumination directed to the subsection 921 during each of the five scan cycles can be greater than the illumination level associated with the predetermined threshold.

[0176] In some embodiments, the processing unit 714 can set an angular distance between a field of view (FOV) angle corresponding to a first portion of the field of view and a FOV angle corresponding to a second portion of the field of view such that the angular distance can be greater than an angular dimension corresponding to a diameter of a human pupil (e.g., 4 mm, 5 mm, 6 mm, 7 mm, 8 mm, etc.) at a predetermined minimum safety distance (e.g., 10 cm, 25 cm, 50 cm, 1 m, etc.) from the lidar system.

[0177] FIG. 8 FIG. 10 is a flowchart illustrating an example process 1000 for detecting objects in an environment of a lidar system, consistent with the disclosed embodiments. One or more steps of the process 1000 can be performed by the lidar system 700 via one or more components thereof (e.g., the processing unit 714).

[0178] The processing unit 714 can be programmed to control at least one light source (e.g., the light source 704) to enable a variation in luminous flux within a scan of a field of view (e.g., the field of view 800) using light from the at least one light source. The field of view 800 can be divided into a plurality of segments. For example, the field of view 800 can be divided into a plurality of segments, including the segment 821, the segment 822, the segment 831, the segment 832, the segment 841, and the segment 842. The field of view 800 can include a first set of non-contiguous segments. Each of the non-contiguous segments included in the first set can be separated by at least one segment from other non-contiguous segments in the first set. For example, the first set of non-contiguous segments can include the segment 821 and the segment 822. The segment 821 can be separated by three segments from the segment 822.

[0179] At step 1001, the processing unit 714 can be programmed to control the light source 704 to sequentially illuminate non-contiguous segments included in a first set of non-contiguous segments. During illumination of a particular non-contiguous segment in the first set of non-contiguous segments, other segments in the plurality of segments can not be illuminated. In some embodiments, other segments in the plurality of segments can not be illuminated between illumination of non-contiguous segments in the first set of non-contiguous segments. For example, the processing unit 714 can be programmed to control the light source to illuminate segment 821 without illuminating other segments in the plurality of segments. The processing unit 714 can be programmed to control the light source to subsequently illuminate segment 822, and other segments in the plurality of segments (including segments between segment 821 and segment 822, such as segments 831, 841, 832, 842) can not be illuminated between illumination of segment 821 and segment 822. Each illumination directed to a non-contiguous segment in the first set of non-contiguous segments can not exceed a predetermined threshold. In some embodiments, the predetermined threshold can be an illumination level associated with a standard maximum permissible exposure (MPE), which can be the highest power or energy density (in W / cm2or J / cm2) of light directed to a human that is considered safe, i.e., the likelihood of causing damage can be negligible. For example, the standard maximum permissible exposure (MPE) can satisfy the requirements of Class 1 eye safety (e.g., according to the International Electrotechnical Commission (IEC) standard 60825-1).

[0180] In some embodiments, the plurality of segments of the field of view can include a second set of non-contiguous segments different from the first set of non-contiguous segments. For example, as shown in FIG. 8, the field of view 800 can include a second set of non-contiguous segments including segment 831 and segment 832. Each of the non-contiguous segments included in the second set is separated from other non-contiguous segments in the second set by at least one segment. For example, segments 831 and 832 can be separated by three segments. FIG. 11

[0181] ​At 1003, the processing unit 714 can be programmed to control the light source to sequentially illuminate non-contiguous segments included in a second set of non-contiguous segments after the sequential illumination of the first set of non-contiguous segments. For example, after the sequential illumination of the first set of non-contiguous segments (e.g., segment 821 and segment 822), the processing unit 714 can be programmed to control the light source to sequentially illuminate segment 831 segment 832. During the illumination of a particular non-contiguous segment in the second set of non-contiguous segments, other segments in the plurality of segments can not be illuminated. In some embodiments, other segments in the plurality of segments can not be illuminated between the illumination of non-contiguous segments in the second set of non-contiguous segments. For example, the processing unit 714 can be programmed to control the light source to illuminate segment 831 without illuminating other segments in the plurality of segments. The processing unit 714 can be programmed to control the light source to subsequently illuminate segment 832, and other segments in the plurality of segments (including 821, 841, 822, 842) can not be illuminated between the illumination of segment 831 and segment 832. Each illumination directed to a non-contiguous segment in the second set of non-contiguous segments can not exceed a predetermined threshold. The predetermined threshold can be an illumination level associated with a standard maximum permissible exposure (MPE), such as an illumination level that satisfies a Class 1 eye safety requirement (e.g., according to International Electrotechnical Commission (IEC) standard 60825-1).

[0182] At step 1005, the processing unit 714 can be programmed to detect an object within the field of view based on reflections from the field of view received by the at least one sensor (e.g., the sensing unit 710). For example, the sensing unit 710 can be configured to receive reflections of light from the environment of the lidar system 700. The processing unit 714 can be programmed to detect an object within the field of view based on the reflections from the field of view received by the sensing unit 710.

[0183] FIG. 12 FIG. 11 is a flowchart illustrating an example process 1100 for detecting an object in an environment of a lidar system, consistent with the disclosed embodiments. One or more steps of the process 1100 can be performed by the lidar system 700 via one or more components thereof (e.g., the processing unit 714).

[0184] The processing unit 714 can be programmed to control the at least one light source (e.g., the light source 704) to enable a variation in luminous flux within a scan of a field of view (e.g., the field of view 900) using light from the at least one light source. For example, the field of view 900 can be divided into a plurality of portions, each of which can include a plurality of subsections. For example, the field of view 900 can include a first portion 920 and a second portion 930. The first portion can include a first subsection and a second subsection different from the first subsection, and the second portion can include a third subsection and a fourth subsection different from the third subsection. For example, the first portion 920 can include a subsection 921 and a subsection 922 (and, in some embodiments, a subsection 923). The second portion 930 can include a subsection 931 and a subsection 932 (and, in some embodiments, a subsection 933).

[0185] At step 1101, the processing unit 714 can be programmed to control the light source 704 to illuminate the first subsection but not the second, third, and fourth subsections. For example, the processing unit 714 can be programmed to control the light source 704 to illuminate the subsection 921 but not the subsection 922, the subsection 931, and the subsection 932.

[0186] At step 1103, the processing unit 714 can be programmed to control the light source 704 to illuminate the third subsection but not the first, second, and fourth subsections. For example, the processing unit 714 can be programmed to control the light source 704 to illuminate the subsection 931 but not the subsection 921, the subsection 922.

[0187] At step 1105, the processing unit 714 can be programmed to control the light source 704 to illuminate the second subsection but not the first, third, and fourth subsections. For example, the processing unit 714 can be programmed to control the light source 704 to illuminate the subsection 931 but not the subsection 921, the subsection 922.

[0188] At step 1107, the processing unit 714 can be programmed to control the light source 704 to illuminate the fourth subsection but not the first, second, and third subsections. For example, the processing unit 714 can be programmed to control the light source 704 to illuminate the subsection 931 but not the subsection 921, the subsection 922.

[0189] In some embodiments, (e.g., during a scan cycle or during a frame) the illumination level delivered to each of the first, second, third, and fourth portions is below a threshold. In some embodiments, the threshold may be an illumination level associated with a standard maximum permissible exposure (MPE), which may be the highest power or energy density (in W / cm² or J / cm²) of light directed to a human being considered safe, i.e., the possibility of causing damage is negligible. For example, a standard MPE may meet Class 1 eye safety requirements (e.g., according to International Electrotechnical Commission (IEC) standard 60825-1). In some embodiments, (e.g., during a scan cycle or during a frame) the total illumination level delivered to each of the first and second portions exceeds the threshold.

[0190] At step 1109, processing unit 714 can be programmed to detect objects within the field of view based on reflections from the field of view received by at least one sensor (e.g., sensing unit 710). For example, sensing unit 710 can be configured to receive reflections of light from the environment of lidar system 700. Processing unit 714 can be programmed to detect objects within the field of view based on reflections from the field of view received by sensing unit 710.

[0191] FIG. 8 This is a flowchart illustrating an exemplary process 1200 for detecting objects in an environment consistent with the disclosed embodiments of a lidar system. One or more steps of process 1100 may be performed by the lidar system 700 via one or more of its components (e.g., processing unit 714).

[0192] Processing unit 714 can be programmed to control at least one light source (e.g., light source 704) such that the luminous flux can vary within a scan of the field of view (e.g., field of view 800) using light from the at least one light source. The field of view may include multiple discontinuous segments, each of which may be discontinuous with each other and may not overlap. For example, as FIG. 8 As shown, the field of view 800 may include a first group of discontinuous segments, including segment 821 and segment 822. Each of the discontinuous segments included in the first group may be separated from the other discontinuous segments in the first group by at least one segment. For example, segment 821 may be separated from segment 822 by three segments. The processing unit 714 may be programmed to control the light source to sequentially illuminate the discontinuous segments included in the first group of discontinuous segments according to steps 1201 and 1203.

[0193] At 1201, processing unit 714 can be programmed to control light source 704 to illuminate a first discontinuous segment of a plurality of discontinuous segments without illuminating any other portion of the field of view. For example, processing unit 714 can be programmed to control light source 704 to illuminateFIG. 8 The illustrated segment 821 is illuminated without illuminating any other portion of the field of view.

[0194] In some embodiments, other segments of the plurality of segments can not be illuminated during illumination of a particular non-contiguous segment of the first set of non-contiguous segments. In some embodiments, other segments of the plurality of segments can not be illuminated between illumination of non-contiguous segments of the first set of non-contiguous segments. For example, processing unit 714 can be programmed to control the light source to illuminate segment 821 without illuminating other segments of the plurality of segments. Processing unit 714 can be programmed to subsequently control the light source to illuminate segment 822, and other segments of the plurality of segments (including segments between segment 821 and segment 822, such as segments 831, 841, 832, and 842) can not be illuminated between illumination of segment 821 and segment 822. Each illumination directed to a non-contiguous segment of the first set of non-contiguous segments can not exceed a predetermined threshold. In some embodiments, the predetermined threshold can be an illumination level associated with a standard maximum permissible exposure (MPE), which can be the highest power or energy density (in W / cm2or J / cm2) of light directed to a human that is considered safe, i.e., the likelihood of causing damage can be negligible. For example, the standard maximum permissible exposure (MPE) can satisfy the requirements of Class 1 eye safety (e.g., according to the International Electrotechnical Commission (IEC) standard 60825-1).

[0195] At 1203, after illuminating a first non-contiguous segment (e.g., segment 821) of the plurality of non-contiguous segments and before illuminating any other portion of the field of view, processing unit 714 can be programmed to control the light source 704 to illuminate a second non-contiguous segment (e.g., segment 822) of the plurality of non-contiguous segments without illuminating any other portion of the field of view.

[0196] In some embodiments, the plurality of segments of the field of view 800 can include a second set of non-contiguous segments that are different from the first set of non-contiguous segments. For example, as illustrated in FIG. 8B, the plurality of segments of the field of view 800 can include a second set of non-contiguous segments 850, 851, 852, and 853. In some embodiments, the second set of non-contiguous segments 850, 851, 852, and 853 can be illuminated in a manner that is different from the first set of non-contiguous segments 820, 821, 822, 823, 824, 825, 826, 827, 828, and 829. For example, the second set of non-contiguous segments 850, 851, 852, and 853 can be illuminated in a manner that is different from the first set of non-contiguous segments 820, 821, 822, 823, 824, 825, 826, 827, 828, and 829. FIG. 8As shown, the field of view 800 can include a second set of non-contiguous segments, including segment 831 and segment 832. Each of the non-contiguous segments included in the second set is separated from other non-contiguous segments in the second set by at least one segment. For example, segment 831 and 832 can be separated by three segments. The processing unit 714 can also be programmed to control the light source to sequentially illuminate the non-contiguous segments included in the second set of non-contiguous segments after the sequential illumination of the first set of non-contiguous segments. For example, after the sequential illumination of the first set of non-contiguous segments (e.g., segment 821 and segment 822), the processing unit 714 can be programmed to control the light source to sequentially illuminate segment 831 segment 832. During the illumination of a particular non-contiguous segment in the second set of non-contiguous segments, other segments in the plurality of segments can not be illuminated. In some embodiments, other segments in the plurality of segments can not be illuminated between the illumination of the non-contiguous segments in the second set of non-contiguous segments. For example, the processing unit 714 can be programmed to control the light source to illuminate segment 831 without illuminating other segments in the plurality of segments. The processing unit 714 can be programmed to control the light source to subsequently illuminate segment 832, and other segments in the plurality of segments (including 821, 841, 822, 842) can not be illuminated between the illumination of segment 831 and segment 832. Each illumination directed to a non-contiguous segment in the second set of non-contiguous segments can not exceed a predetermined threshold. The predetermined threshold can be an illumination level associated with a standard maximum permissible exposure (MPE), such as an illumination level that satisfies a Class 1 eye safety requirement (e.g., according to International Electrotechnical Commission (IEC) standard 60825-1).

[0197] In some embodiments, while illumination of a single segment (e.g., during a scan period or during a frame) can not exceed the predetermined threshold, the total illumination of the illumination of a particular segment and the illumination of a segment adjacent to the particular segment (e.g., during a scan period or during a frame) can exceed the predetermined threshold. Alternatively, the total illumination of the illumination of three (or more) adjacent segments (e.g., during a scan period or during a frame) can exceed the predetermined threshold, while the total illumination of any subset of the illumination of the three (or more) adjacent segments (e.g., during a scan period or during a frame) can not exceed the predetermined threshold. For example, segment 821 and segment 831 can be adjacent to each other. Neither the illumination of segment 821 nor the illumination of segment 831 can exceed the predetermined threshold during a scan period, while the total illumination of the illumination of segment 821 and the illumination of segment 831 can exceed the predetermined threshold.

[0198] In some embodiments, a non-continuous segment included in the second group of non-continuous segments may include a segment adjacent to a first non-continuous segment included in the first group of non-continuous segments. For example, a non-continuous segment included in the second group of non-continuous segments may include segment 831 adjacent to segment 821 (which may be one of the non-continuous segments included in the first group of non-continuous segments), such as... FIG. 8 As shown. As another example, the non-contiguous segments included in the second set of non-contiguous segments may include segment 832 adjacent to segment 822 (which may be one of the non-contiguous segments included in the first set of non-contiguous segments). In some embodiments, the illumination directed to the first non-contiguous segment included in the first set of non-contiguous segments and the segment adjacent to the first non-contiguous segment included in the first set of non-contiguous segments is less than the illumination level associated with a predetermined threshold. For example, during a scan cycle, the illumination of both segment 821 and segment 831 may not exceed the predetermined threshold, while the total illumination of segment 821 and segment 831 may exceed the predetermined threshold. In some embodiments, the total illumination directed to the first non-contiguous segment included in the first set of non-contiguous segments and the segment adjacent to the first non-contiguous segment included in the first set of non-contiguous segments is greater than the illumination level associated with a predetermined threshold. For example, the total illumination of segment 821 and segment 831 during a scan cycle may exceed the predetermined threshold.

[0199] In some embodiments, sequentially illuminating the non-contiguous segments included in the first set of non-contiguous segments can include sequentially illuminating the non-contiguous segments included in the first set of non-contiguous segments in each of a plurality of scans. For example, the processing unit 714 can be programmed to control the light source 704 to sequentially illuminate the segment 821 and the segment 822 in a plurality of scans (e.g., a first scan cycle, a second scan cycle, etc.). The sensing unit 710 can be configured to receive reflections of light from the environment from the light source in each scan. The processing unit 714 can be programmed to construct a point cloud output 822 including the segment 821 and the segment 822 based in part on the reflections summed from the plurality of scans of the non-contiguous segments included in the first set of non-contiguous segments. In some embodiments, as described above, the plurality of segments of the field of view 800 can include a second set of non-contiguous segments different from the first set of non-contiguous segments. The processing unit 714 can also be programmed to control the light source 704 to sequentially illuminate the non-contiguous segments included in the second set of non-contiguous segments (e.g., the segment 831, the segment 832) in each of the plurality of scans. The sensing unit 710 can be configured to receive reflections of light from the environment from the light source in each scan. The processing unit 714 can be programmed to construct a point cloud output 832 including the segment 821, the segment 822, the segment 831, and the segment 832 based in part on the reflections summed from the plurality of scans of the non-contiguous segments included in the first set of non-contiguous segments and the reflections summed from the plurality of scans of the non-contiguous segments included in the second set of non-contiguous segments.

[0200] In some embodiments, a first non-contiguous segment included in the first set of non-contiguous segments and a segment adjacent to the first non-contiguous segment included in the first set of non-contiguous segments have a same size. For example, as shown in FIG. 8A, the segment 821 (one of the non-contiguous segments included in the first set of non-contiguous segments) is adjacent to the segment 831 (one of the non-contiguous segments included in the second set of non-contiguous segments). The segment 821 and the segment 831 can have a same size. Alternatively, a first non-contiguous segment included in the first set of non-contiguous segments and a segment adjacent to the first non-contiguous segment included in the first set of non-contiguous segments have different sizes. Additionally, in some embodiments, a first non-contiguous segment included in the first set of non-contiguous segments and a segment adjacent to the first non-contiguous segment included in the first set of non-contiguous segments can have a same shape or different shapes. ​

[0201] ​In some embodiments, a first non-contiguous segment of the non-contiguous segments included in the first group of non-contiguous segments can be illuminated during a first scan period, and a segment adjacent to the first non-contiguous segment of the non-contiguous segments included in the first group of non-contiguous segments is illuminated during a second scan period. For example, as shown in FIG. 8, segment 821 (one of the non-contiguous segments included in the first group of non-contiguous segments) can be illuminated during a first scan period, and segment 831 (one of the non-contiguous segments included in the second group of non-contiguous segments) can be illuminated during a second scan period. In some embodiments, segment 821 can not be illuminated during the second scan period. Alternatively or additionally, segment 831 can not be illuminated during the first scan period. ​

[0202] In some embodiments, a first non-contiguous segment of the non-contiguous segments included in the first group of non-contiguous segments can be illuminated during a first scan period, and a second non-contiguous segment of the non-contiguous segments included in the second group of non-contiguous segments can be illuminated during a second scan period. For example, segment 821 and segment 822 (the non-contiguous segments included in the first group of non-contiguous segments) can be illuminated during a first scan period. Segment 831 and segment 832 (the non-contiguous segments included in the second group of non-contiguous segments) can be illuminated during a second scan period. In some embodiments, segment 821 and segment 822 can not be illuminated during the second scan period. Alternatively or additionally, segment 831 and segment 832 can not be illuminated during the first scan period.

[0203] In some embodiments, processing unit 714 can be programmed to control light source 704 to illuminate at least one of the non-contiguous segments included in the first group of non-contiguous segments during a plurality of scan periods in a frame. For example, processing unit 714 can be programmed to control light source 704 to illuminate segment 821 (one of the non-contiguous segments included in the first group of non-contiguous segments) during a plurality of scan periods in a frame. The illumination directed to segment 821 during each of the plurality of scan periods can be less than an illumination level associated with a predetermined threshold. In some embodiments, a total illumination of the illumination directed to at least one of the non-contiguous segments included in the first group of non-contiguous segments during each of the plurality of scan periods is greater than the illumination level associated with the predetermined threshold. For example, a frame can include five scan periods, and the illumination directed to segment 821 during each of the five scan periods during the frame can be less than the illumination level associated with the predetermined threshold, and a total illumination of the illumination directed to segment 821 during each of the five scan periods can be greater than the illumination level associated with the predetermined threshold.

[0204] ​At 1205, the processing unit 714 can be programmed to detect an object within a field of view based on reflections from the field of view received by at least one sensor (e.g., the sensing unit 710). For example, the sensing unit 710 can be configured to receive reflections of light from the environment of the lidar system 700. The processing unit 714 can be programmed to detect an object within the field of view based on the reflections from the field of view received by the sensing unit 710.

[0205] The foregoing description has been presented for purposes of illustration. It is not exhaustive and is not limited to the precise forms or embodiments disclosed. Modifications and adaptations will be apparent to those skilled in the art from consideration of the specification and practice of the disclosed embodiments. Additionally, while the aspects of the disclosed embodiments have been described as stored in memory, a skilled artisan will appreciate that these aspects can also be stored on other types of computer- readable media, such as secondary storage devices, e.g., hard disks or CD ROMs, or other forms of RAM or ROM, USB media, DVD, Blu-ray, or other optical media.

[0206] A computer program based on the written description and the disclosed methods is within the skill of an experienced developer. The various program or program modules can be created using any of the techniques of those skilled in the art of computer programming, or combinations thereof, or designed in conjunction with existing software. For example, program portions or program modules can be designed or designed with the aid of the.Net Framework, the.Net Compact Framework (and related languages, such as Visual Basic, C, and so on), Java, C++, Objective-C, HTML, HTML / AJAX combinations, XML, or HTML including Java applets.

[0207] Moreover, while illustrative embodiments have been described herein, the scope of any and all embodiments having equivalent elements, modifications, omissions, combinations (e.g., of aspects across various embodiments), adaptations and / or alterations as would be appreciated by those skilled in the art based on the disclosure herein are encompassed by the scope of the claims. The limitations in the claims are based on the language read, and it is intended that the scope of the claims follow the limitations as interpreted by the arbitrary expert, and not necessarily the ordinary meanings of terms. These examples should be interpreted as non-exhaustive, merely stating more fully the generic scope of terms. Moreover, the steps of the disclosed methods can be modified in any way, including by reordering steps and / or inserting or deleting steps, without departing from the scope of the methods. It is intended, therefore, to be true that the specification and examples are to be considered illustrative only and that the true scope and spirit of the disclosure are indicated by the full scope of the following claims and their equivalents.

Claims

1. An electro-optical system comprising: At least one processor, said at least one processor being programmed to: Control at least one light source such that the luminous flux can vary within a scan of the field of view using light from the at least one light source, wherein: The field of view is divided into a plurality of non-overlapping segments, each of which has a size sufficient to cover the size of a unit light beam directed into the field of view. The plurality of segments includes a first group of discontinuous segments and a second group of discontinuous segments distinct from those in the first group. The second group includes segments in which each of the discontinuous segments in the first group is separated from any other discontinuous segment in the first group by at least one segment, and segments in the second group are also separated from any other discontinuous segment in the second group by at least one segment. in The scanning of the field of view includes: Sequential illumination includes the discontinuous segments within the first group of discontinuous segments, comprising sequential illumination of the discontinuous segments within the first group of discontinuous segments such that during the illumination of a particular discontinuous segment within the first group of discontinuous segments, other segments among the plurality of segments not included in the first group of discontinuous segments are not illuminated, and The other segments among the plurality of segments not included in the first group of discontinuous segments are not illuminated between the illuminations of the discontinuous segments in the first group of discontinuous segments, and Following the sequential illumination of the first set of discontinuous segments, the scanning of the field of view further includes the sequential illumination of the discontinuous segments included in the second set of discontinuous segments.

2. The electro-optical system as claimed in claim 1, wherein: The sequential illumination of the non-continuous segments included in the first group of non-continuous segments includes sequential illumination of the non-continuous segments included in the first group of non-continuous segments in each of multiple scans; and The at least one processor is further programmed to construct a point cloud output in part based on reflections summed from multiple scans of the discontinuous segments included in the first set of discontinuous segments.

3. The electro-optical system as described in claim 1, wherein: The non-continuous segments included in the first group of non-continuous segments are illuminated sequentially in each of multiple scans; The sequential illumination of the non-continuous segments included in the second group of non-continuous segments includes repeatedly and sequentially illuminating the non-continuous segments included in the second group of non-continuous segments in each of multiple scans; and The at least one processor is further programmed to construct a point cloud output in part based on reflections from the sum of multiple scans of the non-continuous segments included in the first group of non-continuous segments and reflections from the sum of multiple scans of the non-continuous segments included in the second group of non-continuous segments.

4. The electro-optical system as claimed in claim 1, wherein: The non-continuous segments included in the second group of non-continuous segments include segments adjacent to the first non-continuous segment included in the first group of non-continuous segments.

5. The electro-optical system of claim 4, wherein each of the illumination directed to the first discontinuous segment included in the first set of discontinuous segments and the segment adjacent to the first discontinuous segment included in the first set of discontinuous segments is less than an illumination level associated with a predetermined threshold.

6. The electro-optical system of claim 5, wherein the total illumination directed to the first discontinuous segment included in the first set of discontinuous segments and the segment adjacent to the first discontinuous segment included in the first set of discontinuous segments is greater than the illumination level associated with the predetermined threshold.

7. The electro-optic system of claim 4, comprising the first discontinuous segment in the first set of discontinuous segments and the segment adjacent to the first discontinuous segment included in the first set of discontinuous segments having the same size.

8. The electro-optic system of claim 4, comprising the first discontinuous segment in the first set of discontinuous segments and the segment adjacent to the first discontinuous segment included in the first set of discontinuous segments having different sizes.

9. The electro-optic system of claim 4, comprising the first discontinuous segment in the first set of discontinuous segments and the segment adjacent to the first discontinuous segment included in the first set of discontinuous segments having the same shape.

10. The electro-optic system of claim 4, comprising the first discontinuous segment in the first set of discontinuous segments and the segment adjacent to the first discontinuous segment included in the first set of discontinuous segments having a different shape.

11. The electro-optical system of claim 4, wherein: The first discontinuous segment, included in the first group of discontinuous segments, is illuminated during the first scan cycle; and The segment adjacent to the first non-continuous segment included in the first group of non-continuous segments is illuminated during the second scan cycle.

12. The electro-optical system as claimed in claim 1, wherein: The discontinuous segments included in the first group of discontinuous segments are illuminated during the first scan cycle; and The discontinuous segments included in the second group of discontinuous segments are illuminated during the second scan cycle.

13. The electro-optical system of claim 1, wherein the at least one processor is further programmed to detect objects within the field of view based on reflections from the field of view received by at least one sensor.

14. The electro-optical system of claim 13, wherein the at least one sensor comprises a detector array.

15. The electro-optic system of claim 14, wherein the detector array comprises a focal plane detector array.

16. The electro-optic system of claim 1, further comprising a light deflector configured to deflect light from the at least one light source into the field of view.

17. The electro-optic system of claim 16, wherein the optical deflector comprises a microelectromechanical system (MEMS) mirror.

18. The electro-optic system of claim 16, wherein the optical deflector comprises a rotating prism.

19. The electro-optic system of claim 16, wherein the optical deflector comprises an optical phased array controller.

20. The electro-optic system of claim 16, wherein the optical deflector comprises a vertical-cavity surface-emitting laser (VCSEL) array controller.

21. The electro-optic system of claim 16, wherein the optical deflector comprises a scanning mirror.

22. The electro-optical system of claim 1, further comprising a light emitting component, the light emitting component including the at least one light source.

23. The electro-optical system of claim 22, wherein the at least one processor is further programmed to cause the light emitting component to scan the field of view multiple times during a frame.

24. The electro-optical system of claim 23, wherein the at least one processor is further programmed to cause the light emitting component to scan the field of view more than 10 times during a frame.

25. The electro-optic system of claim 22, wherein the light emitting component includes a spatial light modulator configured to modulate the light flux to vary within the scan of the field of view.

26. The electro-optic system of claim 1, wherein the scanning of the field of view further comprises: Illumination during multiple scan cycles in a frame includes at least one non-continuous segment in the first set of non-continuous segments, wherein the illumination directed to the at least one non-continuous segment in the first set of non-continuous segments during each of the multiple scan cycles is less than the illumination level associated with a predetermined threshold.

27. The electro-optical system of claim 26, wherein the total illumination directed to at least one discontinuous segment included in the first set of discontinuous segments during each of the plurality of scanning cycles is greater than the illumination level associated with the predetermined threshold.

28. A method for controlling an electro-optical system, comprising: Control at least one light source such that the luminous flux can vary within a scan of the field of view using light from the at least one light source, wherein: The field of view is divided into a plurality of non-overlapping segments, each of which has a size sufficient to cover the size of a unit light beam guided into the field of view. The plurality of segments includes a first group of discontinuous segments and a second group of discontinuous segments different from those in the first group. Each of the discontinuous segments in the first group is separated from the other discontinuous segments in the first group by at least one segment, and each of the discontinuous segments in the second group is separated from the other discontinuous segments in the second group by at least one segment. The scanning of the field of view includes: Sequential illumination includes the discontinuous segments within the first group of discontinuous segments, comprising sequential illumination of the discontinuous segments within the first group of discontinuous segments such that during the illumination of a particular discontinuous segment within the first group of discontinuous segments, other segments among the plurality of segments not included in the first group of discontinuous segments are not illuminated, and The other segments among the plurality of segments not included in the first group of discontinuous segments are not illuminated between the illuminations of the discontinuous segments in the first group of discontinuous segments, and Following the sequential illumination of the first set of discontinuous segments, the scanning of the field of view further includes the sequential illumination of the discontinuous segments included in the second set of discontinuous segments.

29. An electro-optical system comprising: At least one processor, said at least one processor being programmed to: Control at least one light source such that the luminous flux can vary within a scan of the field of view using light from the at least one light source, wherein: The field of view includes a first part and a second part that is different from the first part; The first part includes a first portion and a second portion that is different from the first portion and does not overlap with the first portion; The second part includes a third portion and a fourth portion different from the third portion, wherein the third portion does not overlap with the second portion or the fourth portion, and wherein each of the first portion, the second portion, the third portion and the fourth portion has a size sufficient to cover the size of a unit light guided into the field of view; The scanning of the field of view includes illuminating the first portion, the second portion, the third portion, and the fourth portion in the following order: The first section is illuminated, but the second, third, and fourth sections are not illuminated. Illuminate the third section, but not the first section, the second section, and the fourth section; Illuminate the second section, but not the first section, the third section, and the fourth section; and Illuminate the fourth section, but not the first section, the second section, and the third section; The illumination levels delivered to each of the first, second, third, and fourth divisions are below a threshold; and The total illumination level delivered to each of the first and second divisions exceeds the threshold.

30. An electro-optical system comprising: At least one processor, said at least one processor being programmed to: Control at least one light source such that the luminous flux can vary within a scan of the field of view using light from the at least one light source, wherein: The field of view includes multiple non-continuous segments; Each of the plurality of discontinuous segments is discontinuous and does not overlap with each other; and The scanning of the field of view includes: Illuminate the first discontinuous segment of the plurality of discontinuous segments without illuminating any other portion of the field of view; and After illuminating the first discontinuous segment of the plurality of discontinuous segments and before illuminating any other part of the field of view, illuminating the second discontinuous segment of the plurality of discontinuous segments without illuminating any other part of the field of view, wherein each of the discontinuous segments has a size sufficient to cover the size of a unit light directed into the field of view.

Images