Non-uniform distribution illumination for depth sensors

Abstract

A depth sensor is provided. The depth sensor includes: one or more light sources configured to provide a plurality of light beams; and one or more optical structures, the one or more optical structures coupled to the one or more light sources. The one or more optical structures are configured to receive the plurality of light beams. At least one of the one or more light sources or the one or more optical structures is configured to non-uniformly distribute the plurality of light beams in a vertical field of view (FOV) such that the vertical FOV includes a dense region and a sparse region. The dense region of the vertical FOV has a higher beam density than the sparse region of the vertical FOV, and the depth sensor does not include a mechanically movable part configured to scan light.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims priority to U.S. Patent Application No. 18 / 389,406, filed on November 14, 2023, entitled "Non - Uniform Distributed Illumination for Depth Sensors", and U.S. Provisional Patent Application No. 63 / 425,644, filed on November 15, 2022, entitled "Non - Uniform Distributed Illumination for Depth Sensors". For all purposes, the contents of both applications are hereby incorporated by reference in their entireties. Technical Field

[0003] The present disclosure generally relates to a depth sensor and, more particularly, to non - uniform distributed illumination for a depth sensor. Background Art

[0004] Light Detection and Ranging (LiDAR) systems use light pulses to create an image or a point cloud of the external environment. LiDAR systems can be scanning or non - scanning systems. Some typical scanning LiDAR systems include a light source, a light emitter, a light steering system, and a light detector. The light source generates a light beam that, when emitted from the LiDAR system, is directed by the light steering system in a specific direction. When the emitted light beam is scattered or reflected by an object, a portion of the scattered or reflected light returns to the LiDAR system to form a return light pulse. The light detector detects the return light pulse. Using the difference between the time when the return light pulse is detected and the time when the corresponding light pulse in the light beam is emitted, the LiDAR system can determine the distance to the object based on the speed of light. This technique for determining distance is called Time - of - Flight (ToF) technique. The light steering system can direct the light beam along different paths to allow the LiDAR system to scan the surrounding environment and generate an image or a point cloud. Typical non - scanning LiDAR systems illuminate the entire Field of View (FOV) instead of scanning the entire FOV.

[0005] An example of a non - scanning LiDAR system is Flash LiDAR, which can also use the ToF technique to measure the distance to an object. LiDAR systems can also use techniques other than time - of - flight and scanning to measure the surrounding environment. Summary of the Invention

[0006] A depth sensor, also known as a depth camera or 3D sensor, is a device that can capture the spatial information of objects in its field of view. These sensors are designed to measure the distance from the sensor to different points in the environment, thereby creating a three-dimensional representation of the scene. Depth sensors can use the direct time-of-flight (dToF) method to measure distance (also known as depth), and are thus dToF sensors. Depth sensors can also be indirect time-of-flight (iToF) sensors that use the indirect time-of-flight (iToF) method to measure distance. A solid-state depth sensor is a type of depth sensor that can output three-dimensional (3D) depth measurements of the external environment without any mechanically movable parts inside the sensor. For example, it can be a flash LiDAR that can use vertical-cavity surface-emitting lasers (VCSELs) as light sources and single-photon avalanche diode (SPAD) arrays as light detectors. The absence of mechanically movable parts is an advantage of solid-state depth sensors. When a solid-state depth sensor operates, the laser source emits laser light into the field of view, and the light detector captures the reflected or scattered light (also known as the return light) from the object. In the following disclosure, depth sensors, flash LiDARs, and iToF sensors may also be referred to as LiDARs. Flash LiDAR is also known as a dToF sensor. This disclosure thus uses LiDAR as an example of a depth sensor. However, it should be understood that depth sensors can be ToF sensors, structured light sensors (e.g., using known light patterns to measure depth based on light distortion), stereovision sensors (e.g., using two or more cameras to measure depth), or LiDAR systems.

[0007] This disclosure provides a novel method for optimizing the emission light distribution of a solid-state depth sensor. Using this novel method, the detection range distribution of the depth sensor can be optimized, and energy consumption can be reduced.

[0008] In one embodiment, a depth sensor is provided. The depth sensor includes: one or more light sources configured to provide a plurality of light beams; and one or more optical structures coupled to the one or more light sources. The one or more optical structures are configured to

[0009] receive the plurality of light beams. At least one of the one or more light sources or the one or more optical structures is configured to unevenly distribute the plurality of light beams in a vertical field of view (FOV) such that the vertical FOV includes a dense region and a sparse region. The dense region of the vertical FOV has a higher light beam density than the sparse region of the vertical FOV, and the depth sensor does not include any mechanically movable parts configured to scan the light. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The present application can be best understood by reference to the embodiments described below in conjunction with the accompanying drawings, in which like parts may be denoted by like reference numerals.

[0011] Figure 1 Illustrates one or more exemplary LiDAR systems disposed or included in a motor vehicle.

[0012] Figure 2 Is a block diagram illustrating the interaction between an exemplary LiDAR system and a plurality of other systems including a vehicle perception and planning system.

[0013] Figure 3 Is a block diagram illustrating an exemplary LiDAR system.

[0014] Figure 4 Is a block diagram illustrating an exemplary semiconductor-based laser source.

[0015] Figures 5A to 5C Illustrates an exemplary LiDAR system that uses a pulsed signal to measure the distance to an object disposed in a field of view (FOV).

[0016] Figure 6 Is a block diagram illustrating an exemplary apparatus for implementing systems, devices, and methods in various embodiments.

[0017] Figure 7 Is a block diagram illustrating an exemplary depth sensor according to some embodiments.

[0018] Figure 8 Is a block diagram illustrating another exemplary depth sensor according to some embodiments.

[0019] Figure 9 Illustrates an exemplary depth sensor that provides a non-uniform distribution of light beams in the vertical direction of the FOV according to some embodiments.

[0020] Figure 10 Is a block diagram illustrating the detection range requirement varying according to the transmitted light angle in the vertical FOV according to some embodiments.

[0021] Figure 11 Is a block diagram illustrating providing a non-uniform distribution of light beams by unevenly placing VCSEL elements in a VCSEL laser array according to some embodiments.

[0022] Figure 12 Is a block diagram illustrating providing a non-uniform distribution of light beams by using an optical diffuser according to some embodiments.

[0023] Figure 13 Is a block diagram illustrating providing a non-uniform distribution of light beams by using a semiconductor wafer with a microlens array according to some embodiments.

[0024] Figure 14 is a flowchart illustrating a method of unevenly distributing multiple light beams using a depth sensor according to some embodiments. Detailed Description

[0025] To more thoroughly understand the various embodiments of the present invention, the following description sets forth numerous specific details, such as specific configurations, parameters, examples, etc. However, it should be recognized that such description is not intended to limit the scope of the present invention, but rather to provide a better description of the exemplary embodiments.

[0026] Throughout the specification and claims, unless the context clearly dictates otherwise, the following terms have the meanings explicitly associated herein:

[0027] As used herein, the phrase "in one embodiment" does not necessarily refer to the same embodiment, although it may be. Thus, as described below, the various embodiments of the present invention can be readily combined without departing from the scope or spirit of the present disclosure.

[0028] As used herein, the term "or" is an inclusive "or" operator and is equivalent to the term "and / or" unless the context clearly dictates otherwise.

[0029] The term "based on" is not exclusive and allows for additional factors not described, unless the context clearly dictates otherwise.

[0030] As used herein, unless the context otherwise indicates, the term "coupled to" is intended to include direct coupling (where two elements coupled to each other are in contact with each other) and indirect coupling (where at least one additional element is located between the two elements). Thus, the terms "coupled to" and "coupled with" are used synonymously. In the context of a networking environment where two or more components or devices are capable of exchanging data, the terms "coupled to" and "coupled with" are also used to mean "communicatively coupled with" via one or more intermediate devices. A component or device can be optical, mechanical, and / or electrical.

[0031] Although the following description uses the terms "first", "second", etc. to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the various described examples, a first detection range can be referred to as a second detection range, and similarly, a second detection range can be referred to as a first detection range. The first detection range and the second detection range can both be detection ranges, and in some cases can be separate and distinct detection ranges.

[0032] In addition, throughout the specification, the meanings of "a", "an", and "the" include the plural, and the meaning of "in" may include "in" and "on".

[0033] Although some of the various embodiments presented herein constitute a single combination of elements of the invention, it should be understood that the inventive subject matter is considered to include all possible combinations of the disclosed elements. Thus, if one embodiment includes elements A, B, and C, and another embodiment includes elements B and D, the inventive subject matter is also considered to include the other remaining combinations of A, B, C, or D, even if not explicitly discussed herein. Additionally, the transitional term "comprising" means having the components or elements, or being those components or elements. As used herein, the transitional term "comprising" is inclusive or open-ended and does not exclude additional, unrecited elements or method steps.

[0034] As used throughout the description herein and in the claims that follow, when a system, engine, server, device, module, or other computing element is described as being configured to perform or execute a function on data in a memory, the meaning of "configured to" or "programmed to" is defined as one or more processors or cores of the computing element being programmed by a set of software instructions stored in the memory of the computing element to perform the set of functions on target data or data objects stored in the memory.

[0035] It should be noted that any language directed to a computer should be understood to include any suitable combination of computing devices or network platforms (including servers, interfaces, systems, databases, agents, peers, engines, controllers, modules, or other types of computing devices operating individually or jointly).

[0036] It should be understood that a computing device includes a processor configured to execute software instructions stored on a tangible, non-transitory computer-readable storage medium (e.g., a hard disk drive, FPGA, PLA, solid state drive, RAM, flash memory, ROM, or any other volatile or non-volatile storage device). The software instructions configure or program the computing device to provide roles, responsibilities, or other functionality as discussed below with respect to the disclosed apparatus. Additionally, the disclosed technology may be embodied as a computer program product that includes a non-transitory computer-readable medium storing software instructions that cause a processor to perform the disclosed steps associated with the implementation of computer-based algorithms, processes, methods, or other instructions. In some embodiments, various servers, systems, databases, or interfaces exchange data using standardized protocols or algorithms, possibly based on HTTP, HTTPS, AES, public-private key exchange, web service APIs, known financial transaction protocols, or other methods of electronic information exchange. Data exchange between devices may occur via: packet-switched networks, the Internet, LANs, WANs, VPNs, or other types of packet-switched networks; circuit-switched networks; cell-switched networks; or other types of networks.

[0037] A solid-state depth sensor is a sensor capable of outputting three-dimensional (3D) depth measurements of a field of view without having any mechanically movable parts within the sensor. A solid-state depth sensor can be a semiconductor-based sensor. One type of solid-state sensor is flash LiDAR. When flash LiDAR operates, the entire FOV is typically illuminated by a single pulse or a single emission of a wide-divergence laser beam. Unlike scanning LiDAR (e.g., a LiDAR system having an optical steering mechanism), flash LiDAR may not have any mechanically movable parts optics for scanning the FOV. Thus, flash LiDAR may be more compact than scanning LiDAR without using scanning components. Eliminating mechanically movable parts also makes flash LiDAR (and other solid-state depth sensors) more robust, durable, and reliable.

[0038] Solid-state depth sensors can use vertical-cavity surface-emitting lasers (VCSELs) as light sources. A VCSEL is a type of semiconductor laser diode whose laser beam is emitted perpendicular to the wafer surface or mounting surface. In contrast, edge-emitting semiconductor lasers (EELs) propagate the laser beam in a direction along or parallel to the wafer surface of the semiconductor chip. For an edge-emitting semiconductor laser, the laser is typically reflected or coupled out at the cleavage edge of the wafer. Compared with EELs, VCSELs can provide higher beam quality and thus have better performance. Compared with EELs, VCSELs tend to have lower-power lasers. Additionally, testing VCSELs is usually easier than testing EELs. For example, testing of VCSELs can use wafer probes that are less costly and have simpler procedures, which are readily available in the semiconductor industry.

[0039] Solid-state depth sensors can use single-photon avalanche diode (SPAD) arrays as photodetectors for detecting the returned light. As described above, the returned light is the light formed in the FOV when the transmitted beam from the depth sensor is scattered or reflected by one or more objects in the FOV. A single-photon avalanche diode or SPAD is a solid-state photodetector based on a reverse-biased semiconductor p-n junction, such as a photodiode and an avalanche photodiode (APD). Different from a conventional photodiode, a SPAD operates in a mode called "Geiger mode", in which a single incident photon can generate an electron-hole pair, which is amplified to produce a measurable current. Therefore, a SPAD is inherently capable of detecting single photons with very high time resolution. The key component of a SPAD is the region inside the diode, called the depletion region. This region is designed to have a high electric field, which enables it to be used as a high-gain avalanche photodiode. When a single photon interacts with the depletion region, it generates an electron-hole pair. The high electric field across the depletion region causes the electrons and holes to accelerate, leading to a process called impact ionization, in which each electron or hole can gain enough energy to generate another electron-hole pair, resulting in an avalanche effect. This avalanche process rapidly amplifies the initial signal, converting the weak light signal from a single photon into a detectable electrical pulse. Multiple SPADs can be arranged to form a one-dimensional array, a two-dimensional array, or a three-dimensional array.

[0040] In addition to a light source and a light detector, a depth sensor may have other components, such as optics and control circuitry, which will be described in more detail below. The depth sensor may use a direct time-of-flight (dToF) method to measure distance (also referred to as depth) and is thus a dToF sensor. The depth sensor may also be an indirect time-of-flight (iToF) sensor that uses an indirect time-of-flight (iToF) method to measure distance. The dToF method includes directly measuring the time-of-flight between the time light is emitted from the depth sensor and the time the depth sensor detects the returned light. Using the time-of-flight, the distance between the depth sensor and the target object can be calculated (using the well-known speed of light). The iToF method measures distance by collecting the returned light and discerning the phase shift between the emitted light and the returned light. The iToF method is particularly effective in high-speed, high-resolution 3D imaging of objects located at short and long distances. The indirect ToF sensor emits continuous modulated light and measures the phase of the returned light to calculate the distance to the target object.

[0041] As described above, a solid-state depth sensor may not have any mechanically movable parts. The absence of movable parts is an advantage of the solid-state depth sensor. When the solid-state depth sensor operates, a laser source (e.g., VCSEL) emits laser light into the FOV, and a detector (e.g., SPAD array) captures the returned light formed by the object in the FOV. In the present disclosure, a depth sensor, flash LiDAR, iToF sensor, and dToF sensor may also be referred to as LiDAR. Thus, the present disclosure uses LiDAR as an example and may use LiDAR and depth sensor interchangeably. The present disclosure provides a novel method for optimizing the emission light distribution of a solid-state depth sensor. Using this novel method, the detection range distribution of the depth sensor can be optimized. The energy consumption of the depth sensor can also be reduced.

[0042] In one embodiment, a depth sensor is provided. The depth sensor includes: one or more light sources configured to provide a plurality of light beams; and one or more optical structures coupled to the one or more light sources. The one or more optical structures are configured to receive the plurality of light beams. At least one of the one or more light sources or the one or more optical structures is configured to non-uniformly distribute the plurality of light beams in a vertical field of view (FOV) such that the vertical FOV includes a dense region and a sparse region. The dense region of the vertical FOV has a higher light beam density than the sparse region of the vertical FOV, and the depth sensor does not include any mechanically movable parts configured to scan light into the FOV.

[0043] Figure 1Illustrated are one or more exemplary LiDAR systems 110 and 120A - 120I disposed or included in a motor vehicle 100. The vehicle 100 can be an automobile, a sport utility vehicle (SUV), a truck, a train, a freight car, a bicycle, a motorcycle, a tricycle, a bus, a motor scooter, a tram, a ship, a boat, an underwater vehicle, an airplane, a helicopter, an unmanned aerial vehicle (UAV), a spacecraft, etc. The motor vehicle 100 can be a vehicle with any level of automation. For example, the motor vehicle 100 can be a partially automated vehicle, a highly automated vehicle, a fully automated vehicle, or a driverless vehicle. A partially automated vehicle can perform some driving functions without human driver intervention. For example, a partially automated vehicle can perform blind spot monitoring, lane keeping and / or lane change operations, automatic emergency braking, intelligent cruise and / or traffic following, etc. Certain operations of a partially automated vehicle may be limited to specific applications or driving scenarios (e.g., limited to highway driving). A highly automated vehicle can generally perform all operations of a partially automated vehicle, but with fewer restrictions. A highly automated vehicle can also detect its own limits when operating the vehicle and, when necessary, request the driver to take over control of the vehicle. A fully automated vehicle can perform all vehicle operations without driver intervention, but can also detect its own limits and, when necessary, request the driver to take over. A driverless vehicle can operate on its own without any driver intervention.

[0044] In a typical configuration, the motor vehicle 100 includes one or more LiDAR systems 110 and 120A - 120I. Each of the LiDAR systems 110 and 120A - 120I can be a scanning - based LiDAR system and / or a non - scanning LiDAR system (e.g., flash LiDAR). A scanning - based LiDAR system scans one or more light beams in one or more directions (e.g., horizontal and vertical directions) to detect objects in the field of view (FOV). A non - scanning LiDAR system emits laser light without scanning to illuminate the FOV. For example, flash LiDAR is a type of non - scanning LiDAR system. Flash LiDAR can emit laser light and use a single light pulse or burst to simultaneously illuminate the FOV.

[0045] LiDAR systems are common sensors for at least partially automated vehicles. In one embodiment, as Figure 1As shown, the motor vehicle 100 may include a single LiDAR system 110 (e.g., without LiDAR systems 120A - 120I) disposed at the highest position of the vehicle (e.g., on top of the vehicle). Disposing the LiDAR system 110 on top of the vehicle facilitates 360 - degree scanning around the vehicle 100. In some other embodiments, the motor vehicle 100 may include multiple LiDAR systems, including two or more of systems 110 and / or 120A - 120I. As Figure 1 shown, in one embodiment, multiple LiDAR systems 110 and / or 120A - 120I are attached to the vehicle 100 at different positions of the vehicle. For example, LiDAR system 120A is attached to the vehicle 100 at the right front corner; LiDAR system 120B is attached to the vehicle 100 at the front center position; LiDAR system 120C is attached to the vehicle 100 at the left front corner; LiDAR system 120D is attached to the vehicle 100 at the right side rearview mirror; LiDAR system 120E is attached to the vehicle 100 at the left side rearview mirror; LiDAR system 120F is attached to the vehicle 100 at the rear center position; LiDAR system 120G is attached to the vehicle 100 at the right rear corner; LiDAR system 120H is attached to the vehicle 100 at the left rear corner; and / or LiDAR system 120I is attached to the vehicle 100 at the center facing the rear end (e.g., the rear end of the top of the vehicle). It should be understood that one or more LiDAR systems may be distributed and attached to the vehicle in any desired manner, and Figure 1 only one embodiment is illustrated. As another example, LiDAR systems 120D and 120E may be attached to the B - pillar of the vehicle 100 instead of the rearview mirrors. As another example, LiDAR system 120B may be attached to the windshield of the vehicle 100 instead of the front bumper.

[0046] In some embodiments, LiDAR systems 110 and 120A - 120I are independent LiDAR systems, having their respective laser sources, control electronics, transmitters, receivers, and / or steering mechanisms. In other embodiments, some of LiDAR systems 110 and 120A - 120I may share one or more components, thus forming a distributed sensor system. In one example, optical fibers are used to deliver laser light from a centralized laser source to all LiDAR systems. For example, system 110 (or another system located at the center or any location of vehicle 100) includes a light source, a transmitter, and a photodetector, but no steering mechanism. System 110 can distribute the transmitted light to each of systems 120A - 120I. The transmitted light can be distributed via optical fibers. Optical connectors can be used to couple the optical fibers to each of systems 110 and 120A - 120I. In some examples, one or more of systems 120A - 120I include a steering mechanism, but no light source, transmitter, or photodetector. The steering mechanism can include one or more movable mirrors, such as one or more polygon mirrors, one or more single - plane mirrors, one or more multi - plane mirrors, etc. Embodiments of the light source, transmitter, steering mechanism, and photodetector will be described in more detail below. Via the steering mechanism, one or more of systems 120A - 120I scan light into one or more corresponding FOVs and receive the corresponding returned light. The returned light is formed by scattering or reflecting the transmitted light by one or more objects in the FOV. Systems 120A - 120I may also include collection lenses and / or other optics to focus and / or direct the returned light into an optical fiber, which delivers the received returned light to system 110. System 110 includes one or more photodetectors for detecting the received returned light. In some examples, system 110 is disposed inside the vehicle such that it is in a temperature - controlled environment, while one or more of systems 120A - 120I may be at least partially exposed to the external environment.

[0047] Figure 2 FIG. 200 is a block diagram illustrating the interaction between an in - vehicle LiDAR system 210 and a plurality of other systems including a vehicle perception and planning system 220. The LiDAR system 210 can be mounted on or integrated into a vehicle. The LiDAR system 210 includes sensors that scan laser light into the surrounding environment to measure the distance, angle, and / or velocity of objects. Based on the scattered light returned to the LiDAR system 210, it can generate sensor data (e.g., image data or 3D point cloud data) representing the perceived external environment.

[0048] The LiDAR system 210 may include one or more of a short-range LiDAR sensor, a mid-range LiDAR sensor, and a long-range LiDAR sensor. The short-range LiDAR sensor measures objects up to approximately 20 - 50 meters from the LiDAR sensor. The short-range LiDAR sensor can be used, for example, to monitor nearby moving objects (e.g., pedestrians crossing the road in a school zone), parking assistance applications, etc. The mid-range LiDAR sensor measures objects up to approximately 70 - 200 meters from the LiDAR sensor. The mid-range LiDAR sensor can be used, for example, to monitor road intersections, assist in merging onto or leaving a highway, and so on. The long-range LiDAR sensor measures objects located 200 meters and above. The long-range LiDAR sensor is typically used when the vehicle is traveling at high speed (e.g., on a highway), such that the vehicle's control system may have only a few seconds (e.g., 6 - 8 seconds) to respond to any situation detected by the LiDAR sensor. As Figure 2 shown, in one embodiment, the LiDAR sensor data can be provided to the vehicle perception and planning system 220 via the communication path 213 for further processing and control of vehicle operation. The communication path 213 can be any wired or wireless communication link capable of transmitting data.

[0049] Still referring to Figure 2 , in some embodiments, other vehicle sensors 230 are configured to provide additional sensor data either individually or in conjunction with the LiDAR system 210. The other vehicle sensors 230 can include, for example, one or more cameras 232, one or more radars 234, one or more ultrasonic sensors 236, and / or other sensors 238. The camera 232 can capture images and / or videos of the vehicle's external environment.

[0050] The camera 232 can capture, for example, high-definition (HD) video with millions of pixels per frame. The camera includes an image sensor facilitating the generation of monochromatic or color images and videos. Color information may be important in interpreting data in certain situations (e.g., interpreting an image of a traffic light). Color information may not be obtainable from other sensors such as LiDAR or radar sensors. The camera 232 can include one or more of a narrow focal length camera, a wider focal length camera, a side camera, an infrared camera, a fish-eye camera, etc. The image and / or video data generated by the camera 232 can also be provided to the vehicle perception and planning system 220 via the communication path 233 for further processing and control of vehicle operation. The communication path 233 can be any wired or wireless communication link capable of transmitting data. The camera 232 can be mounted or integrated onto the vehicle at any location (e.g., rearview mirror, pillar, front grille, and / or rear bumper, etc.).

[0051] Other vehicle sensors 230 may also include a radar sensor 234. The radar sensor 234 uses radio waves to determine the distance, angle, and speed of an object. The radar sensor 234 generates electromagnetic waves in the radio or microwave spectrum. The electromagnetic waves are reflected by the object, and some of the reflected waves return to the radar sensor, providing information about the object's position and speed. The radar sensor 234 may include one or more of short-range radar, mid-range radar, and long-range radar. The short-range radar measures objects at a distance of about 0.1 - 30 meters from the radar. The short-range radar is useful for detecting objects near a vehicle (such as other vehicles, buildings, walls, pedestrians, cyclists, etc.). The short-range radar can be used for detecting blind spots, assisting lane changes, providing rear-end collision warnings, assisting parking, providing emergency braking, etc. The mid-range radar measures objects at a distance of about 30 - 80 meters from the radar. The long-range radar measures objects located at about 80 - 200 meters. The mid-range and / or long-range radar can be used for, for example, traffic tracking, adaptive cruise control, and / or highway automatic braking. The sensor data generated by the radar sensor 234 can also be provided to the vehicle perception and planning system 220 via the communication path 233 for further processing and controlling vehicle operations. The radar sensor 234 can be mounted or integrated onto the vehicle at any location (such as the rearview mirror, pillar, front grille, and / or rear bumper, etc.).

[0052] Other vehicle sensors 230 may also include an ultrasonic sensor 236. The ultrasonic sensor 236 uses sound waves or pulses to measure an object located outside the vehicle. The sound waves generated by the ultrasonic sensor 236 are emitted into the surrounding environment. At least some of the emitted waves are reflected by the object and return to the ultrasonic sensor 236. Based on the returned signal, the distance to the object can be calculated. The ultrasonic sensor 236 can be used for, for example, checking blind spots, identifying parking spaces, providing lane change assistance in traffic, etc. The sensor data generated by the ultrasonic sensor 236 can also be provided to the vehicle perception and planning system 220 via the communication path 233 for further processing and controlling vehicle operations. The ultrasonic sensor 236 can be mounted or integrated onto the vehicle at any location (such as the rearview mirror, pillar, front grille, and / or rear bumper, etc.).

[0053] In some embodiments, one or more other sensors 238 may be attached in the vehicle and may also generate sensor data. The other sensors 238 may include, for example, a Global Positioning System (GPS), an Inertial Measurement Unit (IMU), etc. The sensor data generated by the other sensors 238 can also be provided to the vehicle perception and planning system 220 via the communication path 233 for further processing and controlling vehicle operations. It should be understood that the communication path 233 may include one or more communication links to transmit data between the various sensors 230 and the vehicle perception and planning system 220.

[0054] In some embodiments, as Figure 2 shown, sensor data from other vehicle sensors 230 can be provided to the vehicle LiDAR system 210 via a communication path 231. The LiDAR system 210 can process the sensor data from other vehicle sensors 230. For example, sensor data from a camera 232, a radar sensor 234, an ultrasonic sensor 236, and / or other sensors 238 can be correlated or fused with the sensor data of the LiDAR system 210, thereby at least partially offloading the sensor fusion process performed by the vehicle perception and planning system 220. It should be understood that other configurations can also be implemented to transmit and process sensor data from various sensors (e.g., the data can be transmitted to a cloud or an edge computing service provider for processing, and then the processing results can be transmitted back to the vehicle perception and planning system 220 and / or the LiDAR system 210).

[0055] Still referring to Figure 2 , in some embodiments, sensors on other vehicles 250 are used to provide additional sensor data either alone or in conjunction with the LiDAR system 210. For example, two or more nearby vehicles can each have their own LiDAR sensors, cameras, radar sensors, ultrasonic sensors, etc. The nearby vehicles can communicate with each other and share sensor data. Communication between vehicles is also referred to as V2V

[0056] (vehicle-to-vehicle) communication. For example, as Figure 2 shown, sensor data generated by other vehicles 250 can be transmitted to the vehicle perception and planning system 220 and / or the vehicle LiDAR system 210 via communication path 253 and / or communication path 251, respectively. Communication paths 253 and 251 can be any wired or wireless communication link capable of transmitting data.

[0057] Sharing sensor data facilitates better perception of the environment outside the vehicle. For example, a first vehicle may not sense a pedestrian behind a second vehicle but approaching the first vehicle. The second vehicle can share the sensor data related to the pedestrian with the first vehicle, so that the first vehicle can have additional reaction time to avoid colliding with the pedestrian. In some embodiments, similar to the data generated by the sensors 230, the data generated by the sensors on other vehicles 250 can be correlated or fused with the sensor data generated by the LiDAR system 210 (or other LiDAR systems located in other vehicles), thereby at least partially offloading the sensor fusion process performed by the vehicle perception and planning system 220.

[0058] In some embodiments, the intelligent infrastructure system 240 is used to provide sensor data alone or in conjunction with the LiDAR system 210. Certain infrastructure can be configured to communicate with vehicles to transfer information and vice versa.

[0059] Communication between vehicles and infrastructure is commonly referred to as V2I (vehicle-to-infrastructure) communication. For example, the intelligent infrastructure system 240 can include intelligent traffic lights that can convey their status to approaching vehicles in a message such as "turn yellow in 5 seconds". The intelligent infrastructure system 240 can also include its own LiDAR system installed near intersections so that it can transfer traffic monitoring information to vehicles. For example, a vehicle turning left at an intersection may not have sufficient sensing capabilities because some of its own sensors may be blocked by oncoming traffic. In such a case, the sensors of the intelligent infrastructure system 240 can provide useful data to the vehicle turning left. Such data can include, for example, traffic conditions, information about objects in the vehicle's turning direction, traffic light status, and predictions. These sensor data generated by the intelligent infrastructure system 240 can be provided to the vehicle perception and planning system 220 and / or the on-vehicle LiDAR system 210 via communication paths 243 and / or 241 respectively.

[0060] The communication paths 243 and / or 241 can include any wired or wireless communication link capable of transmitting data. For example, sensor data from the intelligent infrastructure system 240 can be transmitted to the LiDAR system 210 and correlated or fused with the sensor data generated by the LiDAR system 210, thereby at least partially offloading the sensor fusion process performed by the vehicle perception and planning system 220. The above V2V and V2I communications are examples of vehicle-to-X (V2X) communications, where "X" represents any other device, system, sensor, infrastructure, etc. that can share data with the vehicle.

[0061] Still referring to Figure 2, via various communication paths, the vehicle perception and planning system 220 receives sensor data from one or more of the LiDAR system 210, other on-vehicle sensors 230, other vehicles 250, and / or the intelligent infrastructure system 240. In some embodiments, different types of sensor data are correlated and / or fused by the sensor fusion subsystem 222. For example, the sensor fusion subsystem 222 may use multiple images or videos captured by multiple cameras disposed at different positions of the vehicle to generate a 360-degree model. The sensor fusion subsystem 222 obtains sensor data from different types of sensors and uses the combined data to more accurately perceive the environment. For example, the on-vehicle camera 232 may not be able to capture clear images because it faces the sun or a light source directly (e.g., the headlights of another vehicle at night). The LiDAR system 210 may not be affected much, and thus the sensor fusion subsystem 222 can combine the sensor data provided by the camera 232 and the LiDAR system 210 and use the sensor data provided by the LiDAR system 210 to compensate for the unclear images captured by the camera 232. As another example, in rainy or foggy weather, the radar sensor 234 may work better than the camera 232 or the LiDAR system 210. Accordingly, the sensor fusion subsystem 222 can use the sensor data provided by the radar sensor 234 to compensate for the sensor data provided by the camera 232 or the LiDAR system 210.

[0062] In other examples, the sensor data generated by the other on-vehicle sensors 230 may have a lower resolution (e.g., radar sensor data), and thus may need to be correlated and confirmed by the LiDAR system 210, which typically has a higher resolution. For example, the radar sensor 234 may detect a manhole cover (also known as an inspection hole cover) as an object that the vehicle is approaching. Due to the low-resolution characteristics of the radar sensor 234, the vehicle perception and planning system 220 may not be able to determine whether the object is an obstacle that the vehicle needs to avoid. Therefore, the high-resolution sensor data generated by the LiDAR system 210 can be used to correlate and confirm that the object is a manhole cover and will not cause damage to the vehicle.

[0063] The vehicle perception and planning system 220 further includes an object classifier 223. Using the raw sensor data and / or the correlated / fused data provided by the sensor fusion subsystem 222, the object classifier 223 can use any computer vision technique to detect and classify objects and estimate the positions of the objects. In some embodiments, the object classifier 223 can use machine learning-based techniques to detect and classify objects. Examples of machine learning-based techniques include algorithms such as region-based convolutional neural network (R-CNN), fast R-CNN, faster R-CNN, histogram of oriented gradients (HOG), region-based fully convolutional network (R-FCN), single shot detector (SSD), spatial pyramid pooling (SPP-net), and / or "You Only Look Once" (Yolo).

[0064] The vehicle perception and planning system 220 further includes a road detection subsystem 224. The road detection subsystem 224 locates the road and identifies objects and / or markings on the road. For example, based on the raw or fused sensor data provided by the radar sensor 234, the camera 232, and / or the LiDAR system 210, the road detection subsystem 224 can build a 3D model of the road based on machine learning techniques (e.g., pattern recognition algorithms for identifying lanes). Using the 3D model of the road, the road detection subsystem 224 can identify objects on the road (e.g., obstacles or debris on the road) and / or markings (e.g., lane lines, turn signs, crosswalk signs, etc.).

[0065] The vehicle perception and planning system 220 further includes a localization and vehicle pose subsystem 225. Based on the raw or fused sensor data, the localization and vehicle pose subsystem 225 can determine the position and the pose of the vehicle. For example, using the sensor data from the LiDAR system 210, the camera 232, and / or the GPS data, the localization and vehicle pose subsystem 225 can determine the precise position of the vehicle on the road and the six degrees of freedom of the vehicle (e.g., whether the vehicle is moving forward or backward, up or down, left or right). In some embodiments, a high-definition (HD) map is used for vehicle localization. The HD map can provide a very detailed three-dimensional computer map to accurately locate the position of the vehicle. For example, using the HD map, the localization and vehicle pose subsystem 225 can accurately determine the current position of the vehicle (e.g., which lane of the road the vehicle is currently in and how close it is to the roadside or sidewalk) and predict the future position of the vehicle.

[0066] The vehicle perception and planning system 220 further includes an obstacle predictor 226. The objects identified by the object classifier 223 can be stationary (e.g., lamp posts, road signs) or dynamic (e.g., moving pedestrians, bicycles, another vehicle). For moving objects, predicting their movement paths or future positions is important for avoiding collisions. The obstacle predictor 226 can predict the obstacle trajectories and / or warn the driver or the vehicle planning subsystem 228 of potential collisions. For example, if there is a high likelihood that the trajectory of an obstacle will intersect the current movement path of the vehicle, the obstacle predictor 226 can generate such a warning. The obstacle predictor 226 can use various techniques to make such predictions. These techniques include, for example, constant speed or acceleration models, constant turn rate and speed / acceleration models, models based on Kalman filters and extended Kalman filters, models based on recurrent neural networks (RNNs), models based on long short-term memory (LSTM) neural networks, encoder-decoder RNN models, and the like.

[0067] Still referring Figure 2 , in some embodiments, the vehicle perception and planning system 220 further includes a vehicle planning subsystem 228. The vehicle planning subsystem 228 can include one or more planners, such as a route planner, a driving behavior planner, and a motion planner. The route planner can plan the route of the vehicle based on the vehicle's current location data, target location data, traffic information, and the like. The driving behavior planner uses the obstacle prediction results provided by the obstacle predictor 226 to adjust the timing and planned movement based on how other objects might move. The motion planner determines the specific actions that the vehicle needs to follow. The planning results are then transmitted to the vehicle control system 280 via the vehicle interface 270. The communication can be performed through communication paths 227 and 271, which include any wired or wireless communication links that can transmit data.

[0068] The vehicle control system 280 controls the steering mechanism, throttle, brakes, etc. of the vehicle to operate the vehicle according to the planned route and movement. In some examples, the vehicle perception and planning system 220 may further include a user interface 260 that provides the user (e.g., the driver) access to the vehicle control system 280 to, for example, override or take over control of the vehicle when necessary. The user interface 260 may also be separate from the vehicle perception and planning system 220. The user interface 260 may communicate with the vehicle perception and planning system 220, for example, to obtain and display raw or fused sensor data, identified objects, the position / attitude of the vehicle, etc. The displayed data may assist the user in better operating the vehicle. The user interface 260 may communicate with the vehicle perception and planning system 220 and / or the vehicle control system 280 via communication paths 221 and 261, respectively, which communication paths include any wired or wireless communication links capable of transmitting data. It should be understood that Figure 2 the various systems, sensors, communication links, and interfaces can be configured in any desired manner and are not limited to Figure 2 the configurations shown.

[0069] Figure 3 is a block diagram illustrating an exemplary LiDAR system 300. The LiDAR system 300 can be used to implement Figure 1 and Figure 2 the LiDAR systems 110, 120A - 120I, and / or 210 shown in. In one embodiment, the LiDAR system 300 includes a light source 310, a transmitter 320, an optical receiver and a photodetector 330, a steering system 340, and a control circuit 350. These components are coupled together using communication paths 312, 314, 322, 332, 342, 352, and 362. These communication paths include communication links (wired or wireless, bidirectional or unidirectional) between the various LiDAR system components, but do not have to be physical components themselves. Although the communication paths can be implemented by one or more wires, buses, or optical fibers, the communication paths can also be wireless channels or free - space optical paths, so that there is no physical communication medium. For example, in one embodiment of the LiDAR system 300, the communication path 314 between the light source 310 and the transmitter 320 can be implemented using one or more optical fibers. The communication paths 332 and 352 can represent optical paths implemented using free - space optical components and / or optical fibers. And the communication paths 312, 322, 342, and 362 can be implemented using one or more wires carrying electrical signals. The communication paths can also include one or more of the above - mentioned types of communication media (e.g., they can include optical fibers and free - space optical components, or include one or more optical fibers and one or more wires).

[0070] In some embodiments, the LiDAR system 300 can be a coherent LiDAR system. Frequency-modulated continuous-wave (FMCW) LiDAR is an example. Coherent LiDAR detects objects by mixing the returned light from the objects and the light from a coherent laser transmitter.

[0071] Thus, as Figure 3 shown, if the LiDAR system 300 is a coherent LiDAR, it can include a path 372 that provides a portion of the transmitted light from the transmitter 320 to the optical receiver and photodetector 330. The path 372 can include one or more optical devices (e.g., optical fibers, lenses, mirrors, etc.) for providing the light from the transmitter 320 to the optical receiver and photodetector 330. The transmitted light provided by the transmitter 320 can be modulated light and can be split into two parts. One part is emitted into the FOV, while the second part is sent to the optical receiver and photodetector of the LiDAR system. The second part is also referred to as the light that is kept local (LO) in the LiDAR system. The transmitted light is scattered or reflected by various objects in the FOV, and at least a portion of it forms the returned light. The returned light is then detected and interfered and recombined with the second part of the transmitted light that is kept local. Coherent LiDAR provides a mechanism for optically sensing the range of an object and its relative velocity along the line of sight (LOS).

[0072] The LiDAR system 300 can also include Figure 3 other components not shown in

[0073] The light source 310 outputs a laser for irradiating an object in the field of view (FOV). The laser can be infrared light with a wavelength in the range of 700 nm to 1 mm. The light source 310 can be, for example, a semiconductor-based laser (e.g., a diode laser) and / or a fiber-based laser. The semiconductor-based laser can be, for example, an edge-emitting laser (EEL), a vertical-cavity surface-emitting laser (VCSEL), an external cavity diode laser, a vertical-cavity surface-emitting laser, a distributed feedback (DFB) laser, a distributed Bragg reflector (DBR) laser, an interband cascade laser, a quantum cascade laser, a quantum well laser, a double heterostructure laser, etc. The fiber-based laser is a laser in which the active gain medium is a fiber doped with rare earth elements such as erbium, ytterbium, neodymium, dysprosium, praseodymium, thulium, and / or holmium. In some embodiments, the fiber laser is based on a double-clad fiber, where the gain medium forms the core of the fiber surrounded by two layers of cladding. The double-clad fiber allows the core to be pumped with a high-power beam, enabling the laser source to be a high-power fiber laser source.

[0074] In some embodiments, the light source 310 includes a master oscillator (also referred to as a seed laser) and a power amplifier (MOPA). The power amplifier amplifies the output power of the seed laser.

[0075] The power amplifier can be a fiber amplifier, a bulk amplifier, or a semiconductor optical amplifier. The seed laser can be a diode laser (e.g., a Fabry - Perot cavity laser, a distributed feedback laser), a solid-state bulk laser, or an external cavity tunable diode laser. In some embodiments, the light source 310 can be an optically pumped microchip laser. A microchip laser is an alignment-free monolithic solid-state laser in which the laser crystal is in direct contact with the end mirrors of the laser resonator. Microchip lasers are typically pumped by laser diodes (directly or using fibers) to obtain the required output power. The microchip laser can be based on a neodymium-doped yttrium aluminum garnet (Y3Al5O12) laser crystal (i.e., Nd:YAG), or a neodymium-doped vanadate (i.e., ND:YVO4) laser crystal. In some examples, the light source 310 can have multiple amplification stages to achieve high power gain, such that the laser output can have high power, enabling the LiDAR system to have a long scanning range. In some examples, the power amplifier of the light source 310 can be controlled such that the power gain can be changed to achieve any desired laser output power.

[0076] Figure 4 is a block diagram illustrating an exemplary semiconductor-based laser source 400. The semiconductor-based laser source 400 is Figure 3 an example of the light source 310 shown. In Figure 4In the example shown, the laser source 400 is a vertical cavity surface emitting laser (VCSEL), which is a type of semiconductor laser diode with a unique structure that allows it to emit light vertically from the chip surface, rather than through the edge of the chip as in the case of an edge emitting laser (EEL) diode. VCSELs have the advantages of high-speed operation and ease of integration into semiconductor devices. Figure 4 A cross-sectional view of an exemplary VCSEL 400 is shown. In this example, the VCSEL 400 includes a metal contact layer 402, an upper Bragg reflector 404, an active region 406, a lower Bragg reflector 408, a substrate 410, and another metal contact layer 412. In the VCSEL 400, the metal contact layers 402 and 412 are used to form electrical contacts so that current and / or voltage can be provided to the VCSEL 400 to generate laser light. The substrate layer 410 is a semiconductor substrate, which can be, for example, a gallium arsenide (GaAs) substrate. The VCSEL 400 uses a laser resonator that includes two distributed Bragg reflectors (DBRs) (i.e., the upper Bragg reflector 404 and the lower Bragg reflector 408), with the active region 406 sandwiched between the DBR reflectors. The active region 406 includes, for example, one or more quantum wells for laser generation. The planar DBR reflectors can be mirrors with alternating high and low refractive index layers. Each layer has a thickness of one quarter of the laser wavelength in the material, resulting in a reflectivity of greater than, for example, 99%. The high reflectivity mirrors in the VCSEL can balance the short axial length of the gain region. In one example of the VCSEL 400, the upper DBR reflector 404 and the lower DBR reflector 408 can be doped as p-type and n-type materials, respectively, to form a diode junction. In another example, the p-type and n-type regions can be embedded between the reflectors, which requires a more complex semiconductor process to make electrical contact with the active region but eliminates the electrical power loss in the DBR structure. The active region 406 is sandwiched between the DBR reflectors 404 and 408 of the VCSEL 400. The active region is where the laser light is generated. The active region 406 typically has a quantum well or quantum dot structure that contains a gain medium responsible for optical amplification.

[0077] When current is applied to the active region 406, it generates photons by stimulated emission. The distance between the upper DBR reflector 404 and the lower DBR reflector 408 defines the cavity length of the VCSEL 400. The cavity length in turn determines the wavelength of the emitted light and affects the performance characteristics of the laser. When current is applied to the VCSEL, it generates light that bounces between the DBR reflector 404 and the DBR reflector 408 and exits the VCSEL 400 through, for example, the lower DBR reflector 408, thereby generating a highly coherent and vertically emitted laser beam 414. The VCSEL 400 can provide improved beam quality, low threshold current, and the ability to produce single-mode or multi-mode output.

[0078] In some variations, the VCSEL 400 can be controlled (e.g., by the control circuit 350) to produce pulses of different amplitudes. The communication path 312 couples the VCSEL 400 to the control circuit 350 (as Figure 3 shown) such that the components of the VCSEL 400 can be controlled by the control circuit 350 or otherwise communicate with the control circuit. Alternatively, the VCSEL 400 can include its own dedicated controller. Instead of the control circuit 350 directly communicating with the components of the VCSEL 400, the dedicated controller of the VCSEL 400 communicates with the control circuit 350 and controls and / or communicates with the components of the VCSEL 400. The VCSEL 400 can also include other components not shown, such as one or more power connectors, power supplies, and / or power lines.

[0079] The VCSEL 400 can be used to generate laser pulses or continuous-wave (CW) lasers. To generate laser pulses, the control circuit 350 modulates the current supplied to the VCSEL 400. By quickly turning the power current on and off, laser pulses can be generated. The duration, repetition rate, and shape of the pulses can be controlled by adjusting the modulation parameters. As another example, the VCSEL 400 can also be a mode-locked VCSEL that uses a combination of current modulation and optical feedback to obtain ultrashort pulses. The mode-locked VCSEL can also be controlled to synchronize the phases of the laser modes to produce very short and high-intensity pulses. As another example, the VCSEL 400 can use Q-switching technology, which includes an optical switch in the laser cavity that temporarily blocks laser action and allows energy to accumulate in the cavity. When the switch is opened, high-intensity pulses are emitted. As another example, the VCSEL 400 can also have external modulation performed by an external modulator, such as an electro-optic or acousto-optic modulator. The external modulation can be used in combination with the VCSEL itself to produce a pulsed output. The external modulator can be used to control the pulse duration and repetition rate. The type of VCSEL used as at least a part of the light source 310 depends on the application and the desired pulse characteristics, such as pulse duration, repetition rate, and peak power. Refer toFigure 3 , typical operating wavelengths of the light source 310 include, for example, approximately 850 nm, approximately 905 nm, approximately 940 nm, approximately 1064 nm, and approximately 1550 nm. For laser safety, the upper limit of the maximum available laser power is set by the regulations of the U.S. Food and Drug Administration (FDA). The optical power limit at a wavelength of 1550 nm is much higher than the optical power limits of the other wavelengths mentioned above. In addition, at 1550 nm, the optical power loss in the optical fiber is very low.

[0080] These characteristics of the 1550 nm wavelength make it more advantageous for long-range LiDAR applications. The amount of optical power output from the light source 310 can be characterized by its peak power, average power, pulse energy, and / or pulse energy density. Peak power is the ratio of pulse energy to pulse width (e.g., full width at half maximum or FWHM). Thus, for a fixed amount of pulse energy, a smaller pulse width can provide a larger peak power. The pulse width can be in the range of nanoseconds or picoseconds. Average power is the product of pulse energy and pulse repetition rate (PRR). As described in more detail below, the PRR represents the frequency of the pulsed laser. Generally, the smaller the time interval between pulses, the higher the PRR. The PRR typically corresponds to the maximum range that a LiDAR system can measure. The light source 310 can be configured to generate pulses at a high PRR to meet the desired number of data points in the point cloud generated by the LiDAR system. The light source 310 can also be configured to generate pulses at a medium or low PRR to meet the desired maximum detection range. Wall plug efficiency (WPE) is another factor for evaluating the total power consumption, which can be a useful metric for evaluating the efficiency of the laser. For example, as Figure 1 shown, multiple LiDAR systems can be attached to a vehicle, which can be an electric vehicle or a vehicle with limited fuel or battery power. Therefore, high WPE and an intelligent way of using laser power are usually important considerations when selecting and configuring the light source 310 and / or designing a laser delivery system for in-vehicle LiDAR applications.

[0081] It should be understood that the above description provides a non-limiting example of the light source 310. The light source 310 can be configured to include many other types of light sources (e.g., laser diodes, short cavity fiber lasers, solid-state lasers, and / or external cavity tunable diode lasers), which are configured to generate one or more optical signals at various wavelengths. In some examples, the light source 310 includes amplifiers (e.g., pre-amplifiers and / or booster amplifiers), which can be doped fiber amplifiers, solid-state body amplifiers, and / or semiconductor optical amplifiers. The amplifiers are configured to receive and amplify the optical signal with a desired gain.

[0082] Return to reference Figure 3, the LiDAR system 300 further includes a transmitter 320. A light source 310 provides laser light (e.g., in the form of a laser beam) to the transmitter 320. The laser light provided by the light source 310 can be amplified laser light having a pre-determined or controlled wavelength, pulse repetition rate, and / or power level. The transmitter 320 receives the laser light from the light source 310 and emits the laser light with low divergence into the steering mechanism 340. In some embodiments, the transmitter 320 can include, for example, optical components (e.g., lenses, optical fibers, mirrors, etc.) for emitting one or more laser beams directly or via the steering mechanism 340 into the field of view (FOV). Although Figure 3 the transmitter 320 and the steering mechanism 340 are illustrated as separate components, in some embodiments, they can be combined or integrated into one system. The steering mechanism 340 will be described in more detail below.

[0083] The laser beam provided by the light source 310 may diverge as it propagates to the transmitter 320. Accordingly, the transmitter 320 typically includes a collimating lens that is configured to collect the diverging laser beam and produce a more parallel beam with reduced or minimal divergence. The collimated beam can then be further directed through various optical devices such as mirrors and lenses. The collimating lens can be, for example, a single plano-convex lens or a lens group. The collimating lens can be configured to achieve any desired characteristics such as beam diameter, divergence, numerical aperture, focal length, etc. The beam propagation ratio or beam quality factor (also referred to as the M 2 factor) is used to measure the quality of the laser beam.

[0084] In many LiDAR applications, it is important to have good laser beam quality in the generated transmitted laser beam. The M 2 factor represents the degree of variation of the beam relative to an ideal Gaussian beam. Thus, the M 2 factor reflects how well the collimated laser beam can be focused on a small spot, or how well the diverging laser beam can be collimated. Accordingly, the light source 310 and / or the transmitter 320 can be configured to meet, for example, the scan resolution requirements while maintaining the desired M 2 factor.

[0085] One or more of the light beams provided by the emitter 320 are scanned into the FOV by the steering mechanism 340. The steering mechanism 340 scans the light beam in multiple dimensions (e.g., in the horizontal dimension and the vertical dimension) to facilitate the LiDAR system 300 to map the environment by generating a 3D point cloud. The horizontal dimension can be the dimension parallel to the horizon or the surface associated with the LiDAR system or the vehicle (e.g., the road surface). The vertical dimension is perpendicular to the horizontal dimension (i.e., the vertical dimension forms a 90-degree angle with the horizontal dimension). The steering mechanism 340 will be described in more detail below. The laser scanned into the FOV may be scattered or reflected by an object in the FOV. At least a portion of the scattered light or reflected light forms the return light that returns to the LiDAR system 300. Figure 3 Further illustrated is the optical receiver and the photodetector 330 configured to receive the return light. The optical receiver and the photodetector 330 include an optical receiver configured to collect the return light from the FOV. The optical receiver may include optical devices (e.g., lenses, optical fibers, mirrors, etc.) for receiving, redirecting, focusing, amplifying, and / or filtering the return light from the FOV. For example, the optical receiver typically includes a collection lens (e.g., a single plano-convex lens or a lens group) to collect the return light and / or focus the collected return light onto the photodetector.

[0086] The photodetector detects the return light focused by the optical receiver and generates a current and / or voltage signal proportional to the incident intensity of the return light. Based on such a current and / or voltage signal, the depth information of the object in the FOV can be derived. An exemplary method for deriving such depth information is based on direct TOF (time of flight), which will be described in more detail below. The photodetector can be characterized by its detection sensitivity, quantum efficiency, detector bandwidth, linearity, signal-to-noise ratio (SNR), overload resistance, interference resistance, etc. Based on the application, the photodetector can be configured or customized to have any desired characteristics. For example, the optical receiver and the photodetector 330 can be configured such that the photodetector has a large dynamic range while having good linearity. The photodetector linearity indicates the ability of the detector to maintain a linear relationship between the input optical signal power and the detector output. A detector with good linearity can maintain a linear relationship within a large dynamic input optical signal range.

[0087] To achieve the desired detector characteristics, the structure of the photodetector and / or the detector's material system can be configured or customized. Various detector structures can be used for the photodetector. For example, the photodetector structure can be a PIN-based structure that has an undoped intrinsic semiconductor region (i.e., the "I" region) between a p-type semiconductor and an n-type semiconductor region. Other photodetector structures include, for example, APD (avalanche photodiode)-based structures, PMT (photomultiplier tube)-based structures, SiPM (silicon photomultiplier)-based structures, SPAD (single-photon avalanche diode)-based structures, and / or quantum wires. For the material system used in the photodetector, Si, InGaAs, and / or Si / Ge-based materials can be used. It should be understood that many other detector structures and / or material systems can be used in the optical receiver and the photodetector 330.

[0088] A photodetector (e.g., an APD-based detector) can have internal gain such that the input signal is amplified when an output signal is generated. However, due to the internal gain of the photodetector, noise can also be amplified. Common types of noise include signal shot noise, dark current shot noise, thermal noise, and amplifier noise. In some embodiments, the optical receiver and the photodetector 330 can include a preamplifier of a low-noise amplifier (LNA). In some embodiments, the preamplifier can further include a transimpedance amplifier (TIA) that converts a current signal into a voltage signal. For a linear detector system, the input equivalent noise or noise equivalent power (NEP) measures the sensitivity of the photodetector to weak signals. Thus, they can be used as an indicator of the overall system performance. For example, the NEP of a photodetector specifies the power of the weakest signal that can be detected, and thus it in turn specifies the maximum range of the LiDAR system. It should be understood that various photodetector optimization techniques can be used to meet the requirements of the LiDAR system 300. Such optimization techniques can include selecting different detector structures, materials, and / or implementing signal processing techniques (e.g., filtering, noise reduction, amplification, etc.). For example, in addition to or instead of using direct detection of the return signal (e.g., by using ToF), coherent detection can also be used for the photodetector. Coherent detection allows the detection of the amplitude and phase information of the received light by interfering the received light with a local oscillator. Coherent detection can improve the detection sensitivity and noise immunity.

[0089] Figure 3The LiDAR system 300 is further illustrated to include a steering mechanism 340. As described above, the steering mechanism 340 guides the light beam from the transmitter 320 for multi-dimensional scanning of the FOV. The steering mechanism is referred to as a raster mechanism, a scanning mechanism, or simply an optical scanner. Scanning the light beam in multiple directions (e.g., in the horizontal and vertical directions) facilitates the LiDAR system to map the environment by generating an image or a 3D point cloud. The steering mechanism can be based on mechanical scanning and / or solid-state scanning. Mechanical scanning uses rotating mirrors to steer the laser beam or physically rotates the LiDAR transmitter and receiver (collectively referred to as the transceiver) to scan the laser beam. Solid-state scanning guides the laser beam to various positions through the FOV without mechanically moving any macroscopic components, such as the transceiver. Solid-state scanning mechanisms include, for example, steering based on optical phased arrays and flash LiDAR-based steering. In some embodiments, since the solid-state scanning mechanism does not physically move macroscopic components, the steering performed by the solid-state scanning mechanism can be referred to as effective steering. A LiDAR system using solid-state scanning can also be referred to as a non-mechanical scanning or simply a non-scanning LiDAR system (a flash LiDAR system is an exemplary non-scanning LiDAR system).

[0090] The steering mechanism 340 can be used with a transceiver (e.g., the transmitter 320 and the optical receiver and light detector 330) to scan the FOV for generating an image or a 3D point cloud. As an example, to implement the steering mechanism 340, a two-dimensional mechanical scanner can be used with a single-point or a few single-point transceivers. The single-point transceiver emits a single light beam or a small number of light beams (e.g., 2 - 8 light beams) into the steering mechanism. The two-dimensional mechanical steering mechanism includes, for example, a polygon mirror, an oscillating mirror, a rotating prism, a rotating tilted mirror surface, a single-plane or multi-plane mirror, or a combination thereof. In some embodiments, the steering mechanism 340 can include a non-mechanical steering mechanism, such as a solid-state steering mechanism. For example, the steering mechanism 340 can be based on the tuned wavelength of the laser that combines the refraction effect, and / or based on a reconfigurable grating / phase array. In some embodiments, the steering mechanism 340 can use a single scanning device to achieve two-dimensional scanning, or use a combination of multiple scanning devices to achieve two-dimensional scanning.

[0091] As another example, to implement the steering mechanism 340, a one-dimensional mechanical scanner can be used with an array or a large number of single-point transceivers. Specifically, the transceiver array can be mounted on a rotating platform to achieve a 360-degree horizontal field of view. Alternatively, a static transceiver array can be combined with a one-dimensional mechanical scanner. The one-dimensional mechanical scanner includes a polygon mirror, an oscillating mirror, a rotating prism, a rotating tilted mirror surface, or a combination thereof, for obtaining a forward horizontal field of view. The steering mechanism using a mechanical scanner can provide robustness and reliability in mass production for automotive applications.

[0092] As another example, to implement the steering mechanism 340, a two-dimensional transceiver can be used to directly generate a scanned image or a 3D point cloud. In some embodiments, stitching or micro-displacement methods can be used to increase the resolution of the scanned image or the field of view being scanned. For example, using a two-dimensional transceiver, signals generated in one direction (e.g., the horizontal direction) and signals generated in another direction (e.g., the vertical direction) can be integrated, interleaved, and / or matched to generate a higher or full-resolution image or 3D point cloud representing the scanned FOV.

[0093] Some implementations of the steering mechanism 340 include one or more optical redirecting elements (e.g., mirrors or lenses) that steer the returned optical signal along the receiving path (e.g., by rotation, vibration, or guiding) to direct the returned optical signal to the optical receiver and photodetector 330. The optical redirecting elements that guide the optical signal along the emission path and the receiving path can be the same component (e.g., shared), separate components (e.g., dedicated), and / or a combination of shared and separate components. This means that in some cases, the emission path and the receiving path are different, although they may partially overlap (or in some cases, substantially overlap or completely overlap).

[0094] Still referring to Figure 3 , the LiDAR system 300 further includes control circuitry 350. The control circuit 350 can be configured and / or programmed to control various parts of the LiDAR system 300 and / or perform signal processing. In a typical system, the control circuit 350 can be configured and / or programmed to perform one or more control operations, including, for example, controlling the light source 310 to obtain desired laser pulse timing, pulse repetition rate, and power; controlling the steering mechanism 340 (e.g., controlling speed, direction, and / or other parameters) to scan the FOV and maintain pixel registration and / or alignment; controlling the optical receiver and photodetector 330 (e.g., controlling sensitivity, noise reduction, filtering, and / or other parameters) such that it is in an optimal state; and monitoring the overall system health / functional safety status (e.g., monitoring the safety of the laser output power and / or the operating status of the steering mechanism).

[0095] The control circuitry 350 can also be configured and / or programmed to perform signal processing on the raw data generated by the optical receiver and photodetector 330 to obtain distance and reflectivity information, and perform data packing and communicate with the vehicle perception and planning system 220, such as Figure 2Communication as shown). For example, the control circuit 350 determines the time taken from transmitting an optical pulse to receiving the corresponding returned optical pulse; determines when no returned optical pulse of the transmitted optical pulse is received; determines the direction of the transmitted optical pulse / returned optical pulse (e.g., horizontal information and / or vertical information); determines the estimated range in a specific direction; derives the reflectivity of an object in the FOV, and / or determines any other type of data related to the LiDAR system 300.

[0096] The LiDAR system 300 can be disposed in a vehicle, which can operate in many different environments, including hot or cold weather, rough road conditions that may cause strong vibrations, high or low humidity, dusty areas, etc. Thus, in some embodiments, the optical and / or electronic components of the LiDAR system 300 (e.g., the optics in the transmitter 320, the optical receiver and the photodetector 330, and the steering mechanism 340) are arranged and / or configured in a manner that maintains long-term mechanical and optical stability. For example, the components in the LiDAR system 300 can be fixed and sealed so that they can operate under all conditions that the vehicle may encounter. As an example, a moisture-proof coating and / or an airtight seal can be applied to the optical components of the transmitter 320, the optical receiver and the photodetector 330, and the steering mechanism 340 (and other moisture-sensitive components). As another example, a housing, a cowling, a fairing, and / or a window can be used in the LiDAR system 300 to provide desired characteristics such as hardness, Ingress Protection (IP) rating, self-cleaning ability, chemical resistance, and impact resistance, etc. Additionally, an efficient and economical method for assembling the LiDAR system 300 can be used to meet the LiDAR operation requirements while keeping the cost low.

[0097] Those of ordinary skill in the art should understand that Figure 3 and the above description is for illustrative purposes only, and the LiDAR system can include other functional units, blocks, or segments, and can include variations or combinations of these above functional units, blocks, or segments. For example, the LiDAR system 300 can also include Figure 3 other components not shown in, such as a power bus, a power supply, an LED indicator, a switch, etc. Additionally, there can be other connections between components, such as a direct connection between the light source 310 and the optical receiver and the photodetector 330, so that the photodetector 330 can accurately measure the time from when the light pulse is emitted from the light source 310 to when the returned light pulse is detected by the photodetector 330.

[0098] Figure 3The components shown are coupled together using communication paths 312, 314, 322, 332, 342, 352, and 362. These communication paths represent communication (bidirectional or unidirectional) between various LiDAR system components, but do not have to be the physical components themselves. While the communication paths can be implemented by one or more wires, buses, or optical fibers, the communication paths can also be wireless channels or free-space optical paths, such that there is no physical communication medium. For example, in an exemplary LiDAR system, communication path 314 includes one or more optical fibers; communication path 352 represents an optical path; and communication paths 312, 322, 342, and 362 are all wires that carry electrical signals. The communication paths can also include more than one of the types of communication media described above (e.g., they can include optical fibers and optical paths, or one or more optical fibers and one or more wires).

[0099] As described above, some LiDAR systems use the time-of-flight (ToF) of an optical signal (e.g., an optical pulse) to determine the distance to an object in the optical path. For example, referring to Figure 5A , exemplary LiDAR system 500 includes a laser light source (e.g., a fiber laser), a steering mechanism (e.g., a system of one or more moving mirrors), and a light detector (e.g., a photodetector having one or more optical devices). LiDAR system 500 can be implemented using, for example, the LiDAR system 300 described above. LiDAR system 500 emits optical pulse 502 along optical path 504 determined by the steering mechanism of LiDAR system 500.

[0100] In the example depicted, optical pulse 502 generated by the laser light source is a short pulse of laser light. Additionally, the signal manipulation mechanism of LiDAR system 500 is a pulsed signal steering mechanism. However, it should be understood that a LiDAR system can operate by generating, emitting, and detecting non-pulsed optical signals and using techniques other than time-of-flight to derive the distance to objects in the surrounding environment. For example, some LiDAR systems use frequency-modulated continuous wave (i.e., "FMCW"). It should also be understood that any techniques described herein for time-of-flight-based systems using pulsed signals can also apply to LiDAR systems that do not use one or both of these techniques.

[0101] Return reference Figure 5A(For example, a time-of-flight LiDAR system using optical pulses is illustrated), when the optical pulse 502 reaches the object 506, the optical pulse 502 scatters or reflects to form a returned optical pulse 508. The returned optical pulse 508 can return to the system 500 along the optical path 510. The time from when the emitted optical pulse 502 leaves the LiDAR system 500 to when the returned optical pulse 508 returns to the LiDAR system 500 can be measured (e.g., by a processor or other electronic device within the LiDAR system, such as the control circuit 350). This time of flight combined with the knowledge of the speed of light can be used to determine the distance / distance from the LiDAR system 500 to the part of the object 506 where the optical pulse 502 is scattered or reflected.

[0102] By directing a number of optical pulses, such as Figure 5B depicted, the LiDAR system 500 scans the external environment (e.g., by directing the optical pulses 502, 522, 526, 530 along the optical paths 504, 524, 528, 532 respectively). As Figure 5C depicted, the LiDAR system 500 receives the returned optical pulses 508, 542, 548 (corresponding to the emitted optical pulses 502, 522, 530 respectively). The returned optical pulses 508, 542, and 548 are formed by scattering or reflecting the emitted optical pulses by one of the objects 506 and 514. The returned optical pulses 508, 542, and 548 can return to the LiDAR system 500 along the optical paths 510, 544, and 546 respectively. Based on the direction of the emitted optical pulses (as determined by the LiDAR system 500) and the calculated distance from the LiDAR system 500 to the part of the object that scatters or reflects the optical pulse (e.g., parts of the objects 506 and 514), the external environment within the detectable range (e.g., the field of view between the paths 504 and 532, including) can be accurately mapped or drawn (e.g., by generating a 3D point cloud or an image).

[0103] If no corresponding optical pulse is received for a particular emitted optical pulse, the LiDAR system 500 can determine that there is no object within the detectable range of the LiDAR system 500 (e.g., the object is beyond the maximum scan distance of the LiDAR system 500). For example, in Figure 5B , the optical pulse 526 may not have a corresponding returned optical pulse (as Figure 5C illustrated), because the optical pulse 526 may not produce a scattering event along its transmission path 528 within a pre-determined detection range. The LiDAR system 500 or an external system communicating with the LiDAR system 500 (e.g., a cloud system or service) can interpret the absence of a returned optical pulse as there being no object along the optical path 528 within the detectable range of the LiDAR system 500.

[0104] InFigure 5B In this case, the optical pulses 502, 522, 526, and 530 can be emitted in any order, serially, in parallel, or based on other timings relative to each other. Additionally, although Figure 5B the emitted optical pulses are depicted as being guided in one dimension or one plane (e.g., the plane of the paper), the LiDAR system 500 can also guide the emitted optical pulses along other dimensions or planes. For example, the LiDAR system 500 can also guide the emitted optical pulses in a dimension or plane perpendicular to Figure 5B the dimension or plane shown, thereby forming a two-dimensional transmission of optical pulses. This two-dimensional transmission of optical pulses can be point-by-point, line-by-line, all-at-once, or otherwise. That is, the LiDAR system 500 can be configured to perform point scanning, line scanning, non-scanning single scans, or combinations thereof. A point cloud or image from a one-dimensional transmission of optical pulses (e.g., a single horizontal line) can generate two-dimensional data (e.g., (1) data from the horizontal transmission direction and (2) the range or distance to the object). Similarly, a point cloud or image from a two-dimensional transmission of optical pulses can generate three-dimensional data (e.g., (1) data from the horizontal transmission direction, (2) data from the vertical transmission direction, and (3) the range or distance to the object). Generally, a LiDAR system that performs an n-dimensional transmission of optical pulses generates (n + 1)-dimensional data. This is because the LiDAR system can measure the depth or distance to an object, which provides an additional data dimension.

[0105] Therefore, 2D scans performed by the LiDAR system can generate a 3D point cloud for mapping the external environment of the LiDAR system.

[0106] The density of the point cloud refers to the number of measurements (data points) for each region performed by the LiDAR system. The point cloud density is related to the LiDAR scan resolution.

[0107] Typically, at least for a region of interest (ROI), a greater point cloud density is desired, and thus a higher resolution is required. The point density in the point cloud or image generated by a LiDAR system is equal to the number of pulses divided by the field of view. In some embodiments, the field of view may be fixed. Thus, to increase the density of points generated by a set of transmit-receive optics (or transceiver optics), the LiDAR system may need to generate pulses more frequently. In other words, the light source in the LiDAR system can have a higher pulse repetition rate (PRR). On the other hand, by generating and transmitting pulses more frequently, the maximum distance that the LiDAR system can detect may be limited. For example, if a return signal from a distant object is received after the system transmits the next pulse, the return signal may be detected in an order different from the order in which the corresponding signal was transmitted, resulting in ambiguity if the system cannot correctly associate the return signal with the transmitted signal.

[0108] For illustration, consider an exemplary LiDAR system that can emit laser pulses with a pulse repetition rate between 500 kHz and 1 MHz. Based on the time it takes for the pulse to return to the LiDAR system, and to avoid confusion of return pulses from consecutive pulses in a typical LiDAR design, for 500 kHz and 1 MHz, the maximum distances that the LiDAR system can detect can be 300 meters and 150 meters, respectively. The point density of a LiDAR system with a repetition frequency of 500 kHz is half that of 1 MHz. Thus, this example shows that increasing the repetition rate from 500 kHz to 1 MHz (and thus increasing the point density of the system) may decrease the detection range of the system if the system cannot correctly associate the return signals that arrive out of order. Various techniques are used to mitigate the trade-off between a higher PRR and a limited detection range. For example, multiple wavelengths can be used to detect objects in different ranges. Optical and / or signal processing techniques (e.g., pulse coding techniques) are also used to correlate the transmitted optical signal with the returned optical signal.

[0109] The various systems, devices, and methods described herein can be implemented using digital circuits or using one or more computers that utilize well-known computer processors, memory units, storage devices, computer software, and other components. Generally, a computer includes a processor for executing instructions and one or more memories for storing the instructions and data. The computer may also include or be coupled to one or more mass storage devices, such as one or more disks, internal hard drives and removable disks, magneto-optical disks, optical disks, etc.

[0110] The various systems, devices, and methods described herein can be implemented using computers operating in a client-server relationship. Generally, in such a system, the client computers are located at a distance from the server computer and interact via a network. The client-server relationship can be defined and controlled by computer programs running on the respective client and server computers. Examples of client computers can include desktop computers, workstations, portable computers, cellular smartphones, tablet computers, or other types of computing devices.

[0111] The various systems, devices, and methods described herein can be implemented using a computer program product tangibly embodied in an information carrier, such as in a non-transitory machine-readable storage device, for execution by a programmable processor; and the method processes and steps described herein (including Figures 1 to 13 one or more of at least some of them) can be implemented using one or more computer programs executable by such a processor. A computer program is a set of computer program instructions that can be used directly or indirectly in a computer to perform a particular activity or produce a particular result. A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0112] Figure 6 A simplified block diagram of an exemplary device that can be used to implement the systems, devices, and methods described herein is illustrated. Device 600 includes a processor 610 operably coupled to a persistent storage device 620 and a main memory device 630. Processor 610 controls the overall operation of device 600 by executing computer program instructions that define these operations. The computer program instructions can be stored in the persistent storage device 620 or other computer-readable media and loaded into the main memory device 630 when it is desired to execute the computer program instructions. For example, processor 610 can be used to implement one or more of the components and systems described herein, such as control circuitry 350 ( Figure 3 as shown), vehicle perception and planning system 220 ( Figure 2 as shown), and vehicle control system 280 ( Figure 2 as shown).

[0113] Thus, Figures 1 to 13 at least some of the method steps can be defined by computer program instructions stored in the main memory device 630 and / or the persistent storage device 620 and controlled by processor 610 executing the computer program instructions. For example, the computer program instructions can be implemented as computer-executable code programmed by those skilled in the art to perform the operations associated hereinFigures 1 to 13 At least some of the method steps discussed herein define algorithms. Accordingly, by executing computer program instructions, the processor 610 executes the algorithms defined by the method steps of these foregoing figures. The apparatus 600 also includes one or more network interfaces 680 for communicating with other devices via a network. The apparatus 600 may also include one or more input / output devices 690 that enable a user to interact with the apparatus 600 (e.g., a display, a keyboard, a mouse, speakers, buttons, etc.).

[0114] The processor 610 may include both a general-purpose microprocessor and a special-purpose microprocessor, and may be the sole processor of the apparatus 600 or one of a plurality of processors. The processor 610 may include one or more central processing units (CPUs) and one or more graphics processing units (GPUs), and the GPUs may, for example, work separately from one or more CPUs and / or perform multitasking with one or more CPUs to accelerate processing, e.g., for various image processing applications described herein. The processor 610, the persistent storage device 620, and / or the main memory device 630 may include one or more application-specific integrated circuits (ASICs) and / or one or more field-programmable gate arrays (FPGAs), or be supplemented by one or more ASICs and / or one or more FPGAs, or incorporated into one or more ASICs and / or one or more FPGAs.

[0115] The persistent storage device 620 and the main memory device 630 each include a tangible non-transitory computer-readable storage medium. The persistent storage device 620 and the main memory device 630 may each include high-speed random access memory, such as dynamic random access memory (DRAM), static random access memory (SRAM), double data rate synchronous dynamic random access memory (DDR RAM), or other random access solid-state memory devices, and may include non-volatile memory, e.g., one or more disk storage devices, such as internal hard disks and removable disks, magneto-optical storage devices, optical disc storage devices, flash memory devices, semiconductor storage devices (such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)), compact disc read-only memory (CD-ROM), digital versatile disc read-only memory (DVD-ROM) discs, or other non-volatile solid-state storage devices.

[0116] The input / output device 690 may include peripheral devices, such as printers, scanners, displays, etc. For example, the input / output device 690 may include a display device (such as a cathode ray tube (CRT), plasma, or liquid crystal display (LCD) monitor, a keyboard) for displaying information to a user, and a pointing device (such as a mouse or trackball) by which the user may provide input to the apparatus 600.

[0117] Any or all of the functions of the systems and devices discussed herein may be performed by a processor 610 and / or incorporated into a device or system such as LiDAR system 300. Additionally, LiDAR system 300 and / or device 600 may utilize one or more neural networks or other deep learning techniques performed by processor 610 or other systems or devices discussed herein.

[0118] Those skilled in the art will recognize that the implementation of an actual computer or computer system may have other structures and may also include other components, and Figure 6 is a simplified representation of some of the components of such a computer for illustrative purposes.

[0119] Figure 7 is a block diagram illustrating an exemplary depth sensor 700 according to some embodiments. Depth sensor 700 includes a light source 710, a transmitter 720, an optical receiver and a photodetector 730, and a control circuit 750. These components may be substantially the same as or similar to the light source 310, transmitter 320, optical receiver and photodetector 330, and control circuit 350 described above with reference to Figure 3 In Figure 7 the communication paths 712, 722, 732, 752, 762, and 772 may also be substantially the same as or similar to the paths 312, 322, 332, 352, 362, and 372 described above, respectively, and thus will not be described again.

[0120] In one embodiment, the light source 710 may include a semiconductor-based laser source (e.g., VCSEL), a fiber-based laser source (e.g., a rare-earth doped fiber for emitting laser light), a liquid-based laser source (e.g., dye lasers such as sodium fluorescein, rhodamine B, and rhodamine 6G), a solid-state laser source (e.g., a laser using a neodymium crystal, typically doped with yttrium aluminum garnet (Nd:YAG), neodymium vanadate (Nd:YVO4), or neodymium lithium fluoride (Nd:YLF)), and / or a gas-based laser source (e.g., a carbon dioxide (CO2)-based laser, an argon-based laser, or a helium-neon-based laser). In the examples described below, a VCSEL is used for illustration. However, it should be understood that other types of laser sources may also be used.

[0121] The optical receiver and photodetector 730 may include any type of photodetector, such as a photodiode, an avalanche photodiode (APD), a SPAD, a phototransistor, a charge-coupled device

[0122] (CCD), CMOS image sensor (CIS) and / or photomultiplier tube (PMT). In the examples described below, a high-sensitivity light detector or a detector similar to an SPAD array is used as an illustrative example.

[0123] Compared with the LiDAR system 300, the depth sensor 700 does not have a steering mechanism or any other mechanically movable scanning optics. Therefore, the depth sensor 700 eliminates any mechanically movable parts configured to scan light. The depth sensor 700 can thus be more compact, robust, durable, and reliable. In one example, the depth sensor 700 is a flash LiDAR that emits a laser to illuminate the entire FOV in a single pulse or single shot. The depth sensor 700 can be a solid-state LiDAR device configured to perform electronic scanning. Compared with optical scanning, electronic scanning does not use a mechanically movable optical system to scan light. Instead, a solid-state LiDAR device can use phase-based scanning, which emits a constant laser beam into multiple phases. Then, it compares the phase shift of the returned laser energy. The laser scanner uses a phase-shift algorithm to determine distance, which is based on the unique properties of each individual phase and is based on the following formula: (time of flight = phase shift / (2π × modulation frequency)). Phase-based scanners can collect data much faster than time-of-flight scanners using mechanical scanning, but their effective detection range may be shorter. Additionally, phase-based scanners may sometimes have more "noise" or false data than time-of-flight scanners. For electronic scanning, in one example, it includes a matrix of light sources and detectors. Each light source has its own column and row index. The excitation order of the light matrix can be programmed and controlled. Each detector can collect the returned light from an object. The light sources and detectors are optically matched to each other. Each detector calculates the time of flight by itself. Then a depth image can be created.

[0124] In some embodiments, the depth sensor 700 can be a flash LiDAR. As described above, when the flash LiDAR operates, the entire field of view is illuminated with a wide-divergence laser beam in a single pulse. In a scanning LiDAR (e.g., Figure 3 the LiDAR system 300 shown), the collimated laser beam scanned by the steering mechanism 340 illuminates a single point in the FOV at a time, and the beam is raster-scanned to illuminate the FOV point by point and row by row. Compared with the scanning LiDAR illumination method, the flash LiDAR illumination method requires a different detection scheme. In both scanning and flash LiDAR systems, light detectors and a time-of-flight engine are used to collect and process data related to the three-dimensional position and intensity of the returned light incident on the light detector in each frame.

[0125] In a scanning LiDAR, the light detector can include point sensors, while in a flash LiDAR, the light detector includes a one- or two-dimensional sensor array, and each pixel in the one- or two-dimensional sensor array collects three-dimensional position and intensity information. In both cases, depth information is calculated using a time-of-flight engine based on the emitted laser pulses and the returned light (i.e., the time it takes for each laser pulse to hit the target object and return to the sensor). The result is a point cloud that includes distance information of the target object. Compared with scanning LiDAR, flash LiDAR is particularly advantageous when the sensor, the FOV, or both are moving because the entire FOV is illuminated simultaneously.

[0126] In some embodiments, the depth sensor 700 can be an iToF sensor that uses the iToF method to measure the distance to a target object. The iToF method measures distance by collecting the returned light and discerning the phase shift between the emitted light and the returned light. The iToF method is particularly effective for high-speed, high-resolution 3D imaging of objects located at short and long distances. A depth sensor based on indirect ToF emits continuous modulated light and measures the phase of the returned light to calculate the distance to the target object.

[0127] As Figure 7 shown, in some embodiments, the light source 710 emits a light beam towards the emitter 720, and the emitter provides light to illuminate the FOV. The emitter 720 can include one or more optical structures configured to distribute the light beam to the FOV. In some embodiments, the light source 710 can directly illuminate the FOV, and thus the depth sensor 700 may not include the emitter 720. Figure 8 is a block diagram illustrating an exemplary depth sensor 800 according to some embodiments. The depth sensor 800 can be used to implement Figure 7 the depth sensor 700 shown. Referring to Figure 8 , in this example, the depth sensor 800 includes a VCSEL laser array 810 as the light source, an SPAD array 830 as one or more light detectors, a time-of-flight engine 850, emission optics 820, and reception optics 840. For simplicity, Figure 8 other components are omitted in

[0128] As Figure 8As shown, in one embodiment, the VCSEL laser array 810 emits a laser beam 832 towards the emission optical device 820, which may include one or more of a lens, a lens group, a mirror, a prism, a microlens, a diffuser, or any other optical device. The emission optical device 820 may include optical structures configured to receive the beam emitted from the VCSEL laser array 810 and transmit the beam as a transmitted beam 832 into the FOV. The emission optical device 820 may be Figure 7 a part of the emitter 720 shown below. Examples of the optical structures are described in more detail below. In one example, the receiving optical device 840 may include a collection lens or lens group, an optical fiber or fiber array, a filter, one or more focusing lenses, one or more beam splitters, or other light separation devices, or a combination thereof, for collecting the return light 852 and guiding it to the SPAD array 830 in the depth sensor 800. As described above, the SPAD array 830 may include highly sensitive light detectors configured to convert the photons detected in the return light 852 into an electrical signal. The electrical signal may be provided to the time-of-flight engine 850 for processing. For example, the time-of-flight engine 850 may use the time and / or phase information associated with the return light 852 and the time and / or phase associated with the transmitted beam 832 (or reference beam) to calculate the distance to the target object 870. The time-of-flight engine 850 may be a part of the control circuit 750 (or control circuit 350) described above. It may include one or more processors and programs for calculating the distance to the target object 870 based on the dToF method and / or iToF method described above. In one embodiment, the depth sensor 800 is a flash LiDAR or iToF sensor.

[0129] Figure 9 An exemplary depth sensor 900 that provides a non-uniformly distributed beam in the vertical direction of the FOV according to some embodiments is illustrated. In one example, the depth sensor 900 may include a light source 902, an optical structure 904, and a receiver 906. The light source 902 may be substantially the same as or similar to the light source 710 or the VCSEL laser array 810. The optical structure 904 may include optical components such as one or more optical diffusers, a microlens array, and / or other optical components (e.g., lenses, mirrors, prisms, etc.). The optical structure 904 may be used to form the above-described emission optical device 820 and will be described in more detail below. The receiver 906 may include a receiving optical device (e.g., substantially the same as or similar to the receiving optical device 840) and a light detector (e.g., a SPAD array). The receiver 906 is configured to receive and detect the return light formed by scattering or reflecting the transmitted beam by an object in the FOV.

[0130] As Figure 9As shown, at least one of one or more light sources 902 or one or more optical structures 904 is configured to non-uniformly distribute a plurality of light beams in a vertical field of view (FOV) such that the vertical FOV includes a dense region and a sparse region. The dense region of the vertical FOV has a higher light beam density than the sparse region of the vertical FOV.

[0131] For example, Figure 9 The figures illustrate that the light beams provided by the light source 902 and / or the optical structure 904 include light beams 932 and 942. The light beam 932 is distributed more densely than the light beam 942. The light beam 932 is used to illuminate and detect objects located in the forward direction, which are within a vertical angular range of, for example, at least one of approximately -5 degrees to 0 degrees or -5 degrees to +5 degrees. One such object 970 is shown in Figure 9 . The light beam 942 is used to illuminate and detect objects located in the forward direction, which are within a vertical angular range of, for example, at least one of -90 degrees to -5 degrees or +5 degrees to +90 degrees. Figure 9 The sparse region shown is part of the region covered by the light beam 942. For example, Figure 9 The complete vertical angular ranges of -90 degrees to -5 degrees and +5 degrees to +90 degrees are not shown, and they are also part of the sparse region. The vertical FOV covered by both the light beam 932 and the light beam 942 can thus be from -90 degrees to +90 degrees.

[0132] As Figure 9 shown, in at least a portion of the vertical angular range of the FOV, the light beams are non-uniformly distributed. Within the dense region (e.g., -5 degrees to 0 degrees, or -5 degrees to +5 degrees), the light beams are dense. Within the sparse region (e.g., -90 degrees to -5 degrees, or +5 degrees to +90 degrees), the light beams are sparser or less dense than the light beams in the dense region. The non-uniform distribution of the light beams optimizes the light distribution and object detection within different detection ranges with satisfactory resolution. The optimization of the detection range with non-uniform light distribution is described in more detail using Figure 10 .

[0133] Figure 10 is a block diagram illustrating the variation of distance detection requirements according to the transmission light angle in the vertical FOV according to some embodiments. As Figure 10 shown, two exemplary depth sensors 1000A and 1000B are mounted on a vehicle 1090. The depth sensor 1000A or 1000B can be implemented using any one of the above-described depth sensors 700, 800, and 900. Figure 10 The depth sensor 1000B is used as an illustration of the distance detection requirements. The depth sensor 1000B is mounted on the vehicle 1090 at a height of approximately, for example, 0.84 m to 1.1 m above the road surface 1002. As Figure 10As shown, if the light beam is transmitted at a vertical angle of 0 degrees (or within a range of approximately -5 degrees to +5 degrees), the detection range of the light beam can reach a long distance, such as 50m to 150m. Such a light beam with a vertical angle of 0 degrees propagates in a direction substantially parallel to the road surface 1002. The vertical angle of the light beam emitted by the depth sensor refers to the angle of the light beam in the vertical direction (e.g., the angle between the light beam and a line parallel to the road surface or the mounting surface of the depth sensor). The vertical direction is generally perpendicular to the road surface 1002, and the horizontal direction is generally parallel to the road surface 1002.

[0134] As Figure 10 As further shown, if the light beam is transmitted at a vertical angle of -5 degrees, the detection range of the light beam in the horizontal direction is significantly reduced to approximately 11.4m before the light beam reaches the road surface. Similarly, if the light beam is transmitted at -15 degrees, -30 degrees, -45 degrees, and -60 degrees, the detection range is further reduced to 3.9m, 2m, 1.4m, and 1.2m respectively (equivalent to 3.7m, 1.7m, 1.0m, and 0.6m in the horizontal direction) before the light beam reaches the road surface 1002. It can be seen that as the vertical angle of the light beam emitted by the depth sensor 1000B increases (in the negative vertical direction), the detection range of the depth sensor 1000B in the horizontal direction rapidly shortens. In other words, as the vertical angle of the light beam increases (e.g., from -5 degrees to -90 degrees), the distance traveled by the light beam before reaching the road surface decreases. Therefore, compared with the light beam emitted between -5 degrees and +5 degrees, the light beam emitted at certain vertical angles (e.g., between -5 degrees and -90 degrees) cannot and does not need to be used for remote detection. Figure 10 It can be seen that as the vertical angle of the light beam emitted by the depth sensor 1000B increases (in the negative vertical direction), the detection range of the depth sensor 1000B in the horizontal direction rapidly shortens. In other words, as the vertical angle of the light beam increases (e.g., from -5 degrees to -90 degrees), the distance traveled by the light beam before reaching the road surface decreases. Therefore, compared with the light beam emitted between -5 degrees and +5 degrees, the light beam emitted at certain vertical angles (e.g., between -5 degrees and -90 degrees) cannot and does not need to be used for remote detection.

[0135] Although Figure 10 not shown, the light beam emitted at a vertical angle between +5 degrees and +90 degrees also cannot be used for detecting distant objects because generally these light beams will be emitted into the sky, and the sky has no reflection or the minimum reflection.

[0136] Returning to the reference Figure 9 , the light beams emitted at different vertical angles can be used to detect objects located within different detection ranges. For example, the light beam 932 transmitted in a dense area is directed to detect objects located within the first detection range, and the light beam 942 transmitted in a sparse area is directed to detect objects located within the second detection range. The first detection range can include, for example, a distance measured 50 meters or more from the depth sensor 900. The second detection range can include, for example, a distance from 0 meters to 20 meters measured from the depth sensor 900. The vertical angle at which the light beam 932 is transmitted can be within a range of, for example, -5 degrees to +5 degrees. And the vertical angle at which the light beam 942 is emitted can be, for example, from -5 degrees to -90 degrees. As described above, and as Figure 9As shown, the light beam 932 is used to detect an object (e.g., object 970) that may be located at a long distance (e.g., greater than 50 meters) from the depth sensor 900. To detect such an object far from the depth sensor 900, the sensing resolution of the depth sensor 900 may need to be high. Resolution refers to the level of detail that the depth sensor 900 can capture. For a LiDAR system (or other depth sensor), the resolution can be expressed by the number of points in a point cloud, which is the number of spatial units or pixels per unit area. Thus, the higher the number of points or pixels, the higher the resolution of the depth sensor.

[0137] When an object (e.g., object 970) is located far from the depth sensor 900 (e.g., 50 meters to 200 meters or more), the depth sensor 900 needs to have a high resolution to detect the object because the object looks very small from a distance. Thus, if the light beams used to detect such a distant object are sparse, the object may not be detected or may have a low-resolution detection because no or few light beams may hit the object. Consequently, there may be no return light, or there may be only very little return light. Therefore, to detect such a distant object, the depth sensor 900 needs to emit light beams with a high beam density. Beam density refers to the number of light beams per unit vertical angle (e.g., 1 degree) or per unit area / volume. The higher the beam density, the more light beams there are per unit vertical angle or per unit area / volume. In Figure 9 it, the beam density in the dense region is greater than that in the sparse region. The absolute values of the beam density in the dense region and the sparse region may depend on the detection distance and the object size. As Figure 9 shown, the light beam 932 has a high beam density and can thus be used to detect an object located in a far detection range (e.g., 50 meters or more). In contrast, the depth sensor 900 does not need to have a high beam density to detect an object (e.g., object 972) located near it with good resolution. As Figure 9 shown, even if the light beam 942 has a lower beam density than the light beam 932, they can be used to detect an object located near the depth sensor 900 (e.g., within 0 meters to 20 meters). This is because the object located near the depth sensor 900 looks large, and thus, even if the light beam 942 is sparse, the object can still be detected.

[0138] Figure 9 and Figure 10It is shown that in order to detect an object at a far detection range corresponding to a dense region within a vertical angular range, a dense beam should be used; and in order to detect an object at a near detection range corresponding to a sparse region within the vertical angular range, a sparse beam can be used. Thus, the depth sensor 900 can be configured to emit beams that are unevenly distributed in the vertical field of view, such that the vertical FOV has a dense region and a sparse region, where the dense region of the vertical FOV has a higher beam density than the sparse region of the vertical FOV. The uneven distribution of the beams along the vertical FOV can optimize the detection range and reduce energy consumption while still meeting the detection resolution requirements for objects in different detection ranges. Compared with using beams with a uniform distribution, a depth sensor configured to provide an uneven distribution of beams improves the overall system efficiency and performance.

[0139] As described above, the beams provided by the depth sensor (e.g., sensors 800, 900) can be directly provided by one or more light sources, or provided by a combination of a light source and one or more optical structures. Figure 11 is a block diagram illustrating an example of providing an uneven distribution of beams by unevenly placing VCSEL elements in the VCSEL laser array 1110 according to some embodiments. As Figure 11 shown, in this embodiment, no optical structure may be required to provide an uneven distribution of beams. The VCSEL laser array 1110 includes a plurality of VCSEL elements 1115A - 1115N and 1117A - 1117M. These VCSEL elements can form an array (e.g., a 1D or 2D array, an array or a matrix). The VCSEL elements 1115A - 1115N can be arranged close to each other (e.g., forming a densely arranged array at a predetermined distance), such that the beams 1132 emitted by the VCSEL elements 1115A - 1115N have a high beam density. The beams 1132 are thus distributed in the dense region of the vertical FOV. In contrast, the VCSEL elements 1117A - 1117M can be arranged sparsely from each other (e.g., arranged at another predetermined distance to form a sparsely arranged array). Thus, the beams 1142 emitted by the elements 1117A - 1117M are distributed in the sparse region of the vertical FOV. Each of the VCSEL elements 1115A - 1115N and 1117A - 1117M can also have a predetermined orientation such that it guides the corresponding beam at a corresponding vertical angle. For example, the element 1117A can be tilted at a certain vertical angle such that its beam has a vertical angle of -5 degrees in the vertical FOV. The element 1117B can be tilted at another vertical angle such that its beam has a vertical angle of -10 degrees, and so on.

[0140] In Figure 11In the configuration shown, by arranging the VCSEL elements 1115A - 1115N and 1117A - 1117M differently, a non-uniform distribution of the light beams 1132 and 1142 can be obtained, where the light beam 1132 has a higher beam density than the light beam 1142. The VCSEL laser array 1110 may not require an additional optical structure to provide the non-uniform distribution. In some embodiments, the emission optical device 1120 can be coupled to the VCSEL laser array 1110 to further shape the light beam or further assist or fine-tune the non-uniform distribution of the light beam. Such an emission optical device 1120 can be, for example, a lens or a lens group. In one example, the emission optical device 1120 can be a collimating lens.

[0141] The above Figure 11 illustrates providing a non-uniform distribution of a light beam through a non-uniform arrangement of light source elements (e.g., VCSEL elements). Figure 12 is a block diagram illustrating providing a non-uniform distribution of a light beam by using an optical diffuser 1224 according to some embodiments. As Figure 12 shown, the depth sensor 1200 includes a light source 1220. The light source 1220 has an array of uniformly distributed elements (e.g., VCSEL elements). The light beams 1222 emitted by these elements of the light source 1220 are thus also uniformly distributed. The optical diffuser 1224 forms an optical structure that can be used to produce a non-uniform distribution of the light beam 1222, thereby forming a non-uniformly distributed light beam 1226. The optical diffuser 1224 can be a device or element for scattering or diffusing light. Its main purpose is to produce a non-uniform distribution of light by decomposing at least a portion of the incident light beam 1222 into a wider and lower-intensity illumination pattern. As Figure 12 shown, the optical diffuser 1222 can change the direction of some light beams more than other light beams, and / or reshape some of the light beams in the light beam to produce a non-uniform distribution of the light beam. The optical diffuser can diffuse light by disturbing the light wavefront and reducing its spatial coherence. Thus, the optical diffuser can obtain a change in the optical phase of different parts of the profile of the incident light beam. The optical diffuser 1224 can be made of various materials, including glass, plastic, and film. They can be customized to scatter the incident light in a specific pattern or texture. The complexity of these patterns varies, from a simple rough surface to a more complex micro-structured design. The optical diffuser 1224 can thus include a surface with micro-optical structures that are configured to receive a uniformly distributed light beam and form a non-uniform distribution of the light beam. The formation of the non-uniform distribution of the light beam can be precisely controlled (by using micro-optical patterns). The optical diffuser 1224 can thus achieve a specific illumination effect and improve the illumination quality of the depth sensor. The choice of diffuser type and design depends on the specific requirements of the depth sensor, including the desired diffusion level and the expected illumination effect.

[0142] Figure 13 is a block diagram illustrating an uneven distribution of light beams provided by using a semiconductor wafer 1323 with a microlens array 1324 according to some embodiments. Similar to Figure 12 the configuration in Figure 13 the depth sensor 1300 shown in Figure 13 includes a light source 1320 having an array of elements (e.g., VCSEL elements) with a uniform distribution. The light beams 1322 emitted by these elements of the light source 1320 are thus also uniformly distributed.

[0143] The semiconductor wafer 1323, also simply referred to as wafer 1323, is a thin, flat, and generally circular slice of semiconductor material, such as silicon, which serves as a substrate for manufacturing electrical and / or optical devices (such as microlens arrays). The wafer 1323 can be silicon-based (e.g., silicon, silicon carbide) or based on other semiconductor materials (e.g., gallium nitride-based). The semiconductor wafer 1323 is transparent to light beams 1322 of a specific wavelength or wavelength range, such that the light beams 1322 can pass through the wafer 1323 and enter the microlens array 1324. In other words, the light beams 1322 can enter from the back side of the wafer 1323 and exit through the microlens array 1324 from the front side. This configuration is also referred to as a back-illumination technique. For example, a silicon-based wafer is transparent to light beams with a wavelength of 905 nm. In some embodiments, the elements of the light source 1320 (e.g., VCSEL elements) can also be disposed on one surface (e.g., the back surface) of the wafer 1323; and the microlens array 1324 can be disposed on the other surface (e.g., the front surface) of the wafer 1323. Thus, the depth sensor 1300 is highly integrated and can be very compact. In other embodiments, the elements of the light source 1320 can be separate and different from the wafer 1323.

[0144] The microlenses in the array 1324 are micro-lenses with very small sizes, typically on the order of micrometers (μm) or even smaller. The microlenses can thus be much smaller than conventional lenses, and thus the microlenses can be easily disposed in the semiconductor wafer, making the entire sensor very compact. The microlenses in the array 1324 can be made of various materials, including glass, polymers, or semiconductor materials. The choice of material depends on the type of the wafer 1323 and the specific optical requirements. As Figure 13As shown, the microlenses in array 1324 can shape and redirect the light beam 1322. When the light beam passes through a specific microlens in array 1324, the light beam may or may not change its direction. For example, when the topmost light beam 1322 passes through microlens 1324A, the light beam can maintain its direction. When other light beams pass through their corresponding microlenses 1324B - 1324N, their directions can be changed to form a set of light beams in the dense region of the vertical FOV and another set of light beams in the sparse region of the vertical FOV.

[0145] Thus, in one embodiment, each different microlens can be configured differently to bend the corresponding incident light beam to its desired direction (or corresponding vertical angle). For example, in Figure 13 , microlens 1324B is designed to bend the light beam slightly downward, so that the output light beam from microlens 1324B is directed at a vertical angle of, for example, -5 degrees. In contrast, microlens 1324N is designed to bend the light beam significantly downward, such that the output light beam from microlens 1324N is oriented at a vertical angle of, for example, -45 degrees.

[0146] Although Figure 13 the figure illustrates that each of the microlenses 1324A - 1324N is configured to distribute one of the light beams 1322, it should be understood that in other embodiments, a subset of the microlenses of array 1324 can be configured to distribute one light beam. For example, a group of two, three, four, or more microlenses 1324 can be arranged together to receive one light beam 1322 and redistribute the light beam to provide an output light beam 1326. Multiple microlenses can form a sub - array (1D or 2D) or subgroup placed in position such that the sub - array or subgroup of microlenses can receive the incident light beam.

[0147] The microlens array 1324 can be fabricated on the semiconductor wafer 1323 via various semiconductor processing techniques. In one example, the surface of the semiconductor wafer 1323 can be processed to form the microlens array 1324 by removing material from the surface to form microlenses. Removing material (e.g., silicon, oxide, metal, etc.) from the wafer 1323 can be performed via lithography (e.g., for patterning), chemical etching (e.g., dry etching or wet etching), and / or precision machining (e.g., chemical mechanical polishing). In another example, the surface of the semiconductor wafer 1323 is processed to form the microlens array 1324 by depositing material onto the surface to form microlenses. The deposited material can include, for example, polymer materials, silicon materials, glass materials, plastic materials, etc. Deposition techniques can include physical vapor deposition (PVD), chemical vapor deposition (CVD), atomic layer deposition (ALD), electrochemical deposition, spin coating, sputtering, chemical solution deposition, etc. As an example, tiny droplets of a polymer can be deposited onto the surface of the wafer 1323 to form microlenses through subsequent heat treatment.

[0148] Figure 14 Method 1400 for unevenly distributing a plurality of light beams using a depth sensor according to some embodiments is illustrated. The depth sensor does not include a mechanically movable part for scanning the light beams. Method 1400 includes step 1402, in which one or more light sources emit a plurality of light beams. In step 1404, the plurality of light beams are received by one or more optical structures coupled to the one or more light sources. In step 1406, at least one of the one or more light sources or the one or more optical structures unevenly distributes the plurality of light beams in a vertical field of view (FOV) such that the vertical FOV includes a dense region and a sparse region. The dense region of the vertical FOV has a higher light beam density than the sparse region of the vertical FOV. In some embodiments, when the light beams are unevenly distributed, the light beams in the dense region of the vertical FOV are directed to detect an object located in a first detection range, and the light beams in the sparse region of the vertical FOV are directed to detect an object located in a second detection range. The first detection range is greater than the second detection range. In some examples, the first detection range includes a distance of 50 meters or more from the depth sensor, and the second detection range includes a distance of 0 meters to 20 meters from the depth sensor. In some examples, the dense region of the vertical FOV corresponds to a vertical angle range of -5 degrees to 0 degrees or -5 degrees to +5 degrees; and the sparse region of the vertical FOV corresponds to at least one of the vertical angle ranges of -90 degrees to -5 degrees or +5 degrees to +90 degrees.

[0149] The foregoing specification is to be considered in all respects as illustrative and not restrictive, and the scope of the invention disclosed herein is not determined by the specification, but by the claims as interpreted to the full extent permitted by patent law. It should be understood that the embodiments shown and described herein are merely illustrative of the principles of the invention and that various modifications can be made by those skilled in the art without departing from the scope and spirit of the invention. Various other combinations of features can be effected by those skilled in the art without departing from the scope and spirit of the invention.

Claims

1. A depth sensor, the depth sensor comprising: One or more light sources configured to provide a plurality of light beams; One or more optical structures coupled to the one or more light sources and configured to receive the plurality of light beams, wherein: At least one of the one or more light sources or the one or more optical structures is configured to non-uniformly distribute the plurality of light beams in a vertical field of view (FOV) such that the vertical FOV includes a dense region and a sparse region, and The dense region of the vertical FOV has a higher light beam density than the sparse region of the vertical FOV, and Wherein the depth sensor does not include a mechanically movable part configured to scan light.

2. The depth sensor according to claim 1, wherein, The depth sensor includes a solid-state light ranging and detection (LiDAR) device configured to perform electronic scanning.

3. The depth sensor according to claim 1 or 2, wherein, The depth sensor includes at least one of a flash LiDAR device or an indirect time-of-flight (iToF) sensor.

4. The depth sensor according to any one of claims 1 to 3, wherein, The one or more light sources include one or more of a semiconductor-based laser source, an optical fiber-based laser source, a liquid-based laser source, a solid-state-based laser source, and a gas-based laser source.

5. The depth sensor according to any one of claims 1 to 4, wherein: The light beams in the dense region of the vertical FOV are directed to detect an object located within a first detection range, The light beams in the sparse region of the vertical FOV are directed to detect an object located within a second detection range, the first detection range being greater than the second detection range.

6. The depth sensor according to claim 5, wherein, The first detection range includes a distance of 50 meters or more from the depth sensor, and wherein the second detection range includes a distance of 0 meters to 20 meters from the depth sensor.

7. The depth sensor according to any one of claims 1 to 6, wherein: The dense region of the vertical FOV corresponds to a vertical angle range of -5 degrees to 0 degrees or -5 degrees to +5 degrees; and The sparse region of the vertical FOV corresponds to a vertical angle range of at least one of -90 degrees to -5 degrees or +5 degrees to +90 degrees.

8. The depth sensor according to any one of claims 1 to 7, wherein: The one or more light sources include a VCSEL array having an array of vertical cavity surface emitting laser (VCSEL) elements, The VCSEL elements are configured to be non-uniformly distributed such that the corresponding light beams in the plurality of light beams are non-uniformly distributed in the vertical FOV.

9. The depth sensor according to any one of claims 1 to 8, wherein: The one or more optical structures include one or more optical diffusers configured to non-uniformly distribute the plurality of light beams in the vertical FOV.

10. The depth sensor according to claim 9, wherein: The plurality of light beams include light beams that are uniformly distributed before the one or more optical diffusers, and The one or more optical diffusers include a surface having a micro-optical structure configured to receive the uniformly distributed light beams and form a non-uniform distribution of the light beams.

11. The depth sensor according to any one of claims 1 to 10, wherein, The one or more optical structures include a semiconductor wafer having a microlens array configured to non-uniformly distribute the plurality of light beams in the vertical FOV.

12. The depth sensor according to claim 11, wherein, The semiconductor wafer is a silicon-based wafer.

13. The depth sensor according to claim 11 or 12, wherein, A subset of the microlenses of the microlens array is configured to distribute one of the plurality of light beams.

14. The depth sensor according to any one of claims 11 to 13, wherein, The surface of the semiconductor wafer is processed to form the microlens array by removing material from the surface to form the microlens.

15. The depth sensor according to any one of claims 11 to 14, wherein, The surface of the semiconductor wafer is processed to form the microlens array by depositing material onto the surface to form the microlens.

16. The depth sensor according to claim 15, wherein, The material deposited onto the surface includes a polymeric material.

17. A method of unevenly distributing light beams using a depth sensor, the depth sensor not including a mechanically movable part for scanning the light beams, the method comprising: Multiple light beams are emitted by one or more light sources; The multiple light beams are received by one or more optical structures coupled to the one or more light sources; and the multiple light beams are unevenly distributed in a vertical field of view (FOV) by at least one of the one or more light sources or the one or more optical structures such that the vertical FOV includes a dense region and a sparse region, wherein the dense region of the vertical FOV has a higher beam density than the sparse region of the vertical FOV.

18. The method according to claim 17, wherein, Unevenly distributing the multiple light beams includes: directing light beams into the dense region of the vertical FOV to detect an object located within a first detection range; and directing light beams into the sparse region of the vertical FOV to detect an object located within a second detection range, the first detection range being greater than the second detection range.

19. The method according to claim 18, wherein, The first detection range includes a distance of 50 meters or more from the depth sensor, and wherein the second detection range includes a distance of 0 meters to 20 meters from the depth sensor.

20. The method according to any one of claims 17 to 19, wherein: The dense region of the vertical FOV corresponds to a vertical angle range of -5 degrees to 0 degrees or -5 degrees to +5 degrees; and The sparse region of the vertical FOV corresponds to a vertical angle range of at least one of -90 degrees to -5 degrees or +5 degrees to +90 degrees.

21. A light detection and ranging (LiDAR) system, the LiDAR system including the depth sensor according to any one of claims 1 to 16.

22. A vehicle, the vehicle including a LiDAR system having the depth sensor according to any one of claims 1 to 16.

Images