Time-Division Multiplexed Access Scanning for Crosstalk Reduction in a Light Detection and Ranging (LIDAR) Device

Abstract

To provide a technique related to time-division multiple access scanning for crosstalk mitigation in light detection and ranging (LIDAR) devices.SOLUTION: An example embodiment includes a method. The method includes emitting a first group of light signals into a surrounding environment. The first group of light signals corresponds to a first angular resolution. The method also includes detecting, during a first listening window, a first group of reflected light signals. Additionally, the method includes emitting a second group of light signals into the surrounding environment. The second group of light signals corresponds to a second angular resolution with respect to the surrounding environment. The second angular resolution is lower than the first angular resolution. Further, the method includes detecting a second group of reflected light signals from the surrounding environment. In addition, the method includes synthesizing, by a controller of the LIDAR device, a dataset usable to generate one or more point groups.SELECTED DRAWING: Figure 10

Description

Background Art

[0001] Unless otherwise stated in this specification, the descriptions in this chapter are not prior art for the claims of this application and should not be considered prior art by inclusion in this chapter.

[0002] Autonomous vehicles or vehicles operating in autonomous mode can detect their surroundings using various sensors. For example, a Light Detection and Ranging (LIDAR) device, a Radio Detection and Ranging (RADAR) device, and / or a camera can be used to identify objects in the surrounding environment of an autonomous vehicle. Such sensors can be used, for example, for object detection and avoidance and / or navigation.

Summary of the Invention

[0003] The embodiments described herein relate to reducing crosstalk between detection channels in a LIDAR device. In particular, an exemplary embodiment includes performing two emission cycles using two corresponding detection cycles. During the first cycle, all the optical emitters within the LIDAR device emit optical signals, and the photodetectors listen for reflected optical signals for a duration corresponding to a relatively long distance (e.g., 300 m to 450 m). However, during the second cycle, only a subset of the optical emitters within the optical device emit optical signals (potentially sequentially), and the corresponding photodetectors listen for reflected optical signals for a duration corresponding to a relatively short distance (e.g., 45 m to 75 m). By comparing the distances represented by the optical signals detected between the two cycles, signals corresponding to crosstalk can be identified and / or removed from the resulting dataset.

[0004] In a first aspect, a method is provided. The method includes emitting, from a first group of optical emitters of a light detection and ranging (LIDAR) device, a first group of optical signals into the surrounding environment. The first group of optical signals corresponds to a first angular resolution with respect to the surrounding environment. The method also includes detecting, by a first group of photodetectors of the LIDAR device, a first group of reflected optical signals from the surrounding environment during a first listening window. The first group of reflected optical signals corresponds to the reflection of the first group of optical signals from objects in the surrounding environment. Further, the method includes emitting, from a second group of optical emitters of the LIDAR device, a second group of optical signals into the surrounding environment. The second group of optical emitters of the LIDAR device represents a subset of the first group of optical emitters of the LIDAR device. The second group of optical signals corresponds to a second angular resolution with respect to the surrounding environment. The second angular resolution is lower than the first angular resolution. Further, the method includes detecting, by a second group of photodetectors of the LIDAR device, a second group of reflected optical signals from the surrounding environment during a second listening window. The second group of photodetectors of the LIDAR device represents a subset of the first group of photodetectors of the LIDAR device. The second group of reflected optical signals corresponds to the reflection of the second group of optical signals from objects in the surrounding environment. The duration of the second listening window is shorter than the duration of the first listening window. Further, the method includes synthesizing, by a controller of the LIDAR device, a data set that can be used to generate one or more point clouds. The data set is based on the detected first group of reflected optical signals and the detected second group of reflected optical signals.

[0005] In a second aspect, a light detection and ranging (LIDAR) device is provided. The LIDAR device includes a first group of optical emitters configured to emit a first group of optical signals into the surrounding environment. The first group of optical signals corresponds to a first angular resolution with respect to the surrounding environment. The LIDAR device also includes a first group of optical detectors configured to detect the first group of reflected optical signals from the surrounding environment during a first listening window. The first group of reflected optical signals corresponds to the reflection of the first group of optical signals from objects in the surrounding environment. Further, the LIDAR device includes a second group of optical emitters configured to emit a second group of optical signals into the surrounding environment. The second group of optical emitters of the LIDAR device represents a subset of the first group of optical emitters of the LIDAR device. The second group of optical signals corresponds to a second angular resolution with respect to the surrounding environment. The second angular resolution is lower than the first angular resolution. Further, the LIDAR device includes a second group of optical detectors configured to detect the second group of reflected optical signals from the surrounding environment during a second listening window. The second group of optical detectors of the LIDAR device represents a subset of the first group of optical detectors of the LIDAR device. The second group of reflected optical signals corresponds to the reflection of the second group of optical signals from objects in the surrounding environment. The duration of the second listening window is shorter than the duration of the first listening window. Further, the LIDAR device includes a controller configured to synthesize a dataset that can be used to generate one or more point clouds. The dataset is based on the detected first group of reflected optical signals and the detected second group of reflected optical signals.

[0006] In a third aspect, a system is provided. The system includes a light detection and ranging (LIDAR) device. The LIDAR device includes a first group of optical emitters configured to emit a first group of optical signals into the surrounding environment. The first group of optical signals corresponds to a first angular resolution with respect to the surrounding environment. The LIDAR device also includes a first group of photodetectors configured to detect a first group of reflected optical signals from the surrounding environment during a first listening window. The first group of reflected optical signals corresponds to the reflection of the first group of optical signals from objects in the surrounding environment. Further, the LIDAR device includes a second group of optical emitters configured to emit a second group of optical signals into the surrounding environment. The second group of optical emitters of the LIDAR device represents a subset of the first group of optical emitters of the LIDAR device. The second group of optical signals corresponds to a second angular resolution with respect to the surrounding environment. The second angular resolution is lower than the first angular resolution. Further, the LIDAR device includes a second group of photodetectors configured to detect a second group of reflected optical signals from the surrounding environment during a second listening window. The second group of photodetectors of the LIDAR device represents a subset of the first group of photodetectors of the LIDAR device. The second group of reflected optical signals corresponds to the reflection of the second group of optical signals from objects in the surrounding environment. The duration of the second listening window is shorter than the duration of the first listening window. Further, the LIDAR device includes a LIDAR controller configured to synthesize a dataset that can be used to generate one or more point clouds. The dataset is based on the detected first group of reflected optical signals and the detected second group of reflected optical signals. The system also includes a system controller. The system controller is configured to receive the dataset from the LIDAR controller. The system controller is also configured to generate one or more point clouds based on the dataset.

[0007] These and other aspects, advantages, and alternatives will be apparent to those of ordinary skill in the art from a reading of the following detailed description, with appropriate reference to the accompanying drawings.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2A

Figure 2B

Figure 2C

Figure 2D

Figure 2E

Figure 2F

Figure 2G

Figure 2H

Figure 2I

Figure 2J

Figure 2K

Figure 3

Figure 4A

Figure 4B

Figure 5A

Figure 5B

Figure 6A

Figure 6B

Figure 6C

Figure 6D

Figure 6E

Figure 6F

Figure 6G

Figure 6H

Figure 7A

Figure 7B

Figure 8A

Figure 8B

Figure 9

Figure 10

[0009] Exemplary methods and systems are contemplated herein. Any example of an exemplary embodiment or feature described herein should not necessarily be construed as preferred or advantageous over other embodiments or features. Further, the exemplary embodiments described herein are not meant to be limiting. Particular aspects of the disclosed systems and methods can be arranged and combined in a variety of different configurations, and it will be readily understood that all of these configurations are contemplated herein. Additionally, the particular arrangements shown in the figures should not be regarded as limiting. It should be understood that other embodiments can include more or fewer of each element shown in a given figure. Additionally, some of the illustrated elements can be combined or omitted. Still further, the exemplary embodiments can include elements not illustrated in the figures.

[0010] The LIDAR devices described herein may include one or more light emitters and one or more detectors used to detect light emitted by the one or more light emitters and reflected by one or more objects in the environment surrounding the LIDAR device. As an example, the surrounding environment may include an internal or external environment such as inside or outside a building. Additionally or alternatively, the surrounding environment may include the interior of a vehicle. Still further, the surrounding environment may include the area around and / or adjacent to a road. Examples of objects in the surrounding environment include, but are not limited to, other vehicles, traffic signs, pedestrians, cyclists, road surfaces, buildings, terrain, etc. Additionally, the one or more light emitters may emit light into the local environment of the LIDAR itself. For example, the light emitted from the one or more light emitters may interact with the housing of the LIDAR and / or a surface or structure coupled to the LIDAR. In some cases, the LIDAR may be attached to a vehicle, in which case the one or more light emitters may be configured to emit light that interacts with objects within the vicinity of the vehicle. The light emitters may include, among other possibilities, optical fiber amplifiers, laser diodes, light emitting diodes (LEDs).

[0011] The term "subset" is used throughout this disclosure to describe groups of channels, photodetectors, light emitters, etc. within various devices and systems (e.g., LIDAR devices). As used herein, the term "subset" represents the mathematical term "proper subset" or "strict subset". Further, as used herein, the term "subset" excludes the empty set. In other words, for the purposes of this disclosure, if a set contains n elements, a "subset" of that set may include any integer number of elements from 1 to a maximum of n - 1 elements.

[0012] A LIDAR device can determine the distance to reflective environmental features while scanning a scene. These distances can then be aggregated into a "point cloud" (or other type of representation) that indicates the surfaces within the surrounding environment. The individual points in the point cloud can be determined, for example, by transmitting a laser pulse, detecting any return pulse reflected from an object in the surrounding environment, and then measuring the distance to the object according to the time delay between the transmission of the pulse and the reception of its reflected pulse. The resulting point cloud can generate, for example, a three-dimensional map of points indicating the positions of the reflective features in the surrounding environment.

[0013] In an exemplary embodiment, a LIDAR device can include one or more light emitters (e.g., laser diodes) and one or more light detectors (e.g., silicon photomultipliers (SiPMs), single photon avalanche diodes (SPADs), and / or avalanche photodiodes (APDs)). For example, a LIDAR device can include an array of channels that each include a light detector and a corresponding light emitter. Such arrays can irradiate objects within a scene, receive reflected light from the objects within the scene, and collect data that can be used to generate a point cloud for a particular field of view with respect to the LIDAR device. Further, to generate a point cloud having an enhanced field of view (e.g., a full 360-degree field of view), an array of light emitters and a corresponding array of light detectors can transmit and receive light at predetermined times and / or positions within that enhanced field of view. For example, a LIDAR device can include an array of channels arranged around a vertical axis such that light is transmitted and received simultaneously in a plurality of directions around a 360-degree field of view. As another example, a LIDAR device can scan (e.g., rotate or beam scan using another mechanism) around a central axis and transmit and receive multiple sets of data. The data can be combined and used to form a point cloud that can generate an enhanced field of view.

[0014] Some LIDAR devices may be susceptible to the effects of noise arising from high-intensity return signals. For example, when one light emitter emits a light pulse towards a highly reflective object (e.g., a retroreflector), the return pulse from that object may have a high intensity. In some cases, if the intensity of the return pulse is large enough, the return pulse may cause cross-talk between the channels of the LIDAR device. In other words, in addition to being detected by the photodetector corresponding to the light emitter that emitted the emitted pulse, the high-intensity return pulse may also be detected by other photodetectors within the LIDAR device (e.g., a photodetector adjacent to the photodetector corresponding to the light emitter that emitted the emitted pulse). Such cross-talk may be the result of, and / or exacerbated by, one or more defects in the optical path between the photodetector and the object being detected. For example, the optical window of a LIDAR device may have rain, condensation, snow, dust, mud, dirt, ice, debris, etc. Such defects may reflect, refract, and / or disperse one or more reflected light signals from one or more objects in the surrounding environment, thereby resulting in cross-talk.

[0015] Crosstalk, regardless of its cause, can be a source of detection errors. For example, when a detector detects a return pulse that is the result of crosstalk, a computing device associated with the LIDAR device may inappropriately determine that an object exists at a location in the surrounding environment when in reality no such object exists (i.e., the LIDAR device can produce false positive detections). Additionally or alternatively, as a result of detecting a high-intensity crosstalk return pulse, an appropriate return pulse (e.g., at a lower intensity) may be inappropriately missed. Thus, the exemplary embodiments disclosed herein can serve to reduce and / or eliminate inappropriate detections resulting from noise sources. Crosstalk from highly reflective objects is referred to throughout the present disclosure, but it is understood that other noise sources are also contemplated and can be reduced using the techniques described herein. For example, interference to various photodetectors (e.g., from spurious light sources such as different LIDAR devices or malicious light sources such as someone shining a laser pointer at a LIDAR device) can also be reduced using one or more of the techniques described herein. Additionally or alternatively, electrical crosstalk can be reduced using the techniques described herein. Electrical crosstalk can include, for example, electrical signals that are coupled into adjacent or nearby photodetectors when one photodetector experiences a large detection signal.

[0016] In some embodiments, a LIDAR device is provided. As described above, the LIDAR device can include an array of channels. Each of the channels in the array can include a photodetector and a corresponding optical emitter. For example, the optical emitter within a given channel may be configured to emit an optical pulse along a particular emission vector, and the corresponding photodetector may be configured to detect an optical pulse reflected from an object in the surrounding environment that is in the path of the emission vector. Each of the photodetectors of different channels of the array of channels may be positioned in close proximity to each other within the LIDAR device's array of photodetectors. Thus, any high-intensity return pulse can affect photodetectors that are near the primary photodetector that detects the high-intensity return pulse.

[0017] One way to reduce such crosstalk is to identify which channels within the array of channels have emission vectors that intersect with the high-reflectivity objects in the surrounding environment that are the cause of the high-intensity return. Next, once the channel that is the source of the crosstalk is identified, the optical emitters within that channel can simply refrain from emitting optical pulses in future emission cycles. However, it can be difficult to determine when (e.g., in which emission cycle) to resume emission of the optical pulses from the optical emitters corresponding to the high-intensity reflections. Similarly, another conceivable mitigation technique would be to simply ignore the pulses that will be detected in future detection cycles, detected by the photodetectors within the array that are near the primary photodetector that detected the high-intensity reflection. However, this can result in a number of channels being essentially unused during one or more detection cycles. As is apparent, the above mitigation strategies can result in multiple detected pulses being ignored, perhaps unnecessarily.

[0018] Accordingly, described herein are alternative noise reduction techniques that can be used in conjunction with, or instead of, the aforementioned mitigation techniques. That is, the techniques described herein can involve emitting / detecting an optical signal over two emission cycles. The first cycle can involve emitting all channels of a LIDAR device and detecting all returns. This first cycle can call for detecting all possible returns, regardless of whether they are relatively long or relatively short distances. However, the second cycle can involve a series of time-offset emissions / detections. The series of emissions / detections can be performed by a subset of channels within the LIDAR device (e.g., a subset that is physically far enough apart from each other so as to be less susceptible to crosstalk effects from each other). Further, the emissions / detections in the second cycle can correspond to emissions / detections at shorter distances than the emissions / detections in the first cycle. As such, the techniques described herein can take advantage of the fact that crosstalk can be a more severe problem at shorter distances (e.g., detection events in the second cycle can be used to detect objects at shorter distances, while detection events in the first cycle can be used to detect objects at longer distances). Finally, detection events from the second cycle can be combined with detection events from the first cycle to form a single data set. The techniques described herein can represent a way of performing time-division multiplexed access to the various channels of a LIDAR device (i.e., by separating detection events in time, crosstalk can be identified and ignored).

[0019] A complete detection cycle can proceed in the following manner. First (i.e., during the first cycle), the optical emitters of each channel within the LIDAR device can emit optical signals. Subsequently, the corresponding photodetectors within the LIDAR device can detect reflections from objects in the surrounding environment during the first detection window. The duration (i.e., the length of time) of the first detection window may be relatively long (e.g., between 2.0 μs and 3.0 μs) to enable the detection of objects at relatively long distances (e.g., up to a range of 300 m to 450 m). Detection events during the first detection window from the photodetectors may then be temporarily stored (e.g., in a memory such as volatile memory). For example, these detection events may be stored as complete waveforms (e.g., intensity waveforms from the corresponding photodetectors) and / or metadata (e.g., data corresponding to the detection time, detected intensity, and / or detected polarization).

[0020] Thereafter (i.e., during the second cycle), a subset of the optical emitters can be fired during shorter time segments. For example, if the LIDAR device has 16 channels (e.g., labeled "channel 0", "channel 1", "channel 2",... "channel 15"), the optical emitters of a preselected subset of the channels can be fired sequentially. For example, the optical emitter of channel 0 can be fired by itself (i.e., without irradiating other optical emitters) during a portion of the second cycle. During this portion of the second cycle, the photodetector of channel 0 can detect reflections from objects in the surrounding environment during a second detection window. The duration of the second detection window may be shorter than the duration of the first detection window. For example, the duration of the second detection window can be 0.3 μs to 0.5 μs to detect objects at relatively short distances (e.g., up to in the range of 45 m to 75 m). Detection events from this portion of the second cycle may also be stored temporarily (e.g., in a memory such as a volatile memory). Similar to the detection events during the first cycle, these detection events may be stored as a complete waveform (e.g., an intensity waveform from the corresponding photodetector) and / or metadata (e.g., data corresponding to the detection time, detected intensity, and / or detected polarization).

[0021] Next, the above-mentioned portion of the second cycle performed on channel 0 may be separately repeated for channels 2, 4, 6, 8, 10, 12, and 14 during the second cycle. As is evident from the fact that not all channels are used (for example, channels 1, 3, 5, 7, 9, 11, 13, and 15 were not used in the previous example), the angular resolution of the channels selected during the second cycle may be less than the angular resolution of all the combined channels (for example, the channels used during the first cycle). However, since the distance probed during the second cycle may be shorter than during the first cycle, a lower angular resolution may be tolerated (for example, if the surrounding environment is linearly resolved overly at a shorter distance so that it can be properly linearly resolved at a longer distance). In other words, even with a decrease in angular resolution, the data captured during the second cycle can provide sufficient linear resolution (for example, dots per inch) when considered for the shorter distances (for example, in the range less than 75 m) involved during the second cycle. The amount of decrease in angular resolution may be based at least in part on the total duration assigned to the second cycle. For example, if 5 μs is assigned to the second cycle and each second detection window has a duration of 0.5 μs, there may be 10 emission slots / portions available during the second cycle. Thus, when channels are emitted individually during the second cycle, the decrease in angular resolution may correspond to the total number of channels divided by the number of available emission slots (for example, 16 total channels / 10 emission slots, or a 1.6-fold decrease in angular resolution).

[0022] Of course, the arrangement of the channels emitted during the above-described second cycle is provided as an example, and other arrangements are conceivable and intended herein. Further, as described above, a light emitter with only a single channel can be emitted between each part of the second cycle, but other numbers of channels can be used between the parts of the second cycle (e.g., pairs of channels, groups of three channels, groups of four channels, and / or groups of five channels). For example, pairs of channels can be selected for simultaneous emission / detection in successive parts of the second cycle. In such embodiments, the pairs of channels selected between each of the parts of the second cycle are selected to be physically sufficiently separated from each other such that the channels used (e.g., detectors within the channels used) prevent crosstalk between the channels for each of the parts of the second cycle. The pairs of successive channels used over multiple parts of the second cycle can also, in some embodiments, represent an interleaving across the LIDAR device / surrounding environment. Further, in some embodiments, the number of channels emitted between one part (e.g., a pair of channels) of the second cycle can be different from the number of channels emitted between another part (e.g., a group of three channels) of the second cycle. Furthermore, it is understood that while the above-described first cycle is a longer distance, increased angular resolution cycle and the above-described second cycle is a shorter distance, decreased angular resolution cycle, the order of these cycles can be reversed (i.e., the first cycle is performed after the second cycle).

[0023] Additionally or alternatively, in some embodiments, previous detection data may be incorporated into the emission scheme. For example, in some embodiments, a highly reflective surface (e.g., a retroreflector) in the surrounding environment may be identified during previous emission cycles (e.g., based on high-intensity reflections detected by one or more photodetectors of a LIDAR device). Further, a channel (e.g., an optical emitter of the channel) targeting the identified highly reflective surface may also be identified. Next, in subsequent emission cycles (e.g., between both the first and second cycles described above), the optical emitter of the channel directed at the retroreflector may refrain from emitting completely. This can provide additional robustness against accidental crosstalk.

[0024] Once all detection events from the first cycle and the second cycle have been collected, they may be combined to form a data set that can be used to generate one or more point clouds. For example, the data from the first cycle may be provided (e.g., to a computing device by a controller of a LIDAR device) as a set of data available to generate a first point cloud, and the data from the second cycle may be provided (e.g., to a computing device by a controller of a LIDAR device) as a set of data available to generate a second point cloud. Alternatively, in some embodiments, the detection events from the two cycles may be combined such that the resulting data set can be used to generate a single point cloud. In such embodiments, the detection events corresponding to a given channel between the first cycle and the second cycle may be compared. For example, the distance to an object in the surrounding environment determined for a given channel (e.g., channel 1) during the first cycle may be compared to the distance to an object in the surrounding environment determined for the same channel (e.g., channel 1) during the second cycle. If the two distances are the same (or within some threshold difference value), the measurements may be determined to be appropriate and not indicative of crosstalk. Thus, one or both of the measured distances may be included in the data set that can be used to generate a single point cloud. Further, if the measurements during the second cycle did not result in a distance measurement, but the measurements during the first cycle did result in a distance measurement and the measurements during the first cycle were over a range that exceeded the range measured during the second cycle, the distance measured during the first cycle may similarly be included in the data set that can be used to generate a single point cloud (e.g., because the measurement can be determined not to correspond to crosstalk).However, during the second cycle, if the detection events detected during the first cycle do not match (e.g., are not within the threshold difference), and both correspond to a target range that is within the range measured during the second cycle, the determined distance may not be included in the dataset (e.g., the measurement during the first cycle could be the result of crosstalk), or only the distance measured during the second cycle may be included in the dataset.

[0025] The following description and the accompanying drawings disclose the features of various exemplary embodiments. The provided embodiments are by way of example and are not intended to be limiting. Accordingly, the dimensions in the drawings are not necessarily to scale.

[0026] Here, an exemplary system within the scope of the present disclosure will be described in more detail. The exemplary system can be implemented in an automobile or can take the form of an automobile. Additionally, the exemplary system can also be implemented in or take the form of various vehicles such as cars, trucks (e.g., pickup trucks, vans, tractors, and / or tractor-trailers, etc.), motorcycles, buses, airplanes, helicopters, drones, lawn mowers, bulldozers, boats, submarines, all-terrain vehicles, snowmobiles, aircraft, recreational vehicles, amusement park vehicles, agricultural implements or agricultural vehicles, construction machinery or construction vehicles, warehouse equipment or warehouse vehicles, factory equipment or factory vehicles, trams, golf carts, trains, trolleys, sidewalk conveyances, robotic devices, etc. Other vehicles are also possible. Further, in some embodiments, the exemplary system may not include a vehicle.

[0027] Referring now to the figures, FIG. 1 is a functional block diagram illustrating an exemplary vehicle 100 that can be configured to operate fully or partially in autonomous mode. More specifically, vehicle 100 can operate in autonomous mode without human interaction by receiving control instructions from a computing system. As part of the operation in autonomous mode, vehicle 100 can use sensors to detect and optionally identify objects in the surrounding environment to enable safe navigation. Additionally, exemplary vehicle 100 can operate in a partially autonomous (i.e., semi-autonomous) mode where some functions of vehicle 100 are controlled by a human driver of vehicle 100 and some functions of vehicle 100 are controlled by a computing system. For example, vehicle 100 can also include a subsystem that enables a driver to control the operation of vehicle 100 such as steering, acceleration, and braking, while the computing system implements assistive functions such as lane departure warning / lane keeping assist or adaptive cruise control based on other objects (e.g., vehicles) in the surrounding environment.

[0028] As described herein, in the partially autonomous driving mode, the vehicle assists with one or more driving operations (e.g., lane centering, adaptive cruise control, advanced driver assistance systems (ADAS), and / or steering, braking, and / or acceleration for implementing emergency braking), while the human driver is expected to situationally perceive the surroundings of the vehicle and monitor the assisted driving operations. Here, the vehicle can perform all driving tasks in certain situations, while the human driver is expected to take responsibility for control as needed.

[0029] For simplicity and brevity, various systems and methods are described below in conjunction with autonomous vehicles, but these or similar systems and methods can be used in a variety of driver assistance systems that do not reach the level of a fully autonomous driving system (i.e., a partially autonomous driving system). In the United States, the Society of Automotive Engineers (SAE) has defined different levels of automated driving maneuvers to indicate how much or how little a vehicle controls the driving, but different organizations in the United States or other countries may classify the levels differently. More specifically, the systems and methods of the present disclosure can be used in SAE Level 2 driver assistance systems that implement steering, braking, acceleration, lane centering, adaptive cruise control, etc., and other driver support. The disclosed systems and methods can be used in SAE Level 3 driver assistance systems that allow for autonomous driving under limited (e.g., highway) conditions. Similarly, the disclosed systems and methods can be used in vehicles that use an SAE Level 4 automated driving system that operates autonomously in most normal driving situations and requires only occasional attention from a human operator. In all such systems, accurate lane estimation is automatically performed without driver input or control (e.g., while the vehicle is in motion), resulting in improved reliability of vehicle positioning and navigation, as well as overall safety of autonomous driving, semi-autonomous driving, and other driver assistance systems. As noted above, in addition to the way SAE classifies the levels of automated driving maneuvers, other organizations in the United States or other countries may classify the levels of automated driving maneuvers differently. Without limitation, the systems and methods disclosed herein can be used in driver assistance systems defined by the levels of automated driving maneuvers of these other organizations.

[0030] As shown in FIG. 1, vehicle 100 may include various subsystems such as a propulsion system 102, a sensor system 104, a control system 106, one or more peripheral devices 108, a power source 110, a computer system 112 (which may also be referred to as a computing system) having a data storage 114, and a user interface 116. In other examples, vehicle 100 may include more or fewer subsystems, each of which may include a plurality of elements. The subsystems and components of vehicle 100 may be interconnected in various ways. Additionally, the functions of vehicle 100 described herein may be divided among additional functional or physical components, or combined into fewer functional or physical components within an embodiment. For example, control system 106 and computer system 112 may be combined into a single system that operates vehicle 100 according to various operations.

[0031] Propulsion system 102 may include one or more components operable to provide powered movement for vehicle 100 and may include, among other possible components, engine / motor 118, energy source 119, transmission 120, and wheels / tires 121. For example, engine / motor 118 may be configured to convert energy source 119 into mechanical energy and may correspond to one or a combination of, among other possible options, an internal combustion engine, an electric motor, a steam engine, or a Stirling engine. For example, in some embodiments, propulsion system 102 may include multiple types of engines and / or motors such as a gasoline engine and an electric motor.

[0032] Energy source 119 represents an energy source that may fully or partially power one or more systems of vehicle 100 (e.g., engine / motor 118). For example, energy source 119 may correspond to gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and / or other power sources. In some embodiments, energy source 119 may include a combination of a fuel tank, a battery, a capacitor, and / or a flywheel.

[0033] The transmission 120 can transmit the mechanical power from the engine / motor 118 to the wheels / tires 121 and / or other possible systems of the vehicle 100. Thus, the transmission 120 can include, among other possible components, a gearbox, a clutch, a differential, and a drive shaft. The drive shaft can include an axle that connects to one or more wheels / tires 121.

[0034] The wheels / tires 121 of the vehicle 100 can have various configurations within the exemplary embodiments. For example, the vehicle 100 can exist in the form of a unicycle, a bicycle / motorcycle, a tricycle, or a four-wheel form of an automobile / truck, among other possible configurations. Thus, the wheels / tires 121 can be connected to the vehicle 100 in various ways and can exist in different materials such as metal and rubber.

[0035] The sensor system 104 can include various types of sensors, among other possible sensors, particularly the Global Positioning System (GPS) 122, the Inertial Measurement Unit (IMU) 124, the RADAR 126, the LIDAR 128, the camera 130, the steering sensor 123, and the throttle / brake sensor 125. In some embodiments, the sensor system 104 can also include sensors configured to monitor the internal systems of the vehicle 100 (e.g., an O2 monitor, a fuel gauge, an engine oil temperature, and / or brake wear).

[0036] The GPS 122 can include a transceiver operable to provide information regarding the position of the vehicle 100 relative to the Earth. The IMU 124 can have a configuration that uses one or more accelerometers and / or gyroscopes and can sense changes in the position and orientation of the vehicle 100 based on inertial acceleration. For example, the IMU 124 can detect the pitch and yaw of the vehicle 100 while the vehicle 100 is stationary or moving.

[0037] RADAR 126 can represent one or more systems configured to sense objects in the surrounding environment of vehicle 100 using radio signals, including the speed and azimuth of the objects. Thus, RADAR 126 can include an antenna configured to transmit and receive radio signals. In some embodiments, RADAR 126 can correspond to an attachable RADAR configured to obtain measurements of the surrounding environment of vehicle 100.

[0038] LIDAR 128 can include, among other system components, one or more laser sources, a laser scanner, and one or more detectors, and can operate in a coherent mode (e.g., using heterodyne detection) or an incoherent detection mode (i.e., time-of-flight mode). In some embodiments, one or more detectors of LIDAR 128 can include one or more photodetectors that can be particularly sensitive detectors (e.g., avalanche photodiodes). In some examples, such photodetectors can be capable of detecting single photons (e.g., SPAD). Further, such photodetectors can be arranged in an array (e.g., like SiPM) (e.g., through series electrical connections). In some examples, one or more photodetectors are devices that operate in Geiger mode, and the LIDAR includes sub-components designed for such Geiger mode operation.

[0039] Camera 130 can include one or more devices (e.g., a still camera, a video camera, a thermal imaging camera, a stereo camera, and / or a night vision camera, etc.) configured to capture an image of the surrounding environment of vehicle 100.

[0040] The steering sensor 123 can sense the steering angle of the vehicle 100, which may include measuring the angle of the steering wheel or measuring an electrical signal representing the angle of the steering wheel. In some embodiments, the steering sensor 123 can measure the angle of the wheels of the vehicle 100, such as detecting the angle of the wheels relative to the front axle of the vehicle 100. The steering sensor 123 can also be configured to measure a combination (or subset) of the angle of the steering wheel, an electrical signal representing the angle of the steering wheel, and the angle of the wheels of the vehicle 100.

[0041] The throttle / brake sensor 125 can detect either the throttle position or the brake position of the vehicle 100. For example, the throttle / brake sensor 125 can measure the angles of both the accelerator pedal (throttle) and the brake pedal, or, for example, measure an electrical signal representing the angle of the accelerator pedal (throttle) and / or the angle of the brake pedal. The throttle / brake sensor 125 can also measure the angle of the throttle body of the vehicle 100, which may include a part of a physical mechanism that provides modulation of the energy source 119 to the engine / motor 118 (such as a butterfly valve or carburetor). Additionally, the throttle / brake sensor 125 can measure the pressure of one or more brake pads on the rotor of the vehicle 100, or a combination (or subset) of the angles of the accelerator pedal (throttle) and the brake pedal, an electrical signal representing the angles of the accelerator pedal (throttle) and the brake pedal, the angle of the throttle body, and the pressure applied by at least one brake pad to the rotor of the vehicle 100. In other embodiments, the throttle / brake sensor 125 can be configured to measure the pressure applied to a vehicle pedal such as the throttle or brake pedal.

[0042] The control system 106 may include components configured to assist in navigating the vehicle 100, such as a steering unit 132, a throttle 134, a brake unit 136, a sensor fusion algorithm 138, a computer vision system 140, a navigation / route finding system 142, and an obstacle avoidance system 144. More specifically, the steering unit 132 may be operable to adjust the orientation of the vehicle 100, and the throttle 134 may control the operating speed of the engine / motor 118 to control the acceleration of the vehicle 100. The brake unit 136 can decelerate the vehicle 100, which may involve using friction to decelerate the wheels / tires 121. In some embodiments, the brake unit 136 may convert the kinetic energy of the wheels / tires 121 into an electric current for subsequent use by one or more systems of the vehicle 100.

[0043] The sensor fusion algorithm 138 may include a Kalman filter, a Bayesian network, or other algorithms capable of processing data from the sensor system 104. In some embodiments, the sensor fusion algorithm 138 may provide an assessment based on incoming sensor data, such as an assessment of individual objects and / or features, an assessment of a particular situation, and / or an assessment of possible effects within a given situation.

[0044] The computer vision system 140 can include hardware and software (e.g., a general-purpose processor such as a central processing unit (CPU), a dedicated processor such as a graphical processing unit (GPU) or a tensor processing unit (TPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), volatile memory, non-volatile memory, and / or one or more machine learning models) operable to process and analyze an image to determine objects that are in motion (e.g., other vehicles, pedestrians, cyclists, and / or animals) and objects that are not in motion (e.g., traffic signals, lane boundaries, speed bumps, and / or depressions). Accordingly, the computer vision system 140 can use object recognition, structure from motion (SFM), video tracking, and other algorithms used in computer vision, such as, for example, to recognize objects, map the environment, track objects, and estimate the speed of objects.

[0045] The navigation / route finding system 142 can determine a driving route for the vehicle 100, which can involve dynamically adjusting the navigation during operation. Accordingly, the navigation / route finding system 142 can navigate the vehicle 100 using data from among a number of information sources, particularly the sensor fusion algorithm 138, the GPS 122, and the map. The obstacle avoidance system 144 can evaluate potential obstacles based on sensor data and cause the vehicle 100's systems to avoid or otherwise maneuver around the potential obstacles.

[0046] As shown in FIG. 1, vehicle 100 may also include peripheral devices 108 such as a wireless communication system 146, a touch screen 148, an internal microphone 150, and / or a speaker 152. The peripheral devices 108 may provide controls or other elements for a user to interact with the user interface 116. For example, the touch screen 148 may provide information to a user of the vehicle 100. The user interface 116 may also receive input from the user via the touch screen 148. The peripheral devices 108 may also enable the vehicle 100 to communicate with devices such as devices of other vehicles.

[0047] The wireless communication system 146 may communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 146 may use 3G cellular communication such as Code Division Multiple Access (CDMA), Evolution-Data Optimized (EVDO), Global System for Mobile Communications (GSM) / General Packet Radio Service (GPRS), or cellular communication such as 4G Worldwide Interoperability for Microwave Access (WiMAX) or Long Term Evolution (LTE), or 5G. Alternatively, the wireless communication system 146 may communicate with a Wireless Local Area Network (WLAN) using Wi-Fi (registered trademark) or other possible connections. The wireless communication system 146 may also communicate directly with devices using, for example, an infrared link, Bluetooth, or ZigBee. Other wireless protocols such as various vehicle communication systems are possible within the context of the present disclosure. For example, the wireless communication system 146 may include one or more dedicated short range communication (DSRC) devices that may include public and / or private data communication between the vehicle and / or a roadside fueling station.

[0048] Vehicle 100 may include a power source 110 for supplying power to components. In some embodiments, the power source 110 may include a rechargeable lithium-ion or lead battery. For example, the power source 110 may include one or more batteries configured to provide power. Vehicle 100 may also obtain power using other types of power sources. In an exemplary embodiment, the power source 110 and the energy source 119 may be integrated into a single energy source.

[0049] Vehicle 100 may also include a computer system 112 for performing operations such as those described therein. Thus, the computer system 112 may include at least one processor 113 (which may include at least one microprocessor) operable to execute instructions 115 stored on a non-transitory computer-readable medium such as data storage 114. In some embodiments, the computer system 112 may represent a plurality of computing devices that may function to control the individual components or subsystems of Vehicle 100 in a distributed manner.

[0050] In some embodiments, the data storage 114 may include instructions 115 (e.g., program logic) executable by the processor 113 for performing various functions of Vehicle 100, including those described above in connection with FIG. 1. The data storage 114 may also include additional instructions that include instructions for sending data, receiving data, interacting with, and / or controlling one or more of the propulsion system 102, the sensor system 104, the control system 106, and the peripheral devices 108.

[0051] In addition to the instructions 115, the data storage 114 may store data such as, among other things, road maps, route information, etc. Such information may be used by Vehicle 100 and the computer system 112 during operation of Vehicle 100 in autonomous mode, semi-autonomous mode, and / or manual mode.

[0052] Vehicle 100 may include a user interface 116 for providing information to or receiving input from a user of vehicle 100. The user interface 116 may control or enable the control of the layout of content and / or interactive images that may be displayed on the touch screen 148. Additionally, the user interface 116 may include one or more input / output devices within a set of peripheral devices 108 such as the wireless communication system 146, the touch screen 148, the microphone 150, and the speaker 152.

[0053] Computer system 112 may control the functions of vehicle 100 based on inputs received from various subsystems (e.g., propulsion system 102, sensor system 104, and / or control system 106), as well as from the user interface 116. For example, computer system 112 may utilize inputs from sensor system 104 to estimate the outputs generated by propulsion system 102 and control system 106. Depending on the embodiment, computer system 112 may be operable to monitor many aspects of vehicle 100 and its subsystems. In some embodiments, computer system 112 may disable some or all of the functions of vehicle 100 based on signals received from sensor system 104.

[0054] The components of vehicle 100 can be configured to function in a manner interconnected with other components within or external to their respective systems. For example, in an exemplary embodiment, camera 130 can capture a plurality of images that can represent information regarding the state of the surrounding environment of vehicle 100 operating in autonomous or semi-autonomous mode. The state of the surrounding environment can include parameters of the road on which the vehicle is operating. For example, computer vision system 140 may be able to recognize inclinations (gradients) or other features based on a plurality of images of the road. Additionally, the combination of GPS 122 and features recognized by computer vision system 140 can be used together with map data stored in data storage 114 to determine specific road parameters. Further, RADAR 126 and / or LIDAR 128, and / or some other environmental mapping, ranging, and / or positioning sensor systems can also provide information about the surroundings of the vehicle.

[0055] In other words, the combination of various sensors (which can be referred to as input indicator sensors and output indicator sensors) and computer system 112 can interact to provide an indicator of the input provided to control the vehicle or an indicator of the surroundings of the vehicle.

[0056] In some embodiments, computer system 112 can make determinations regarding various objects based on data provided by systems other than wireless systems. For example, vehicle 100 may have a laser or other optical sensor configured to sense objects within the field of view of the vehicle. Computer system 112 can use the outputs from the various sensors to determine information regarding objects within the field of view of the vehicle and can determine distance and direction information to various objects. Computer system 112 can also determine whether an object is desirable or undesirable based on the outputs from the various sensors.

[0057] FIG. 1 shows various components of vehicle 100 (i.e., wireless communication system 146, computer system 112, data storage 114, and user interface 116) as being integrated into vehicle 100, although one or more of these components may be attached or associated separately from vehicle 100. For example, data storage 114 can exist partially or completely separate from vehicle 100. Thus, vehicle 100 can be provided in the form of device elements that can be located separately or together. The device elements that make up vehicle 100 can be communicatively coupled together in a wired and / or wireless manner.

[0058] FIGS. 2A-2E show an exemplary vehicle 200 (e.g., a fully autonomous vehicle, a semi-autonomous vehicle) that may include some or all of the functions described in relation to vehicle 100 with reference to FIG. 1. Vehicle 200 is illustrated in FIGS. 2A-2E as a van with side mirrors for illustrative purposes, but the present disclosure is not so limited. For example, vehicle 200 can represent a truck, a passenger vehicle, a semi-trailer truck, a motorcycle, a golf cart, an off-road vehicle, an agricultural vehicle, or any other vehicle described elsewhere in this specification (e.g., a bus, a boat, an airplane, a helicopter, a drone, a lawn mower, a bulldozer, a submarine, an all-terrain vehicle, a snowmobile, an aircraft, a recreational vehicle, an amusement park vehicle, an agricultural implement, a construction machine or construction vehicle, a warehouse facility or warehouse vehicle, a factory facility or factory vehicle, a tram, a train, a trolley, a pedestrian conveyance vehicle, and / or a robotic device, etc.).

[0059] Exemplary vehicle 200 may include one or more sensor systems 202, 204, 206, 208, 210, 212, 214, and 218. In some embodiments, sensor systems 202, 204, 206, 208, 210, 212, 214, and / or 218 may represent one or more optical systems (e.g., cameras), one or more LIDARs, one or more radars, one or more inertial sensors, one or more humidity sensors, one or more acoustic sensors (e.g., microphones, sonar devices), or one or more other sensors configured to sense information about the environment surrounding vehicle 200. In other words, any sensor system currently known or later developed may be coupled to vehicle 200 and / or utilized in conjunction with various operations of vehicle 200. As an example, LIDAR may be utilized for autonomous driving or other types of navigation, planning, perception, and / or mapping operations of vehicle 200. Additionally, sensor systems 202, 204, 206, 208, 210, 212, 214, and / or 218 may represent combinations of sensors described herein (e.g., one or more LIDARs and RADARs, one or more LIDARs and cameras, one or more cameras and RADARs, one or more LIDARs, cameras, and RADARs, etc.).

[0060] Note that the number, location, and type of sensor systems (e.g., 202, 204) depicted in FIGS. 2A - E are intended as non-limiting examples of the location, number, and type of such sensor systems for autonomous or semi-autonomous vehicles. Alternative numbers, locations, types, and configurations of such sensors are possible (e.g., to conform to vehicle size, shape, aerodynamics, fuel economy, aesthetics, and / or to reduce cost, or other conditions for compatibility with special environments or application scenarios, etc.). For example, sensor systems (e.g., 202, 204) may be disposed at various other locations on the vehicle (e.g., at location 216) and may have a field of view corresponding to the interior and / or surrounding environment of vehicle 200.

[0061] The sensor system 202 may be attached to the upper part of the vehicle 200 and may include one or more sensors configured to detect information about the environment surrounding the vehicle 200 and output an indicator of the information. For example, the sensor system 202 may include any combination of cameras, RADAR, LIDAR, inertial sensors, humidity sensors, and acoustic sensors (e.g., microphones, and / or sonar devices). The sensor system 202 may include one or more movable mounts operable to adjust the orientation of one or more sensors within the sensor system 202. In one embodiment, the movable mount may include a rotating platform capable of scanning the sensors to obtain information from each direction around the vehicle 200. In another embodiment, the movable mount of the sensor system 202 may be movable to scan within a specific range of angles and / or azimuth angles and / or elevation angles. The sensor system 202 may be attached on the roof of the vehicle, although other attachment locations are also possible.

[0062] Additionally, the sensors of the sensor system 202 may be distributed in various locations and do not need to be collocated in a single location. Further, each sensor of the sensor system 202 may be configured to move or scan independently of the other sensors of the sensor system 202. Additionally or alternatively, a plurality of sensors may be attached to one or more of the sensor locations 202, 204, 206, 208, 210, 212, 214, and / or 218. For example, there may be two LIDAR devices attached to a sensor location, and / or there may be one LIDAR device and one RADAR attached to a sensor location.

[0063] One or more sensor systems 202, 204, 206, 208, 210, 212, 214, and / or 218 may include one or more LIDAR sensors. For example, a LIDAR sensor may include a plurality of light emitter devices disposed over an angular range with respect to a given plane (e.g., the x-y plane). For example, one or more of the sensor systems 202, 204, 206, 208, 210, 212, 214, and / or 218 may be configured to rotate or pivot about an axis (e.g., the z-axis) perpendicular to a given plane so as to illuminate the environment surrounding the vehicle 200 with light pulses. Information about the surrounding environment may be determined based on detection of various aspects of the reflected light pulses (e.g., elapsed flight time, polarization, and / or intensity).

[0064] In an exemplary embodiment, the sensor systems 202, 204, 206, 208, 210, 212, 214, and / or 218 may be configured to provide respective point cloud information related to physical objects within the surrounding environment of the vehicle 200. The vehicle 200, as well as the sensor systems 202, 204, 206, 208, 210, 212, 214, and 218, are illustrated as including certain features, but it will be understood that other types of sensor systems are contemplated within the scope of the present disclosure. Further, the exemplary vehicle 200 may include any of the components described in connection with the vehicle 100 of FIG. 1.

[0065] In an exemplary configuration, one or more RADARs may be located on vehicle 200. Similar to the RADAR 126 described above, one or more RADARs may include antennas configured to transmit and receive radio waves (e.g., electromagnetic waves having frequencies in the range of 30 Hz to 300 GHz). Such radio waves may be used to determine the distance and / or velocity of one or more objects in the surrounding environment of vehicle 200. For example, one or more sensor systems 202, 204, 206, 208, 210, 212, 214, and / or 218 may include one or more RADARs. In some examples, one or more RADARs are located near the rear of vehicle 200 (e.g., sensor systems 208, 210) and may actively scan the environment near the rear of vehicle 200 for the presence of radio wave reflecting objects. Similarly, one or more RADARs are located near the front of vehicle 200 (e.g., sensor systems 212, 214) and may actively scan the environment near the front of vehicle 200. The RADAR may be positioned in a location suitable for illuminating an area including the forward path of vehicle 200 without being blocked by other features of vehicle 200, for example. For example, the RADAR may be embedded in and / or attached to or near a front bumper, front headlight, cowl, and / or hood. Additionally, one or more additional RADARs may be positioned to actively scan the sides and / or rear of vehicle 200 for the presence of radio wave reflecting objects, such as by including such devices on or near a rear bumper, side panel, rocker panel, and / or vehicle chassis.

[0066] Vehicle 200 may include one or more cameras. For example, one or more sensor systems 202, 204, 206, 208, 210, 212, 214, and / or 218 may include one or more cameras. The cameras may be photosensitive devices such as still cameras, video cameras, thermal imaging cameras, stereo cameras, night vision cameras, etc. configured to capture multiple images of the surrounding environment of vehicle 200. For this purpose, the cameras may be configured to detect visible light and, additionally or alternatively, may be configured to detect light from other portions of the spectrum such as infrared or ultraviolet light. The cameras may be two-dimensional detectors and optionally may have a sensitivity range in three-dimensional space. In some embodiments, the cameras may include, for example, a range detector configured to generate a two-dimensional image indicating the distance from the camera to several points within the surrounding environment. For this purpose, the cameras may use one or more range detection techniques. For example, the cameras can provide range information by using structured light techniques, in which vehicle 200 illuminates objects in the surrounding environment with a predetermined light pattern such as a grid or checkerboard pattern, and the cameras are used to detect the reflection of the predetermined light pattern from the surrounding environment. Based on the distortion of the reflected light pattern, vehicle 200 may determine the distance to points on the object. The predetermined light pattern may be composed of infrared light or emission lines of other wavelengths suitable for such measurements. In some examples, the cameras may be mounted inside the front windshield of vehicle 200. Specifically, the cameras may be positioned to capture images from a forward view with respect to the orientation of vehicle 200. Other mounting locations and viewing angles of the cameras can also be used and can be either inside or outside vehicle 200. Also, the cameras may have associated optical elements operable to provide an adjustable field of view. Furthermore, the cameras may be mounted to vehicle 200 using a movable mount to change the pointing angle of the cameras, such as via a pan / tilt mechanism.

[0067] Vehicle 200 may also include one or more acoustic sensors used to sense the surrounding environment of vehicle 200 (e.g., one or more of sensor systems 202, 204, 206, 208, 210, 212, 214, 216, 218 may include one or more acoustic sensors). The acoustic sensors may include microphones (e.g., piezoelectric microphones, condenser microphones, ribbon microphones, and / or microelectromechanical system (MEMS) microphones) used to sense acoustic waves (i.e., pressure differences) in the fluid (e.g., air) of the environment surrounding vehicle 200. Such acoustic sensors may be used to identify sounds (e.g., sirens, human speech, animal sounds, and / or alarms) in the surrounding environment on which the control strategy of vehicle 200 may be based. For example, if the acoustic sensor detects a siren (e.g., a mobility siren and / or a fire truck siren), vehicle 200 may decelerate and / or navigate to the edge of the road.

[0068] Although not shown in FIGS. 2A-2E, vehicle 200 may include a wireless communication system (e.g., similar to the wireless communication system 146 of FIG. 1 and / or in addition to the wireless communication system 146 of FIG. 1). The wireless communication system may include a wireless transmitter and a wireless receiver configured to communicate with devices external or internal to vehicle 200. Specifically, the wireless communication system may include, for example, a transceiver configured to communicate with other vehicles and / or computing devices at, for example, a vehicle communication system or a road refueling station. Examples of such vehicle communication systems include DSRC, radio frequency identification (RFID), and other communication standards proposed for intelligent transport systems.

[0069] In addition to or instead of those shown, vehicle 200 may include one or more other components. The additional components may include electrical or mechanical functions.

[0070] The control system of vehicle 200 can be configured to control vehicle 200 according to a control strategy selected from among a plurality of possible control strategies. The control system can receive information from sensors (on or outside vehicle 200) coupled to vehicle 200, modify the control strategy (and associated driving behavior) based on that information, and be configured to control vehicle 200 according to the modified control strategy. The control system can be further configured to monitor the information received from the sensors and continuously evaluate the driving state, and can also be configured to modify the control strategy and driving behavior based on changes in the driving state. For example, the route taken by the vehicle from one destination to another can be modified based on driving conditions. Additionally or alternatively, speed, acceleration, turning angle, inter-vehicle distance (i.e., the distance to the vehicle in front of the current vehicle), lane selection, etc. can all be modified in response to changes in driving conditions.

[0071] As described above, in some embodiments, vehicle 200 can take the form of a van, although alternative forms are also possible and are contemplated herein. Accordingly, FIGS. 2F-2I illustrate embodiments in which vehicle 250 takes the form of a semi-truck. For example, FIG. 2F illustrates a front view of vehicle 250, and FIG. 2G illustrates an isometric view of vehicle 250. In embodiments where vehicle 250 is a semi-truck, vehicle 250 can include a tractor portion 260 and a trailer portion 270 (illustrated in FIG. 2G). FIGS. 2H and 2I provide a side view and a top view of tractor portion 260, respectively. Similar to vehicle 200 illustrated above, vehicle 250 illustrated in FIGS. 2F-2I can also include various sensor systems (e.g., similar to sensor systems 202, 206, 208, 210, 212, 214 shown and described with reference to FIGS. 2A-2E). In some embodiments, vehicle 200 of FIGS. 2A-2E can include only a single copy of some sensor systems (e.g., sensor system 204), while vehicle 250 illustrated in FIGS. 2F-2I can include multiple copies of its sensor systems (e.g., sensor systems 204A and 204B as illustrated).

[0072] The drawings and the overall description may refer to a given vehicle form (e.g., semi - truck vehicle 250 or van vehicle 200), but it is understood that the embodiments described herein may be equally applicable in the context of various vehicles (e.g., using modifications employed to account for the vehicle's form factor). For example, sensors and / or other components described or illustrated as part of van vehicle 200 may also be used in semi - truck vehicle 250 (e.g., for navigation and / or obstacle detection and avoidance).

[0073] FIG. 2J illustrates various sensor fields of view (e.g., associated with vehicle 250 as described above). As noted above, vehicle 250 may contain a plurality of sensors / sensor units. The locations of the various sensors may correspond, for example, to the sensor locations disclosed in FIGS. 2F - 2I. However, in some cases, the sensors may have other locations. For the sake of simplicity of the drawings, sensor location reference numbers are omitted from FIG. 2J. For each sensor unit of vehicle 250, FIG. 2J illustrates representative fields of view (e.g., fields of view labeled as 252A, 252B, 252C, 252D, 254A, 254B, 256, 258A, 258B, and 258C). A sensor's field of view may include the angular region (e.g., azimuthal region and / or elevation region) within which the sensor can detect an object.

[0074] FIG. 2K illustrates beam steering for sensors of a vehicle (e.g., vehicle 250 shown and described with reference to FIGS. 2F - 2J) according to an exemplary embodiment. In various embodiments, the sensor unit of vehicle 250 can be RADAR, LIDAR, sonar, etc. Further, in some embodiments, during operation of the sensor, the sensor can be scanned within the field of view of the sensor. Various different scan angles for an exemplary sensor are shown as region 272, each indicating the angular region in which the sensor is operating. The sensor can vary the region in which it is operating periodically or iteratively. In some embodiments, multiple sensors can be used by vehicle 250 to measure region 272. Additionally, other regions can be included in other examples. For example, one or more sensors can measure aspects of trailer 270 of vehicle 250 and / or regions in front of vehicle 250.

[0075] At some angles, the operating region 275 of the sensor can include the rear wheels 276A, 276B of trailer 270. Thus, the sensor can measure rear wheels 276A and / or 276B during operation. For example, rear wheels 276A, 276B can reflect a LIDAR signal or a RADAR signal transmitted by the sensor. The sensor can receive the signal reflected from rear wheels 276A, 276. Therefore, the data collected by the sensor can include data from reflections from the wheels.

[0076] In some cases, such as when the sensor is RADAR, the reflections from rear wheels 276A, 276B can appear as noise in the received RADAR signal. As a result, RADAR can operate with an enhanced signal - to - noise ratio in cases where rear wheels 276A, 276B direct the RADAR signal away from the sensor.

[0077] Figure 3 is a conceptual illustration of wireless communication between various computing systems related to autonomous or semi-autonomous vehicles, according to an exemplary embodiment. In particular, wireless communication can occur between the remote computing system 302 and the vehicle 200 via the network 304. Wireless communication can also occur between the server computing system 306 and the remote computing system 302, and between the server computing system 306 and the vehicle 200.

[0078] The vehicle 200 can correspond to various types of vehicles capable of transporting passengers or objects between locations and can take any one or more of the forms of the vehicles discussed above. In some cases, the vehicle 200 can operate in an autonomous or semi-autonomous mode that enables the control system to use sensor measurements to safely navigate the vehicle 200 between destinations. When operating in the autonomous or semi-autonomous mode, the vehicle 200 can navigate regardless of the presence of passengers. As a result, the vehicle 200 can pick up and drop off passengers between desired destinations.

[0079] The remote computing system 302 can represent any type of device related to remote assistance technologies, including but not limited to those described herein. Among the examples, the remote computing system 302 can be any type of device configured to (i) receive information related to the vehicle 200, (ii) provide an interface through which a human operator can then become aware of the information and enter a response related to the information, and (iii) transmit the response to the vehicle 200 or to another device. The remote computing system 302 can take various forms, such as a workstation, a desktop computer, a laptop, a tablet, a mobile phone (e.g., a smartphone), and / or a server. In some examples, the remote computing system 302 can include multiple computing devices operating together in a network configuration.

[0080] The remote computing system 302 may include one or more subsystems and components that are similar or identical to the subsystems and components of the vehicle 200. At a minimum, the remote computing system 302 can include a processor configured to perform the various operations described herein. In some embodiments, the remote computing system 302 may also include a user interface that includes input / output devices such as a touch screen and speakers. Other examples are also possible.

[0081] The network 304 represents an infrastructure that enables wireless communication between the remote computing system 302 and the vehicle 200. The network 304 also enables wireless communication between the server computing system 306 and the remote computing system 302, and between the server computing system 306 and the vehicle 200.

[0082] The location of the remote computing system 302 can vary within the scope of the example. For example, the remote computing system 302 can be at a remote location from the vehicle 200 having wireless communication via the network 304. In another example, the remote computing system 302 can correspond to a computing device within the vehicle 200 that is separate from the vehicle 200 but where a human operator can interact with a passenger or driver of the vehicle 200. In some examples, the remote computing system 302 can be a computing device with a touch screen that can be operated by a passenger of the vehicle 200.

[0083] In some embodiments, the operations described herein performed by the remote computing system 302 can be performed additionally or alternatively by the vehicle 200 (i.e., by any system or subsystem of the vehicle 200). In other words, the vehicle 200 can be configured to provide a remote assistance mechanism with which a driver or passenger of the vehicle can interact.

[0084] The server computing system 306 may be configured to wirelessly communicate with the remote computing system 302 and the vehicle 200 via the network 304 (or, in some cases, directly with the remote computing system 302 and / or the vehicle 200). The server computing system 306 may represent any computing device configured to receive, store, determine, and / or transmit information regarding the vehicle 200 and its remote assistance. Thus, the server computing system 306 may be configured to perform any operation or portion of such operations described herein as being performed by the remote computing system 302 and / or the vehicle 200. In some embodiments of the wireless communication related to remote assistance, the server computing system 306 can be utilized, while in other embodiments it cannot be utilized.

[0085] The server computing system 306 may include one or more subsystems and components similar or identical to those of the remote computing system 302 and / or the vehicle 200, such as a processor configured to perform various operations described herein, and a wireless communication interface for receiving information from and providing information to the remote computing system 302 and the vehicle 200.

[0086] The various systems described above may perform various operations. Here, these operations and related features will be described.

[0087] In accordance with the above considerations, a computing system (e.g., the remote computing system 302, the server computing system 306, and / or a computing system local to the vehicle 200) may operate to capture an image of the surrounding environment of an autonomous or semi-autonomous vehicle using a camera. Generally, at least one computing system can analyze the image and, if possible, control the autonomous or semi-autonomous vehicle.

[0088] In some embodiments, to facilitate autonomous or semi-autonomous operation, a vehicle (e.g., vehicle 200) may receive data representing objects in the environment surrounding the vehicle (also referred to herein as "environmental data") in various ways. The vehicle's sensor system may provide environmental data representing objects in the surrounding environment. For example, the vehicle may have various sensors including cameras, RADAR, LIDAR, microphones, wireless units, and other sensors. Each of these sensors may communicate environmental data to a processor within the vehicle regarding the information received by each respective sensor.

[0089] In one example, a camera may be configured to capture still images and / or video. In some embodiments, the vehicle may have two or more cameras positioned in different orientations. Also, in some embodiments, the camera may be capable of moving to capture images and / or video in different directions. The camera may be configured to store the captured images and video in memory for later processing by the vehicle's processing system. The captured images and / or video may be environmental data. Further, the camera may include an image sensor as described herein.

[0090] In another example, RADAR may be configured to transmit electromagnetic signals that are reflected by various objects near the vehicle and then capture the electromagnetic signals reflected from the objects. The captured reflected electromagnetic signals may enable RADAR (or the processing system) to make various determinations about the objects that reflected the electromagnetic signals. For example, the distance and position to various reflecting objects may be determined. In some embodiments, the vehicle may have two or more RADARs in different orientations. RADAR may be configured to store the captured information in memory for later processing by the vehicle's processing system. The information captured by RADAR may be environmental data.

[0091] In another example, the LIDAR can be configured to transmit electromagnetic signals (e.g., infrared light such as from a gas or diode laser, or from other possible light sources) reflected by target objects near the vehicle. The LIDAR may be capable of acquiring the reflected electromagnetic (e.g., infrared light) signals. The captured reflected electromagnetic signals may enable a ranging system (or a processing system) to determine the distances to various objects. The LIDAR can also determine the speed or velocity of the target object and store it as environmental data.

[0092] Additionally, in one example, a microphone can be configured to capture the audio of the vehicle's surrounding environment. The sounds captured by the microphone may include the sirens of emergency vehicles and the sounds of other vehicles. For example, the microphone may capture the sounds of the sirens of ambulances, fire trucks, and police vehicles. The processing system may be able to identify that the captured audio signal indicates an emergency vehicle. In another example, the microphone may capture the sound of the exhaust of another vehicle, such as from a motorcycle. The processing system may be able to identify that the captured audio signal indicates a motorcycle. The data captured by the microphone may form part of the environmental data.

[0093] In yet another example, a radio unit can be configured to transmit electromagnetic signals that can take the form of Bluetooth signals, 802.11 signals, and / or other wireless technology signals. The first emitted electromagnetic signal can be transmitted via one or more antennas located in the wireless unit. Further, the first emitted electromagnetic signal can be transmitted in one of many different wireless signal modes. However, in some embodiments, it is desirable to transmit the first emitted electromagnetic signal in a signal mode that requests a response from a device located near the autonomous or semi-autonomous vehicle. The processing system may be able to detect nearby devices based on the responses returned to the radio unit and use this communicated information as part of the environmental data.

[0094] In some embodiments, the processing system may be able to combine information from various sensors to further determine the vehicle's surrounding environment. For example, the processing system may combine data from both RADAR information and captured images to determine whether another vehicle or a pedestrian is in front of the autonomous or semi-autonomous vehicle. In other embodiments, other combinations of sensor data may be used by the processing system to make determinations about the surrounding environment.

[0095] While operating in autonomous mode (or semi-autonomous mode), the vehicle may control its operation with little or no human input. For example, if a human operator inputs an address into the vehicle, the vehicle may be able to drive to the specified destination without further input from the human (e.g., without the need for the human to operate or touch the brake / accelerator pedals). Further, while the vehicle is operating autonomously or semi-autonomously, the sensor system may receive environmental data. The vehicle's processing system may change the control of the vehicle based on environmental data received from various sensors. In some examples, the vehicle may change its speed in response to environmental data from various sensors. The vehicle may change its speed to avoid obstacles and to comply with traffic laws. When the processing system in the vehicle identifies an object near the vehicle, the vehicle may be able to change its speed or move in another way.

[0096] If the vehicle detects an object but does not have sufficient confidence in the detection of the object, the vehicle can request a human operator (or a more powerful computer) to perform one or more remote assistance tasks, such as (i) confirming whether the object actually exists in the surrounding environment (e.g., whether there is actually a stop sign or not), (ii) confirming whether the vehicle's identification of the object is correct, (iii) correcting the identification if it is incorrect, and / or (iv) providing supplementary instructions (or modifying the current instructions) to an autonomous or semi-autonomous vehicle. The remote assistance tasks may also include providing instructions for a human operator to control the operation of the vehicle (e.g., when the human operator determines that the object is a stop sign, instructing the vehicle to stop at the stop sign), but in some scenarios, the vehicle itself may control its own operation based on the feedback of the human operator related to the identification of the object.

[0097] To facilitate this, the vehicle can analyze environmental data representing objects in the surrounding environment and determine at least one object having a detection reliability below a threshold. The vehicle's processor can be configured to detect various objects in the surrounding environment based on environmental data from various sensors. For example, in one embodiment, the processor can be configured to detect objects that may be important for the vehicle to recognize. Such objects may include pedestrians, cyclists, street signs, other vehicles, indicator signals of other vehicles, and various other objects detected in the captured environmental data.

[0098] The detection reliability can indicate the likelihood that the determined object is correctly identified or exists in the surrounding environment. For example, the processor can perform object detection of an object in the image data in the received environmental data and determine that the object has a detection reliability below the threshold based on the inability to identify that at least one object has a detection reliability exceeding the threshold. When the result of object detection or object recognition of an object is not conclusive, the detection reliability may be low or below a set threshold.

[0099] The vehicle can detect objects in the surrounding environment in various ways depending on the source of the environmental data. In some embodiments, the environmental data can be image or video data coming from a camera. In other embodiments, the environmental data may come from LIDAR. The vehicle can analyze the captured image or video data to identify objects within the image or video data. The method and apparatus can be configured to monitor the image and / or video data for the presence of objects in the surrounding environment. In other embodiments, the environmental data can be RADAR, audio, or other data. The vehicle can be configured to identify objects in the surrounding environment based on RADAR, audio, or other data.

[0100] In some embodiments, the technology used by the vehicle to detect objects can be based on a known set of data. For example, data related to environmental objects can be stored in a memory located in the vehicle. The vehicle can compare the received data with the stored data to determine the object. In other embodiments, the vehicle can be configured to determine the object based on the context of the data. For example, street signs related to construction can generally have an orange color. Thus, the vehicle can be configured to detect an orange object located near the roadside as a construction-related street sign. Additionally, when the vehicle's processing system detects an object in the captured data, it can also calculate the confidence level of each object.

[0101] Furthermore, the vehicle can also have a confidence threshold. The confidence threshold can vary depending on the type of object detected. For example, for an object that may require a quick response action from the vehicle, such as the brake light of another vehicle, the confidence threshold can be low. However, in other embodiments, the confidence threshold can be the same for all detected objects. If the confidence level associated with the detected object is higher than the confidence threshold, the vehicle can assume that the object is correctly recognized and adjust the vehicle's control accordingly in a responsive manner.

[0102] If the confidence level associated with a detected object is lower than the confidence threshold, the actions taken by the vehicle may vary. In some embodiments, the vehicle may react as if the detected object is present, despite the low confidence level. In other embodiments, the vehicle may react as if the detected object is not present.

[0103] When the vehicle detects an object in the surrounding environment, it can also calculate the confidence level associated with a particular detected object. The confidence level can be calculated in various ways depending on the embodiment. In one example, when detecting an object in the surrounding environment, the vehicle may compare the environmental data with predetermined data associated with known objects. The closer the match between the environmental data and the predetermined data, the higher the confidence level. In other embodiments, the vehicle may use a mathematical analysis of the environmental data to determine the confidence level associated with the object.

[0104] In response to a determination that an object has a detection confidence level below a threshold, the vehicle may send a request for remote assistance to a remote computing system, along with the identification of the object. As discussed above, the remote computing system can take various forms. For example, the remote computing system can be a computing device within a vehicle separate from the vehicle, with a touch screen interface for displaying remote assistance information by which a human operator can interact with the vehicle's passengers or driver. Additionally or alternatively, as another example, the remote computing system can be a remote computer terminal or other device located in a place not near the vehicle.

[0105] A request for remote assistance may include environmental data including an object, such as image data, audio data, etc. The vehicle may transmit environmental data to a remote computing system over a network (e.g., network 304), and in some embodiments, via a server (e.g., server computing system 306) to a remote computing system. A human operator of the remote computing system may then use the environmental data as a basis for responding to the request.

[0106] In some embodiments, if an object is detected as having a confidence level below a confidence threshold, the object may be given a preliminary identification, and the vehicle may be configured to adjust the operation of the vehicle in response to the preliminary identification. Such adjustment of the operation may take the form of, among other possible adjustments, stopping the vehicle, switching the vehicle to a human control mode, changing the speed (e.g., speed and / or direction) of the vehicle.

[0107] In other embodiments, even if the vehicle detects an object having a confidence level that meets or exceeds a threshold, the vehicle may operate in accordance with the detected object (e.g., stop if the object is identified with a high confidence level as a stop sign), but may be configured to request remote assistance either simultaneously with (or after) operating in accordance with the detected object.

[0108] FIG. 4A is a block diagram of a system according to an exemplary embodiment. In particular, FIG. 4A shows a system 400 including a system controller 402, a LIDAR device 410, a plurality of sensors 412, and a plurality of controllable components 414. The system controller 402 includes a processor 404, a memory 406, and instructions 408 stored on the memory 406 and executable by the processor 404 to perform functions.

[0109] Processor 404 may include one or more processors such as one or more general-purpose microprocessors (e.g., having a single core or multiple cores) and / or one or more special-purpose microprocessors. The one or more processors may include, for example, one or more central processing units (CPUs), one or more microcontrollers, one or more graphics processing units (GPUs), one or more tensor processing units (TPUs), one or more ASICs, and / or one or more field-programmable gate arrays (FPGAs). Other types of processors, computers, or devices configured to execute software instructions are also contemplated herein.

[0110] Memory 406 may include a computer-readable medium such as a non-transitory computer-readable medium including, without limitation, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), non-volatile random access memory (e.g., flash memory), solid state drive (SSD), hard disk drive (HDD), compact disk (CD), digital video disk (DVD), digital tape, read / write (R / W) CD, R / W DVD, etc.

[0111] The LIDAR device 410, further described below, includes a plurality of light emitters configured to emit light (e.g., in light pulses) and one or more light detectors configured to detect light (e.g., the reflected portion of a light pulse). The LIDAR device 410 may generate three-dimensional (3D) point cloud data from the output of the light detector and provide the 3D point cloud data to the system controller 402. The system controller 402 may then perform operations on the 3D point cloud data to determine characteristics of the surrounding environment (e.g., the relative position of objects in the surrounding environment, edge detection, object detection, and / or proximity sensing).

[0112] Similarly, system controller 402 can determine the characteristics of system 400 and / or the characteristics of the surrounding environment using the outputs from multiple sensors 412. For example, sensors 412 can include one or more of GPS, IMU, an image capture device (e.g., a camera), a light sensor, a thermal sensor, and other sensors that indicate parameters related to system 400 and / or the surrounding environment. LIDAR device 410 is depicted as separate from sensors 412 by way of example, and in some examples, can be considered as part of sensors 412 or as sensors 412.

[0113] Based on the characteristics of the surrounding environment determined by system controller 402 based on the outputs from system 400 and / or LIDAR device 410 and sensors 412, system controller 402 can control controllable component 414 to perform one or more actions. For example, system 400 can correspond to a vehicle, in which case controllable component 414 can include the vehicle's braking system, steering system, and / or acceleration system, and system controller 402 can change the manner of these controllable components based on characteristics determined from LIDAR device 410 and / or sensors 412 (e.g., when system controller 402 controls the vehicle in autonomous or semi-autonomous mode). Among the examples, LIDAR device 410 and sensors 412 are also controllable by system controller 402.

[0114] FIG. 4B is a block diagram of a LIDAR device according to an exemplary embodiment. In particular, FIG. 4B shows LIDAR device 410 having a controller 416 configured to control a plurality of optical emitters 424 and one or more optical detectors, e.g., a plurality of optical detectors 426. LIDAR device 410 further includes a transmission circuit 428 configured to select and provide power to each of the plurality of optical emitters 424, and can include a selector circuit 430 configured to select each of the plurality of optical detectors 426. Controller 416 includes a processor 418, a memory 420, and instructions 422 stored on memory 420.

[0115] Similar to processor 404, processor 418 may include one or more processors such as one or more general-purpose microprocessors and / or one or more special-purpose microprocessors. The one or more processors may include, for example, one or more CPUs, one or more microcontrollers, one or more GPUs, one or more TPUs, one or more ASICs, and / or one or more FPGAs. Other types of processors, computers, or devices configured to execute software instructions are also contemplated herein.

[0116] Similar to memory 406, memory 420 may include computer-readable media such as, but not limited to, non-transitory computer-readable media such as ROM, PROM, EPROM, EEPROM, non-volatile random access memory (e.g., flash memory), SSD, HDD, CD, DVD, digital tape, R / W CD, R / W DVD, etc.

[0117] Instructions 422 are stored on memory 420 and are executable by processor 418 and perform functions related to controlling emission circuit 428 and selector circuit 430 to generate 3D point cloud data and to process 3D point cloud data (or perhaps to facilitate processing of 3D point cloud data by another computing device such as system controller 402).

[0118] The controller 416 can determine 3D point cloud data by using the optical emitter 424 to emit pulses of light. The emission times are established for each optical emitter, and the relative locations of the emission times are also tracked. Aspects of the surrounding environment of the LIDAR device 410, such as various objects, reflect the pulses of light. For example, if the LIDAR device 410 is in a surrounding environment that includes a road, such objects can include vehicles, signs, pedestrians, road surfaces, construction cones, and the like. Some objects may be more reflective than others such that the intensity of the reflected light can indicate the type of object that reflected the light pulse. Further, the surfaces of the objects are at different positions relative to the LIDAR device 410, and thus it may take some time for a portion of the light pulse to be reflected and return to the LIDAR device 410. Accordingly, the controller 416 can track the detection time at which the reflected light pulse is detected by the photodetector, and the relative position of the photodetector at the detection time. By measuring the time difference between the emission time and the detection time, the controller 416 can determine how far the light pulse has traveled before being received, and thus the relative distance of the corresponding object. By tracking the relative positions at the emission time and the detection time, the controller 416 can determine the orientation of the light pulse and the reflected light pulse relative to the LIDAR device 410, and thus the relative orientation of the object. By tracking the intensity of the received light pulse, the controller 416 can determine how reflective the object is. The 3D point cloud data determined based on this information can thus indicate the relative positions of the detected reflected light pulses (e.g., within a coordinate system such as a Cartesian coordinate system) and the intensity of each reflected light pulse.

[0119] The firing circuit 428 is used to select an optical emitter for emitting light pulses. Similarly, the selector circuit 430 is used to sample the output from the photodetector.

[0120] FIG. 5A is an illustrative diagram of a LIDAR device that can be used to emit a group of optical signals and detect a group of reflected optical signals according to an exemplary embodiment. For example, FIG. 5A may represent the physical arrangement of the optical emitter 424 and the photodetector 426 within the LIDAR device 410, as shown and described with reference to FIG. 4B. Such optical emitter 424 and photodetector 426 may be positioned (e.g., attached to or fabricated on) the substrate 500 in some embodiments. Additionally, the optical emitter 424 and the photodetector 426 may be disposed within channels. Each channel may include a single optical emitter 424 and a single corresponding photodetector 426. For example, as shown in FIG. 5A, the photodetector 426 may be positioned adjacent (e.g., vertically along the z direction as shown) to its corresponding optical emitter 424 on the substrate 500. However, other embodiments are also possible and contemplated herein. For example, multiple photodetectors may correspond to a single optical emitter, multiple optical emitters may correspond to a single photodetector, and / or the photodetector may not be positioned adjacent to the corresponding optical emitter.

[0121] Furthermore, as shown in FIG. 5A, the optical emitter 424 and the photodetector 426 may each be (e.g., similar to FIG. 4B) connected (e.g., electrically) to the emission circuit 428 and the selector circuit 430. Such connections may be generated, for example, using conductive traces 522. However, the techniques described herein are widely applicable, and it is understood that the arrangement of FIG. 5A, including the incorporation of the emission circuit 428 and the selector circuit 430, is provided merely as an example.

[0122] The light emitters 424 within the array can include a light source such as a laser diode. In some embodiments, the light emitter 424 can include a pulsed light source. For example, the light source can include one or more pulsed lasers (e.g., a Q-switched laser). In alternative embodiments, a continuous wave (CW) light source may be used. In some embodiments, the light emitter 424 can include a fiber laser coupled to an optical amplifier. In particular, the fiber laser can be a laser in which the active gain medium (i.e., the source of optical gain within the laser) is within an optical fiber. Further, the fiber laser can be arranged in various ways within the LIDAR device 410 (e.g., partially disposed on the substrate 500 or fully disposed on the substrate 500). However, in still other embodiments, one or more of the light emitters 424 within the array can additionally or alternatively include an LED, a vertical cavity surface emitting laser (VCSEL), an organic light emitting diode (OLED), a polymer light emitting diode (PLED), a light emitting polymer (LEP), a liquid crystal display (LCD), MEMS, and / or any other device configured to selectively transmit, reflect, and / or emit light to provide an emitted light beam and / or pulse. The light emitter 424 can be configured to emit an optical signal toward an object in the surrounding environment that can be detected by the light detector 426 when reflected by such an object to determine the distance between the LIDAR device 410 and each object.

[0123] The wavelength range emitted by the light emitter 424 can be, for example, within the ultraviolet, visible, and / or infrared portions of the electromagnetic spectrum. In some examples, the wavelength range can be a narrow wavelength range such as provided by a laser. In some embodiments, the wavelength range includes a wavelength of about 905 nm. It should be noted that this wavelength is provided as an example only and is not intended to be limiting.

[0124] Although not shown in FIG. 5A, it is understood that the optical signals (e.g., optical pulses) emitted by the light emitters 424 within the array can be transmitted into the surrounding environment via one or more lenses, mirrors, color filters, polarizers, waveguides, apertures, and the like. For example, in some embodiments, the optical signals from the light emitters 424 may be adjusted by redirecting, focusing, collimating, filtering, and / or other means before being transmitted to the surrounding environment. In some embodiments, the light emitters 424 may use shared optical elements (e.g., a single lens shared among all light emitters 424 or a group of light emitters 424) and / or optical elements corresponding only to a single light emitter 424 (e.g., a polarizer or color filter used only by that light emitter 424) to transmit the optical signals into the surrounding environment.

[0125] In some embodiments, for example, each of the light emitters 424 may transmit optical signals to different regions of the surrounding environment to observe the field of view of the surrounding environment. The position within the surrounding environment where a given light emitter 424 can transmit an optical signal may depend on the position of the light emitter 424 (e.g., the (y, z) position of the light emitter 424 on the substrate 500), the angular orientation of the light emitter 424 relative to the surface of the substrate 500 (if present), and / or the position / orientation of the optical element (e.g., mirror and / or lens) by which the light emitter 424 provides the optical signal to the surrounding environment. As a non-limiting example, the light emitters 424 on the substrate 500 can emit optical signals into the surrounding environment over a range of azimuth and / or elevation angles (e.g., to determine the corresponding angular range within the surrounding environment) based on the position of the light emitters 424 on the substrate 500 relative to the shared telecentric lens assembly used by each of the light emitters 424, thereby providing light to the surrounding environment. From the shape of the shared telecentric lens assembly, the optical signals can spread over the entire range of azimuth and / or elevation angles.

[0126] The photodetector 426 may include various types of detectors (e.g., single-photon detectors). For example, the photodetector 426 may include a SPAD and / or a SiPM. A SPAD may use an avalanche breakdown within a reverse-biased p-n junction (i.e., a diode) to increase the output current of a given incident illumination to the SPAD. Further, a SPAD can generate multiple electron-hole pairs for a single incident photon. In some embodiments, the photodetector 426 may be biased above the avalanche breakdown voltage. Such a biasing condition may create a positive feedback loop having a loop gain greater than one. Further, a SPAD biased above the threshold avalanche breakdown voltage may be single-photon sensitive. In other examples, the photodetector 426 may include a photoresistor, a charge-coupled device (CCD), a photovoltaic cell, and / or any other type of photodetector.

[0127] In some implementations, the array of photodetectors 426 can include multiple types of photodetectors across the entire array. For example, the array of photodetectors 426 can be configured to detect multiple predetermined wavelengths of light (e.g., in embodiments where the light emitter 424 emits different wavelengths of light across the array of light emitters 424). For that purpose, for example, the array of photodetectors 426 can include some SPADs sensitive to a certain range of wavelengths and other SPADs sensitive to different ranges of wavelengths. In some embodiments, the photodetector 426 can be sensitive to wavelengths in the range of 400 nm to 1.6 μm (visible and / or infrared wavelengths). Further, the photodetectors 426 can have various sizes and shapes. For example, the photodetector 426 can include SPADs having a package size that is 1%, 0.1%, or 0.01% of the total area of the substrate 500. Further, in some embodiments, one or more of the photodetectors 426 can include detector-specific optical elements. For example, each of the photodetectors 426 can include a microlens disposed on the photodetector 426 to enhance the amount of received light transmitted to the detection surface of the photodetector 426. Additionally or alternatively, one or more of the photodetectors 426 can include one or more optical filters (e.g., can include ND filters, polarizing filters, and / or color filters).

[0128] As described above, each of the photodetectors 426 can correspond to a light emitter 424. In some embodiments, the photodetector 426 can receive light from the surrounding scene via one or more optical elements (e.g., color filters, polarizers, lenses, mirrors, and / or waveguides). Such optical elements can be specific to one of the photodetectors 426 and / or can be shared by a group of photodetectors 426 (e.g., all the photodetectors on the substrate 500). Further, in some embodiments, in addition to being part of the receiving path of one or more of the photodetectors 426, one or more of the receiving optical elements can be part of the transmitting path of one or more of the light emitters 424. For example, a mirror can reflect light from one or more of the light emitters 424 to the surrounding environment and can also direct light received from the surrounding environment towards one or more of the photodetectors 426.

[0129] As described above, the optical emitter 424 can be configured to transmit an optical signal to the surrounding environment over a range of azimuth and / or elevation angles (i.e., yaw angle and / or pitch angle). Similarly, based on the position of the photodetector 426 within the LIDAR device 410, the photodetector 426 can be arranged to receive an optical signal reflected from an object in the environment surrounding the LIDAR device 410 over a corresponding range of azimuth and / or elevation angles (i.e., yaw angle and / or pitch angle).

[0130] The array 424 of optical emitters can be powered and / or controlled by the emission circuit 428. Similarly, the photodetector 426 can be powered by the selector circuit 430, controlled thereby, and / or provide a detection signal. As shown in FIG. 5A, the emission circuit 428 may be connected to one or more optical emitters 424 by conductive traces 522 defined on the substrate 500, and the selector circuit 430 may be connected to one or more photodetectors 426 by conductive traces 522 defined on the substrate 500. FIG. 5A shows a first conductive trace 522 connecting the emission circuit 428 to the optical emitter 424 and a second conductive trace 522 connecting the selector circuit 430 to the photodetector 426. It is understood that this is provided as an example only. In other embodiments, the emission circuit 428 may be individually connected to each optical emitter 424 by separate conductive traces 522. Similarly, the selector circuit 430 may be individually connected to each of the photodetectors 426 by separate conductive traces 522. As another alternative, the emission circuit 428 may be connected to a bank of optical emitters 424 by a single conductive trace 522, and / or the selector circuit 430 may be connected to a bank of photodetectors 426 by a single conductive trace 522. For example, a group of four optical emitters 424 may be connected to the emission circuit 428 by a single conductive trace 522. In this way, a bank of four optical emitters 424 can be fired simultaneously by the emission circuit 428. Other numbers of optical emitters 424 or photodetectors 426 within the group are possible.

[0131] In some embodiments, the emission circuit 428 may include one or more capacitors. Such capacitors may be charged by one or more power supplies. Next, to cause the optical emitter 424 to emit an optical signal (i.e., “transmit”), the stored energy in the capacitor may be discharged through the optical emitter 424. In some embodiments, the emission circuit 428 may cause the optical emitter 424 to emit optical signals simultaneously with each other. In other embodiments, the emission circuit 428 may cause the optical emitter 424 to emit optical signals sequentially. Other emission patterns (including random and pseudo-random emission patterns) are also possible and are contemplated herein.

[0132] Furthermore, in some embodiments, the emission circuit 428 may be controlled by a controller (e.g., the controller 416 illustrated and described with reference to FIG. 4B). The controller 416 may selectively emit the optical emitter 424 using the emission circuit 428 via an emission control signal (e.g., according to a predetermined pattern). In some embodiments, the controller 416 may also be configured to control other functions of the LIDAR device 410. For example, the controller may control the movement of one or more movable stages associated with the LIDAR device 410, and / or generate a point cloud representation of the environment surrounding the LIDAR device 410 based on an electronic signal received from the photodetector 426 within the LIDAR device 410 corresponding to a detected optical signal reflected from an object in the environment. Generation of the point cloud representation may be performed in various embodiments based on the intensity of the detected signal compared to the intensity of the emitted signal and / or based on the timing of the detected signal compared to the timing of the emitted signal. In alternative embodiments, data regarding the detected optical signal and / or the emitted optical signal (e.g., timing data or intensity data) may be transmitted to a separate computing device (e.g., a remotely located server computing device or an in-vehicle vehicle controller such as the system controller 402 illustrated and described with reference to FIG. 4A). The separate computing device may be configured to generate the point cloud representation (e.g., store the point cloud representation in a memory such as the memory 406 and / or transmit the point cloud representation to the LIDAR controller).

[0133] The arrangement illustrated in FIG. 5A is provided by way of example, and other embodiments are also possible and contemplated herein. For example, the LIDAR device 410 may alternatively include a plurality of substrates 500, each having an optical emitter 424 and an optical detector 426 thereon. Additionally or alternatively, in some embodiments, the number of optical emitters 424 on the substrate 500 may be different from that illustrated in FIG. 5A (e.g., more or fewer than 16 optical emitters 424), the number of optical detectors 426 on the substrate 500 may be different from that illustrated in FIG. 5A (e.g., more or fewer than 16 optical detectors 426), the arrangement of the optical emitters 424 on the substrate 500 may be different from that illustrated in FIG. 5A, the arrangement of the optical detectors 426 on the substrate 500 may be different from that illustrated in FIG. 5A, the position and / or number of the conductive traces 522 may be different from that illustrated in FIG. 5A, the relative size of one or more of the optical emitters 424 may be different from that illustrated in FIG. 5A, and / or the relative size of one or more of the optical detectors 426 may be different from that illustrated in FIG. 5A. Other differences are also possible and contemplated herein.

[0134] FIG. 5B is an illustration of potential crosstalk between channels within a LIDAR device (e.g., the LIDAR device 410 illustrated and described with reference to FIGS. 4A, 4B, and 5A) according to an exemplary embodiment. For example, the LIDAR device 410 may include a substrate 500, an array of channels (e.g., each channel including an optical emitter 424 and an optical detector 426 as shown in FIG. 5A), a transmit circuit 428, a selector circuit 430, and conductive traces 522.

[0135] As an example, the LIDAR device 410 may include a first light emitter 502. The first light emitter 502 may emit an optical signal into the surrounding environment. Typically, when the emitted optical signal is reflected by a surface having a medium reflectivity, a reflected optical signal 504 of medium intensity may be returned to the LIDAR device. As shown in FIG. 5B, the reflected optical signal 504 of medium intensity may irradiate a corresponding first photodetector 514 within the LIDAR device 410. Further, the size of the reflected optical signal 504 of medium intensity may not be large enough to substantially and / or measurably irradiate other photodetectors 426 within the LIDAR device. However, when the optical signal emitted from the first light emitter 502 into the surrounding environment is reflected by a surface having a high reflectivity (e.g., a retroreflector), the intensity of the reflected signal may be higher and / or may occupy a larger detectable area when incident on the array of photodetectors 426. As illustrated in FIG. 5B, the high-intensity reflected optical signal 506 may irradiate a plurality of photodetectors 426. For example, the high-intensity reflected optical signal 506 may irradiate the first photodetector 514 as well as one or more second photodetectors 516. The second photodetector 516 may also be referred to herein as a photodetector susceptible to crosstalk (e.g., crosstalk from a reflected optical signal resulting from an emission signal from the first light emitter 502), which means that the second photodetector 516 may unnecessarily detect light from the first channel (e.g., may result in noise or inappropriate detection events based on the detection of the second photodetector 516).

[0136] Which of the photodetectors 426 within the LIDAR device can detect a given reflected signal (e.g., which photodetectors 426 are more susceptible to the effects of crosstalk) can depend on the intensity of the reflected signal (e.g., based on the reflectivity of surfaces in the surrounding environment), the sensitivity of the photodetectors 426, the position of the photodetectors 426 within the LIDAR device, the orientation of the photodetectors 426 within the LIDAR device (e.g., the azimuth / roll angle orientation and / or the elevation / pitch angle orientation of the photodetectors 426), the distance to the reflecting surface within the surrounding environment, and the like. For example, in some embodiments, the farther the distance from the LIDAR device 410 to the reflecting surface within the surrounding environment, the fewer the number of photodetectors 426 that can be affected by crosstalk (e.g., the farther the distance to the reflecting surface, the smaller the radius of the high-intensity reflected optical signal 506). This may be the result of attenuation / divergence of the reflected optical signal propagating through the surrounding environment (e.g., due to dust, smoke, etc. in the surrounding environment), such that the intensity of the optical signal decreases and the separation between the LIDAR device and the reflecting surface further progresses.

[0137] In view of the above, it is understood that in various embodiments, the reflected signal is detected by an unintended photodetector 426 of the LIDAR device 410, thereby causing crosstalk. The embodiments described herein can attempt to reduce crosstalk within the LIDAR device 410 regardless of the cause (e.g., whether the crosstalk is caused by a high-reflectivity surface and / or regardless of the separation between the LIDAR device 410 and the high-reflectivity surface).

[0138] Figures 6A and 6B illustrate techniques according to exemplary embodiments that can be used to reduce crosstalk. The techniques can include different channels of a LIDAR device (e.g., the LIDAR device 410 shown and described with respect to FIGS. 4B, 5A, and 5B) that emit optical signals according to different emission patterns and / or emission sequences and then detect reflected optical signals. For example, FIG. 6A can illustrate an optical signal emitted by the optical emitter 424 of the LIDAR device 410 and a reflected optical signal detected by the optical detector 426 of the LIDAR device 410 during a first cycle. As shown in FIG. 5A, the LIDAR device 410 may have 16 channels (e.g., numbered channels 0, channel 1, channel 2, ..., channel 15), each of which includes an optical emitter 424 and a corresponding optical detector 426. During the first cycle, the optical emitter 424 of each of the 16 channels may emit an optical signal, and if the emitted optical signal is reflected from an object in the surrounding environment, the corresponding optical detector 426 may detect the reflected optical signal. The optical detector 426 within the LIDAR device 410 may wait for the reflected optical signal during a listening window of the first cycle. This listening window may be of a duration sufficient to allow an optical signal reflected from a distant object (e.g., an object more than 150 m away, more than 200 m away, more than 250 m away, more than 300 m away, more than 350 m away, more than 400 m away, more than 450 m away, or more than 500 m away) to still be detected by the optical detector 426.

[0139] On the one hand, FIG. 6B may show the optical signal emitted by the optical emitter 424 of the LIDAR device 410 and the reflected optical signal detected by the photodetector 426 of the LIDAR device 410 during the second cycle. As shown in FIG. 6B, only a subset of the channels may emit light during the second cycle (e.g., between sequential listening windows). Such an emission strategy may prevent crosstalk from occurring due to adjacent channels emitting / detecting simultaneously. For example, as illustrated, only the optical emitters 424 of the even-numbered channels (e.g., channel 0, channel 2, channel 4) may emit an optical signal during the second cycle. Similar to the first cycle, the photodetector 426 within the LIDAR device 410 can wait for the reflected optical signal during the listening window of the second cycle. However, this second listening window may only be of sufficient duration to enable the optical signal reflected from a relatively nearby object (e.g., an object located less than 150 m, less than 125 m, less than 100 m, less than 75 m, or less than 50 m away) to be detected by the photodetector 426. The listening window during the second cycle may be shorter than the listening window during the first cycle, so the overall duration of the second cycle may be shorter than the overall duration of the first cycle. Alternatively (e.g., when the overall duration of the second cycle is the same as or greater than the overall duration of the first cycle), one or more of the emissions from the optical emitter 424 during the second cycle may be offset in time relative to each other. This may provide additional robustness against crosstalk between channels as their emission / detection windows during the second cycle do not overlap with each other (e.g., as a result of the time offset).

[0140] By combining the detection ability of a longer distance in the first cycle with the detection ability of a shorter distance in the second cycle but with higher crosstalk tolerance, an enhanced data set (e.g., usable to generate one or more point clouds) can be generated. For example, a complete emission cycle of the LIDAR device 410 can include a second cycle following the first cycle. During the emission cycle, a plurality of emission / detection events between the first and second cycles are recorded and combined to generate a data set that reduces the negative effects of crosstalk.

[0141] FIG. 6C illustrates an emission diagram of a first cycle (e.g., the first cycle illustrated and described with reference to FIG. 6A). “1” indicates that each channel is emitting an optical signal at a specified time point, while “0” indicates that each channel is holding back the emission of the optical signal. Thus, as illustrated in FIG. 6C, each of the optical emitters 424 of each channel of the LIDAR device 410 may be emitted simultaneously. Thereafter, a single listening window having a sufficient duration for relatively distant objects may be used by the photodetectors 426 of the channels of the LIDAR device 410. For example, a listening window of 2.0 μs to 3.0 μs (e.g., at 2.5 μs, which corresponds to a distance of 375 m) may be used. By emitting a group of optical emitters 424 simultaneously in a plurality of channels of the LIDAR device 410 (e.g., by simultaneously emitting all of the optical emitters 424 of all channels of the LIDAR device 410 as shown in FIG. 6C), the adverse effects of internal reflections within the LIDAR device 410 can be reduced. For example, after the optical emitters 424 are emitted, there may be a short period during which reflections from the internal components of the LIDAR device 410 are detected by one or more of the photodetectors 426 of the LIDAR device 410. Such detected internal reflections can effectively prevent those photodetectors 426 from detecting signals reflected from the surrounding environment during that short period (i.e., effectively blocking the photodetectors 426). By emitting a plurality of optical emitters 424 simultaneously, the times of internal reflections based on the emission of those optical emitters 424 overlap, thereby reducing the overall time during which the photodetectors 426 within the LIDAR device 410 are blocked (e.g., when compared to an alternative sequential emission sequence).

[0142] On the one hand, FIG. 6D shows an emission diagram of a second cycle (e.g., the second cycle illustrated and described with reference to FIG. 6B). As shown in FIG. 6D, the second cycle may include a plurality of emission times and a plurality of corresponding listening windows. During each emission time, only a single channel may emit an optical signal. For example, as illustrated, during the first emission time of the second cycle, the optical emitter 424 of channel 0 may emit an optical signal. After channel 0 emits the optical signal, the photodetector 426 of the LIDAR device 410 may attempt to detect a reflected signal for a listening window having a duration sufficient for relatively nearby objects. For example, a listening window of 0.3 μs to 0.7 μs (e.g., 0.5 μs, which corresponds to a distance of 75 m) may be used. Thereafter, during the second emission time of the second cycle, the optical emitter 424 of channel 2 may emit an optical signal. After channel 2 emits the optical signal, the photodetector 426 of the LIDAR device 410 may again attempt to detect a reflected signal of a listening window (e.g., a listening window having the same duration or a different duration as the previous listening window). This process continues for channel 4, then channel 6, then channel 8, then channel 10, then channel 12, and finally channel 14. This is shown in FIG. 6D by three dots adjacent to the fifth listening window.

[0143] In an alternative embodiment, until the total time allocated to the second cycle elapses, during sequential emission times, emissions can be sequentially made to every other channel (e.g., if 3.0 μs is allocated to the second cycle, only 6 emission times / each 0.5 μs listening window may be used). The channels may be selected such that the channels used during the second cycle are uniformly distributed within the photodetector 426 of the LIDAR device 410. Additionally or alternatively, in some embodiments, the channels (e.g., channels 0 to 15) are in an interleaved pattern during the second cycle (e.g., channel 0, then channel 2, then channel 4, then channel 6, then channel 8, then channel 10, then channel 12, then channel 14, then channel 1, then channel 3, then channel 5, then channel 7, then channel 9, then channel 11, then channel 13, then channel 15 or channel 0, then channel 3, then channel 6, then channel 9, then channel 12, then channel 15, then channel 1, then channel 4, then channel 7, then channel 10, then channel 13, then channel 2, then channel 5, then channel 8, then channel 11, then channel 14) may be emitted. In yet other embodiments, the channels used during the second cycle can be determined based on the degree of contamination (e.g., existing condensation, existing rain, existing snow, existing ice, existing cracks, existing insect debris, and / or existing dust) on one or more optical elements (e.g., windows, lenses, and / or mirrors) of the LIDAR device 410. For example, if there is a crack in the optical window located in front of the optical emitter 424 or photodetector 426 of a given channel, that channel can be avoided during the second cycle. The degree of contamination may be based on previous measurements made using the LIDAR device 410 or a different sensor, and / or based on ambient weather conditions (e.g., weather prediction, current temperature, and / or Doppler radar data) near the LIDAR device 410. After the second cycle is completed, the detection results from the second cycle can be combined with the detection results from the first cycle to generate a dataset that can be used to generate one or more point clouds.

[0144] The channels used during the second cycle (e.g., repeated with successive emission times / listening windows) can be selected according to various methodologies. For example, one or more of the arrangements described above (e.g., every other channel, every two channels, and / or channels interleaved in pairs) may be stored in the memory of the controller of the LIDAR device. Such memory may include a predetermined list of which channels are to be emitted in which order during the second cycle. In such embodiments, the order of the channels used is fixed and may be repeated throughout the emission cycle. In some embodiments, for example, the channels may be emitted in a brute-force manner during the second cycle of successive emission cycles according to a stored predetermined list.

[0145] Alternatively, which channels are selected for use during a second cycle may be based on one or more previous emission cycles and / or based on the first cycle of each emission cycle. For example, by analyzing the first cycle, high-intensity return signals can be identified. Such high-intensity return signals may indicate the presence of one or more highly reflective surfaces (e.g., retroreflectors) in the surrounding environment. Further, such highly reflective surfaces may be more likely to cause crosstalk. Therefore, the channel in which a high-intensity return was detected during the first cycle (e.g., the channels within a predetermined angle of that channel within the LIDAR device) may not be used during the second cycle. In yet other embodiments, however (e.g., and perhaps more likely), the channels near the detected high-intensity return signal may be intentionally probed during the second cycle. The channels near the channel that received the high-intensity return during the first cycle are most likely to have been affected by crosstalk during the first cycle, so it may be most beneficial to probe those channels separately during the second cycle. Thus, the channels within a predetermined separation angle from the channel that received the high-intensity return during the first cycle may be repeated during the second cycle (e.g., sequentially emitted during the emission slots of the second cycle).

[0146] In yet other embodiments, the channels may be selected for emission during the second cycle in order to provide coverage robustness across the entire field of view as much as possible during the second cycle. For example, a subset of channels (e.g., pairs of channels, triplets of channels, quadruplets of channels) may be selected for each emission slot of the second cycle. Each channel within a given subset may be selected such that a minimum predetermined angular resolution condition is met. For example, the channels may be selected such that each channel within each subset of channels emitted during each emission slot of the second cycle is separated by at least a predetermined number of degrees in azimuth and / or elevation. The predetermined number of degrees may be determined based on one or more optical components of the LIDAR device (e.g., aperture, lens, waveguide, mirror, and / or window).

[0147] FIG. 6E is an illustrative view of channels within LIDAR device 410 that can emit and detect an optical signal during a second cycle when emitting and detecting the optical signal according to the emission sequence illustrated and described in relation to FIG. 6D. For example, the channels used during the second cycle of FIG. 6D are shown in the box surrounded by the dashed line. As shown in FIG. 6E, the optical detectors 426 used are such that light reflected from an object in the surrounding environment (e.g., reflected during a previous listening window) does not reach the detection surface of the LIDAR device 410 until a subsequent listening window and crosstalk cannot be detected (e.g., because adjacent optical detectors 426 within the LIDAR device 410 are not used even in sequential listening windows), and may be time-differential (e.g., perpendicular along the z-direction) with respect to each other. It is understood that other arrangements of the other optical detectors 426 can be used during the second cycle (e.g., the arrangement illustrated in FIG. 6F).

[0148] Furthermore, the emission sequence illustrated in FIG. 6D is an example, and it is understood that other emission patterns are possible for the second cycle and are contemplated herein. For example, each of the channels (rather than every other channel) may be configured to emit light during sequential emission times / listening windows (e.g., channel 0, then channel 1, then channel 2, then channel 3). Alternatively, every other channel (rather than every other channel) may be configured to emit light during sequential emission times / listening windows (e.g., channel 0, then channel 3, then channel 6, then channel 9). The spacing between channels that are emitted during sequential time windows may be based on the physical spacing of the optical detectors 426 of the LIDAR device 410. For example, how close the optical detectors 426 are to each other may be used to determine the number of subsequent channels that are susceptible to the effects of crosstalk from adjacent channels, and only channels that are not affected by crosstalk with each other may be selected for use in adjacent emission times / listening windows.

[0149] In yet other embodiments, multiple channels may be used during each transmit time / listening window during the second cycle. For example, as illustrated in FIG. 6G, two channels may be transmitted during each transmit time of the second cycle. In some embodiments, the two channels selected for simultaneous transmission may be as far apart as possible from each other within the LIDAR device 410 (e.g., by channel index and / or physical location within the LIDAR device 410). This can prevent crosstalk between the two channels used during the associated listening window. For example, as illustrated, during the first transmit time / listening window of the second cycle, the optical emitters 424 of channel 0 and channel 8 may emit optical signals, while during the second transmit time / listening window of the second cycle, the optical emitters 424 of channel 1 and channel 9 may emit optical signals. This can potentially allow for the use of a greater total number of channels during the second cycle (e.g., thereby increasing the resolution of the resulting dataset), yet still prevent the possibility of crosstalk. Further, in some embodiments, the channels selected for simultaneous transmission during the second cycle may be selected such that crosstalk does not occur between the channels even when irradiating a highly reflective object (e.g., a retroreflector) located at the maximum detectable distance based on the duration of the second listening window. Throughout this disclosure, the phrase "such that crosstalk does not occur between the channels" is used. Of course, embodiments that do not literally provide no crosstalk between the channels are clearly contemplated, but embodiments that provide substantially reduced crosstalk are also contemplated by this phrase. For example, in some embodiments, there may be a minimum threshold intensity used for detection by the photodetector of the LIDAR device. Below that minimum threshold intensity, a detection event may not be registered (e.g., by the photodetector and / or a computing device analyzing detection data from the photodetector).In such embodiments, the phrase "such that crosstalk does not occur between channels" may correspond to a level of crosstalk between channels that is lower than a minimum threshold intensity (e.g., on the one hand, maintaining a crosstalk signal between channels having non-zero intensities).

[0150] In yet other embodiments, three channels, four channels, five channels, etc. may be configured to emit / detect optical signals during each emission time / listing window. Further, in some embodiments, different numbers of channels may be used during different portions of the second cycle. For example, as shown in FIG. 6H, channels 0, 7, and 14 are used during the first emission time / listing window of the second cycle, and then channels 1, 5, 9, and 13 follow during the second emission time / listing window of the second cycle, and then channels 2, 8, and 15 may follow during the third emission time / listing window of the second cycle. Of course, other embodiments for the emission sequence of the second cycle are possible and are contemplated herein. For example, the duration of the sequential listing windows of the second cycle may vary from listing window to listing window (e.g., a listing window of 0.3 μs, followed by a listing window of 0.5 μs, followed by a listing window of 0.7 μs, followed by a listing window of 0.5 μs, followed by a listing window of 0.3 μs).

[0151] The embodiments described herein may utilize the fact that when detecting nearby objects in the ambient environment, the channels of the LIDAR device 410 can be linearly over-resolved (i.e., have a higher resolution than necessary). This concept is shown in FIGS. 7A and 7B. FIG. 7A shows the LIDAR device 410 emitting an optical signal at a first angular resolution during a first cycle, and FIG. 7B shows the LIDAR device 410 emitting an optical signal at a second angular resolution during a second cycle. As shown, the first angular resolution in FIG. 7A (e.g., the number of emitted optical signals per degree) is higher than the second angular resolution in FIG. 7B. Further, the listening window of the first cycle may correspond to a wider range than the listening window of the second cycle. For example, as shown, the listening window of the first cycle may correspond to 300 m, while the listening window of the second cycle may correspond to 50 m (the same principle illustrated will apply to ranges different from those shown). As shown, the linear resolution (e.g., represented by a line corresponding to 1 cm along the y direction in FIGS. 7A and 7B) may be the same for the two emission patterns for the emission pattern of the second cycle, despite the lower second angular resolution. In other words, both emission patterns may be able to resolve objects up to 1 cm in their respective ranges, even if they do not exhibit the same angular resolution. Thus, the results of the second cycle for nearby objects may still be usable even if certain optical signals are dropped out from the emission pattern to prevent crosstalk.

[0152] In some embodiments, to generate different firing patterns used between the first cycle and the second cycle (e.g., as illustrated in FIGS. 6A and 6B), the firing circuit may be designed to accommodate both the firing sequence of the first cycle and the firing sequence of the second cycle. As an example, FIG. 8A shows a firing circuit 428 along with an associated optical emitter 424 of a LIDAR device (e.g., the LIDAR device 410 shown and described with reference to FIG. 4B). As shown in FIG. 8A, each channel (e.g., channels 0 through 15) may have a corresponding signal line (e.g., CHG0 through CHG15) that can be used to select whether that channel is fired during a given firing time. Using these signal lines, a charging switch (e.g., a transistor) can be inverted to charge the respective capacitor (e.g., the firing voltage, V LASER ) for each channel in question. Next, at the desired firing time, by closing a firing switch (e.g., a transistor) using a trigger control signal so that current flows from the capacitor through each optical emitter 424, one or more optical emitters 424 (e.g., laser diodes) can be fired.

[0153] FIG. 8A is provided merely as an example, and it is understood that other firing circuits 428 are possible and contemplated herein. As an alternative, if the optical emitters 424 are fired in groups of four (e.g., during the second cycle), the arrangement of the firing circuit 428 and optical emitters 424 of FIG. 8B may be used. Unlike the firing circuit 428 of FIG. 8A, the firing circuit 428 of FIG. 8B may include only four signal lines (e.g., CHG0-CHG3). Each signal line can be used to select whether to fire a particular group of four channels during a given firing time / associated listening window. As shown in FIG. 8B, the signal lines are for four respective capacitors for each of the four channels (e.g., the firing voltage, V LASER) can be used to charge it. Next, at a desired emission time, using a trigger control (i.e., discharge) signal (e.g., signals DIS0 to DIS3), for a given group of optical emitters 424, to allow current to flow through each of the optical emitters 424 in the group from the charged capacitor, by closing four corresponding emission switches (e.g., transistors), one or more of the four optical emitters 424 (e.g., laser diodes) can be made to emit.

[0154] The techniques used to reduce or eliminate the above crosstalk can be enhanced by additional or alternative techniques. As an example of an additional crosstalk reduction technique that can be employed in conjunction with the above first cycle / second cycle techniques, the identification of specific objects in the surrounding environment can be used to prevent crosstalk in future detection cycles. For example, highly reflective (e.g., retroreflective) objects in the surrounding environment can generate high-intensity reflections, which can cause crosstalk. Thus, if a highly reflective object is identified in a given detection cycle (e.g., based on the intensity of the reflection detected from the object), the LIDAR device 410 may refrain from emitting an optical signal towards that object in future detection cycles. For example, FIG. 9 shows an alternative set of potential signals emitted during the second cycle. The signals emitted during the second cycle of FIG. 9 (e.g., sequentially during different emission times / listening windows) may be similar to the signals emitted during the second cycle of FIG. 6B. However, the traffic signs illustrated in FIGS. 6B and 9 may have highly reflective portions (e.g., the retroreflector of the sign may include a retroreflector). Thus, to further reduce crosstalk, the embodiment shown in FIG. 9 may also refrain from emitting an optical signal towards a portion of the scene that includes the highly reflective portion of the traffic sign (e.g., in FIG. 9, unlike FIG. 6B, the optical emitter 424 of channel 10 may refrain from emitting an optical signal during the second cycle). Highly reflective objects within the surrounding environment can be identified based on the optical signals emitted into the surrounding environment for the purpose of detection during one or more previous detection cycles. Additionally or alternatively, for the purpose of identifying highly reflective objects prior to the emission / detection being performed at runtime, the highly reflective objects within the surrounding environment can be identified based on the calibration signals emitted into the surrounding environment by the optical emitter 424 of the LIDAR device 410.

[0155] In addition to, or instead of, adjusting the listening window and / or angular resolution associated with the first cycle / second cycle as described above, power modulation can also be implemented to clarify the crosstalk signal from the detection signal. For example, a lower emission power can be used for the optical signal emitted during the second cycle rather than for the optical signal emitted during the first cycle. This dichotomy of emission power can result in energy savings (e.g., the emitted optical signal does not have to travel as far during the second cycle since the listening window / range is shorter), reduced charging time (e.g., for the capacitor used to fire the optical emitter 424 based on the RC or RLC time constant of the charging circuit), reduced likelihood of inducing crosstalk in adjacent channels (e.g., due to lower intensity reflections as a result of the reduced emission power), reduced amount of dynamic range required for the photodetector / receiver of the LIDAR device, and / or prevention of saturation of the photodetector during the second cycle. In some embodiments, for example, the emission power used during the second cycle may be less than 75%, less than 50%, less than 25%, or less than 10% of the emission power used during the first cycle. Regardless of the differences used between the first cycle and the second cycle, those described herein

[0156] Embodiments also include techniques for combining a detection event of a first cycle and a detection event of a second cycle. In some embodiments, the LIDAR device 410 may simply generate a data set including two segments of data, one segment (e.g., corresponding to the first cycle of the emission / detection signal) that can be used to generate a first point cloud, and one segment (e.g., corresponding to the second cycle of the emission / detection signal) that can be used to generate a second point cloud. In other embodiments, the LIDAR device 410 (e.g., the controller 416 of the LIDAR device 410) may determine, for each channel used between both the first cycle and the second cycle, a target line-of-sight distance based on detection events during the first cycle, and a target line-of-sight distance based on detection events during the second cycle. Next, the LIDAR device 410 (e.g., the controller 416 of the LIDAR device 410) may determine the difference between the two line-of-sight distances for each channel. Next, for each channel, the LIDAR device 410 (e.g., the controller 416 of the LIDAR device 410) may compare the difference in line-of-sight distance to a threshold difference value (e.g., 0.1 to 5.0 m, e.g., 0.5 m, 1.0 m, or 2.5 m), and if the difference is less than the threshold difference value, include the line-of-sight distance from one of the cycles (e.g., the second cycle or the first cycle) in a data set that can be used to generate a single point cloud representing a combination of detection events between the two cycles. If the difference value is greater than the threshold difference, the LIDAR device 410 (e.g., the controller 416 of the LIDAR device 410) may instead calculate a hybrid distance representing some combination of the two measurements that excludes both line-of-sight distances, includes by default one of the line-of-sight distances, and include that hybrid distance (e.g., with an associated confidence level based on the difference value).

[0157] Figure 10 is a flowchart diagram of method 1000 according to an exemplary embodiment. In some embodiments, method 1000 may be implemented to reduce crosstalk from adjacent channels within a LIDAR device. In some embodiments, method 1000 may be implemented by a system that includes a LIDAR device (e.g., LIDAR device 410 illustrated in FIGS. 4B and 5A).

[0158] At block 1002, method 1000 may include emitting, from a first group of optical emitters of a light detection and ranging (LIDAR) device, a first group of optical signals into the surrounding environment. The first group of optical signals may correspond to a first angular resolution with respect to the surrounding environment.

[0159] At block 1004, method 1000 may include detecting, by a first group of optical detectors of the LIDAR device, a first group of reflected optical signals from the surrounding environment during a first listening window. The first group of reflected optical signals may correspond to reflections of the first group of optical signals from objects in the surrounding environment.

[0160] At block 1006, method 1000 may include emitting, from a second group of optical emitters of the LIDAR device, a second group of optical signals into the surrounding environment. The second group of optical emitters of the LIDAR device may represent a subset of the first group of optical emitters of the LIDAR device. The second group of optical signals may correspond to a second angular resolution with respect to the surrounding environment. The second angular resolution may be lower than the first angular resolution.

[0161] At block 1008, method 1000 may include detecting, by a second group of optical detectors of the LIDAR device, a second group of reflected optical signals from the surrounding environment during a second listening window. The second group of optical detectors of the LIDAR device may represent a subset of the first group of optical detectors of the LIDAR device. The second group of reflected optical signals may correspond to reflections of the second group of optical signals from objects in the surrounding environment. The duration of the second listening window may be shorter than the duration of the first listening window.

[0162] In block 1010, method 1000 may include synthesizing a data set that can be used by a controller of a LIDAR device to generate one or more point clouds. The data set may be based on a detected first group of reflected light signals and a detected second group of reflected light signals.

[0163] In some embodiments, method 1000 may also include emitting a third group of optical signals from a third group of optical emitters of the LIDAR device into the surrounding environment. The third group of optical emitters of the LIDAR device may represent a subset of the first group of optical emitters of the LIDAR device, rather than the second group of optical emitters. The third group of optical signals may correspond to a third angular resolution with respect to the surrounding environment. The third angular resolution may be the same as the second angular resolution. Method 1000 may also include detecting, by a third group of optical detectors of the LIDAR device, a third group of reflected light signals from the surrounding environment during a third listening window. The third group of optical detectors of the LIDAR device may represent a subset of the first group of optical detectors of the LIDAR device that is different from the second group of optical detectors. The third group of reflected light signals may correspond to the reflection of the third group of optical signals from an object in the surrounding environment. The duration of the third listening window may be the same as the duration of the second listening window. The data set may be based on the detected third group of reflected light signals.

[0164] In some embodiments of method 1000, the third listening window may not overlap with the second listening window.

[0165] In some embodiments of method 1000, the second group of optical signals may include a plurality of optical signals. The third group of optical signals may include a plurality of optical signals. The group of the second group of optical signals and the third group of optical signals may be interleaved with respect to the surrounding environment.

[0166] In some embodiments of method 1000, the second group of photodetectors may include a plurality of photodetectors. The second group of photodetectors may be selected from the first group of photodetectors so as to be uniformly distributed across the first group of photodetectors.

[0167] In some embodiments of method 1000, when the second group of photodetectors irradiate a retroreflector located at the maximum detectable distance with the optical signals within the group of optical signals of the second group, the second group of photodetectors may be dispersed with sufficient gaps throughout the first group of photodetectors so that crosstalk does not occur among the photodetectors within the second group. The maximum detectable distance may be based on the duration of the second listening window.

[0168] In some embodiments, method 1000 may also include emitting calibration optical signals from each optical emitter of the LIDAR device into the surrounding environment. Further, method 1000 may include detecting, by each of the photodetectors of the LIDAR device during a calibration listening window, the reflected calibration optical signals from the surrounding environment. Each reflected calibration optical signal may correspond to a reflection of one of the calibration optical signals from an object within the surrounding environment. Further, method 1000 may include identifying, based on the detected reflected calibration optical signals, one or more optical emitters within the LIDAR device for which the corresponding calibration optical signal was reflected from a retroreflector within the surrounding environment. Further, method 1000 may include selecting, from the set of all emitters of the LIDAR device, a first group of optical emitters of the LIDAR device. The first group of optical emitters may correspond to those optical emitters that were not identified as one or more optical emitters within the LIDAR device for which the corresponding calibration optical signal was reflected from a retroreflector within the surrounding environment.

[0169] In some embodiments of method 1000, one or more optical emitters within the LIDAR device for which the corresponding calibration optical signal was reflected from a retroreflector within the surrounding environment may be identified based on the detected intensity of the corresponding detected reflected calibration optical signal from the surrounding environment.

[0170] In some embodiments of method 1000, emitting a group of first optical signals into the ambient environment may include emitting the group of first optical signals into the ambient environment using a first emission power. Emitting a group of second optical signals into the ambient environment may include emitting the group of second optical signals into the ambient environment using a second emission power. The second emission power may be less than the first emission power.

[0171] In some embodiments of method 1000, the second emission power may be less than 25% of the first emission power.

[0172] In some embodiments of method 1000, the dataset may be usable to generate a first point cloud and a second point cloud. The first point cloud may include data related to the detected reflected optical signals of the first group. The second point cloud may include data related to the detected reflected optical signals of the second group.

[0173] In some embodiments of method 1000, synthesizing the dataset may include, for each detected reflected optical signal in the detected reflected optical signals of the second group, determining a second target distance based on each detected reflected optical signal. Synthesizing the dataset may also include, for each detected reflected optical signal in the detected reflected optical signals of the second group, determining a first target distance based on the corresponding detected reflected optical signal in the detected reflected optical signals of the first group. The corresponding detected reflected optical signal in the detected reflected optical signals of the first group may be detected by the same photodetector within the LIDAR device. Further, synthesizing the dataset may include, for each detected reflected optical signal in the detected reflected optical signals of the second group, determining the difference between the second target distance and the first target distance. Further, synthesizing the dataset may include, for each detected reflected optical signal in the detected reflected optical signals of the second group, including the second target distance or the first target distance in the dataset if the difference is less than a threshold difference value.

[0174] In some embodiments of method 1000, the threshold difference value can be from 0.1 m to 5.0 m.

[0175] In some embodiments of method 1000, the duration of the first listening window can be from 2.0 μs to 3.0 μs. The duration of the second listening window can be from 0.3 μs to 0.5 μs.

[0176] In some embodiments, method 1000 can also include determining which of the light emitters within the first group of light emitters are to be included in the second group of light emitters based on the degree of contamination of one or more optical elements of the LIDAR device.

[0177] In some embodiments of method 1000, the degree of contamination can be determined based on previous measurements using the LIDAR device or a different sensor.

[0178] In some embodiments of method 1000, the degree of contamination can be determined based on ambient weather conditions near the LIDAR device.

[0179] In some embodiments of method 1000, the data set can include a plurality of points associated with each of the detected reflected light signals in the second group of detected reflected light signals. Each of the plurality of points can include a target distance. Each target distance can have an associated confidence level determined based on the detected reflected light signal in the second group of detected reflected light signals and the corresponding first detected reflected light signal in the first group of detected reflected light signals.

[0180] This disclosure is not limited to the specific embodiments described in this application, and the specific embodiments are intended as illustrations of various aspects. As will be apparent to those skilled in the art, many modifications and variations can be made without departing from the spirit and scope of this disclosure. In addition to the methods and apparatuses recited herein, functionally equivalent methods and apparatuses within the scope of this disclosure will be apparent to those skilled in the art from the foregoing description. Such modifications and variations are intended to be within the scope of the appended claims.

[0181] The foregoing detailed description has described various features and functions of the disclosed systems, devices, and methods with reference to the accompanying drawings. In the figures, like symbols typically refer to like components unless the context indicates otherwise. The exemplary embodiments described herein and in the figures are not meant to be limiting. Other embodiments may be utilized and other changes may be made without departing from the scope of the subject matter presented herein. It will be readily understood that the aspects of this disclosure generally described herein and illustrated in the figures can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are explicitly contemplated.

[0182] For any or all of the message flow diagrams, scenarios, and flowcharts shown in the figures and discussed herein, each step, block, operation, and / or communication may represent the processing and / or transmission of information according to an exemplary embodiment. Alternative embodiments are within the scope of these exemplary embodiments. In these alternative embodiments, for example, operations described as steps, blocks, transmissions, communications, requests, responses, and / or messages may be performed in an order different from that shown or discussed, such as substantially simultaneously or in the reverse order, depending on the relevant functions. Further, more or fewer blocks and / or operations may be used in any of the message flow diagrams, scenarios, and flowcharts discussed herein, and these message flow diagrams, scenarios, and flowcharts may be combined with each other, in part or in whole.

[0183] Steps, blocks, or operations corresponding to the processing of information may correspond to circuitry configured to perform specific logical functions of the methods or techniques described herein. Alternatively or additionally, steps or blocks corresponding to the processing of information may correspond to a portion of a module, segment, or program code (including associated data). The program code may include one or more instructions executable by a processor for performing specific logical operations or actions in a method or technique. The program code and / or associated data may be stored on any type of computer-readable medium, such as a storage device including RAM, disk drive, solid state drive, or another storage medium.

[0184] Further, steps, blocks, or operations corresponding to one or more transmissions of information may correspond to the transmission of information between software modules and / or hardware modules in the same physical device. However, other transmissions of information may be the transmission of information between software modules and / or hardware modules in various physical devices.

[0185] The specific arrangements shown in the figures should not be regarded as limitations. It should be understood that other embodiments can include more or fewer of each of the elements shown in a given figure. Further, some of the illustrated elements can be combined or omitted. Still further, the exemplary embodiments can include elements not shown in the figures.

[0186] Although various aspects and embodiments are disclosed herein, other aspects and embodiments will be apparent to those of ordinary skill in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and not intended to be limiting, and the true scope is defined by the following claims.

Claims

Claim 1 Emitting a first group of optical signals from a first group of optical emitters of a light detection and ranging (LiDAR) device into the surrounding environment, wherein the first group of optical signals corresponds to a first angular resolution with respect to the surrounding environment; During a first listening window, detecting, by a first group of optical detectors of the LiDAR device, a first group of reflected optical signals from the surrounding environment, wherein the first group of reflected optical signals corresponds to reflections of the first group of optical signals from objects within the surrounding environment; Emitting a second group of optical signals from a second group of optical emitters of the LiDAR device into the surrounding environment, wherein the second group of optical emitters of the LiDAR device represents a subset of the first group of optical emitters of the LiDAR device, the second group of optical signals corresponds to a second angular resolution with respect to the surrounding environment, and the second angular resolution is lower than the first angular resolution; During a second listening window, detecting, by a second group of optical detectors of the LiDAR device, a second group of reflected optical signals from the surrounding environment, wherein the second group of optical detectors of the LiDAR device represents a subset of the first group of optical detectors of the LiDAR device, the second group of reflected optical signals corresponds to reflections of the second group of optical signals from objects within the surrounding environment, and the duration of the second listening window is shorter than the duration of the first listening window; Synthesizing, by a controller of the LiDAR device, a dataset that can be used to generate one or more point clouds, the dataset being based on the detected first group of reflected optical signals and the detected second group of reflected optical signals. A method comprising: Claim 2 Emitting a third group of optical signals from a third group of optical emitters of the LiDAR device into the surrounding environment, wherein the third group of optical emitters of the LiDAR device represents a subset of the first group of optical emitters of the LiDAR device that is not the second group of optical emitters, the third group of optical signals corresponds to a third angular resolution with respect to the surrounding environment, and the third angular resolution is the same as the second angular resolution; During a third listening window, detecting, by a third group of photodetectors of the LIDAR device, a third group of reflected light signals from the surrounding environment, wherein the third group of photodetectors of the LIDAR device represents a subset of the first group of photodetectors of the LIDAR device that is not the second group of photodetectors, the third group of reflected light signals corresponding to reflections of the third group of optical signals from objects within the surrounding environment, and the duration of the third listening window being the same as the duration of the second listening window, and including detecting; The method according to claim 1, wherein the dataset is based on the detected third group of reflected light signals. **Claim 3** The method according to claim 2, wherein the third listening window does not overlap with the second listening window. **Claim 4** The method according to claim 2, wherein the second group of optical signals includes a plurality of optical signals, the third group of optical signals includes a plurality of optical signals, and the second group of optical signals and the third group of optical signals are emitted spatially alternately in the surrounding environment. **Claim 5** The method according to claim 1, wherein the second group of photodetectors includes a plurality of photodetectors and is selected from the first group of photodetectors such that the second group of photodetectors is uniformly distributed throughout the first group of photodetectors. **Claim 6** The method according to claim 1, wherein when the second group of photodetectors irradiates a retroreflector located at the maximum detectable distance with an optical signal within the group of the second group of optical signals, the second group of photodetectors are dispersed with gaps throughout the first group of photodetectors so that crosstalk does not occur between the photodetectors within the second group, and the maximum detectable distance is based on the duration of the second listening window. **Claim 7** Emitting, from each of the optical emitters within the LIDAR device, a calibration optical signal into the surrounding environment; During a calibration listening window, detecting, by each of the photodetectors of the LIDAR device, a reflected calibration optical signal from the surrounding environment, each reflected calibration optical signal corresponding to one reflection of the calibration optical signal from an object within the surrounding environment, and including detecting; Identifying, based on the detected reflected calibration optical signals, one or more optical emitters within the LIDAR device from which the corresponding calibration optical signal was reflected from a retroreflector in the surrounding environment; Selecting, from the set of all emitters within the LIDAR device, the first group of optical emitters of the LIDAR device, wherein the first group of optical emitters corresponds to optical emitters within the LIDAR device that were not identified as the one or more optical emitters from which the corresponding calibration optical signal was reflected from a retroreflector in the surrounding environment, and further comprising: selecting, and the method according to claim 1.

8. The method according to claim 7, wherein the one or more optical emitters within the LIDAR device from which the corresponding calibration optical signal was reflected from a retroreflector in the surrounding environment are identified based on the intensity detected for the corresponding detected reflected calibration optical signal from the surrounding environment.

9. Emitting the first group of optical signals into the surrounding environment includes emitting the first group of optical signals into the surrounding environment using a first emission power, emitting the second group of optical signals into the surrounding environment includes emitting the second group of optical signals into the surrounding environment using a second emission power, and the second emission power is less than the first emission power, the method according to claim 1.

10. The method according to claim 9, wherein the second emission power is less than 25% of the first emission power.

11. The dataset is usable to generate a first point cloud and a second point cloud, the first point cloud includes data related to the detected reflected optical signals of the first group, and the second point cloud includes data related to the detected reflected optical signals of the second group, the method according to claim 1.

12. Synthesizing the dataset includes, for each of the detected reflected optical signals within the detected reflected optical signals of the second group, determining a second target distance based on each of the detected reflected optical signals; determining a first target distance based on the corresponding detected reflected optical signal within the detected reflected optical signals of the first group, wherein the corresponding detected reflected optical signal within the detected reflected optical signals of the first group is detected by the same photodetector within the LIDAR device; determining the difference between the second target distance and the first target distance; When the difference is less than the threshold difference value, including including the second target distance or the first target distance in the dataset, the method according to claim 1.

13. The method according to claim 12, wherein the threshold difference value is 0.1 m to 5.0 m.

14. The method according to claim 1, wherein the duration of the first listening window is 2.0 μs to 3.0 μs, and the duration of the second listening window is 0.3 μs to 0.5 μs.

15. The method according to claim 1, further comprising determining which of the light emitters in the first group of light emitters are included in the second group of light emitters based on the degree of contamination of one or more optical elements of the LIDAR device.

16. The method according to claim 15, wherein the degree of contamination is determined based on previous measurements using the LIDAR device or a different sensor.

17. The method according to claim 15, wherein the degree of contamination is determined based on ambient weather conditions near the LIDAR device.

18. The dataset includes a plurality of points associated with each of the detected reflected light signals in the second group of detected reflected light signals, each of the plurality of points includes a target distance, and each target distance is based on the detected reflected light signal in the second group of detected reflected light signals and the corresponding first detected reflected light signal in the first group of detected reflected light signals. The method according to claim 1, having a related confidence level determined thereby.

19. A light detection and ranging (LIDAR) device, A first group of light emitters configured to emit a first group of light signals into the surrounding environment, wherein the first group of light signals corresponds to a first angular resolution with respect to the surrounding environment, a first group of light signals, A first group of light detectors configured to detect a first group of reflected light signals from the surrounding environment during a first listening window, wherein the first group of reflected light signals corresponds to the reflection of the first group of light signals from an object within the surrounding environment, a first group of light detectors, A second group of optical emitters configured to emit a second group of optical signals into the surrounding environment, wherein the second group of optical emitters of the LIDAR device represents a subset of the first group of optical emitters of the LIDAR device, the second group of optical signals corresponds to a second angular resolution with respect to the surrounding environment, and the second angular resolution is lower than the first angular resolution, and a second group of optical emitters. A second group of optical detectors configured to detect a second group of reflected optical signals from the surrounding environment during a second listening window, wherein the second group of optical detectors of the LIDAR device represents a subset of the first group of optical detectors of the LIDAR device, the second group of reflected optical signals corresponds to the reflection of the second group of optical signals from an object within the surrounding environment, and the duration of the second listening window is shorter than the duration of the first listening window, and a second group of optical detectors. A controller configured to synthesize a dataset that can be used to generate one or more point clouds, wherein the dataset is based on the detected first group of reflected optical signals and the detected second group of reflected optical signals, and a controller. A light detection and ranging (LIDAR) device comprising.

20. A system, A light detection and ranging (LIDAR) device, A first group of optical emitters configured to emit a first group of optical signals into the surrounding environment, wherein the first group of optical signals corresponds to a first angular resolution with respect to the surrounding environment, and a first group of optical signals. A first group of optical detectors configured to detect a first group of reflected optical signals from the surrounding environment during a first listening window, wherein the first group of reflected optical signals corresponds to the reflection of the first group of optical signals from an object within the surrounding environment, and a first group of optical detectors. A second group of optical emitters configured to emit a second group of optical signals into the surrounding environment, wherein the second group of optical emitters of the LIDAR device represents a subset of the first group of optical emitters of the LIDAR device, the second group of optical signals corresponds to a second angular resolution with respect to the surrounding environment, and the second angular resolution is lower than the first angular resolution, and a second group of optical emitters. A second group of photodetectors configured to detect a second group of reflected light signals from the surrounding environment during a second listening window, wherein the second group of photodetectors of the LIDAR device represents a subset of the first group of photodetectors of the LIDAR device, the second group of reflected light signals corresponding to reflections of the second group of optical signals from objects within the surrounding environment, and the duration of the second listening window being shorter than the duration of the first listening window, and a second group of photodetectors; A LIDAR controller configured to synthesize a dataset that can be used to generate one or more point clouds, wherein the dataset is based on the detected first group of reflected light signals and the detected second group of reflected light signals, and a LIDAR controller; A system controller, Receiving the dataset from the LIDAR controller, A system controller configured to generate the one or more point clouds based on the dataset. A system comprising:

Images