Time division multiple access scanning for crosstalk mitigation in light detection and ranging (lidar) devices

Abstract

The example embodiment relates to time-division multiple access scanning for crosstalk mitigation in a lidar (light detection and ranging) device. The example embodiment includes a method. The method includes emitting a first set of optical signals into the surrounding environment. The first set of optical signals corresponds to a first angular resolution. The method further includes detecting a first set of reflected optical signals during a first listening window. Additionally, the method includes emitting a second set of optical signals into the surrounding environment. The second set of optical signals corresponds to a second angular resolution relative to the surrounding environment. The second angular resolution is lower than the first angular resolution. Furthermore, the method includes detecting a second set of reflected optical signals from the surrounding environment. Additionally, the method includes synthesizing a dataset that can be used to generate one or more point clouds by a controller of the lidar device.

Description

Background Technology

[0001] Unless otherwise stated herein, the descriptions in this section are not prior art to the claims of this application and are not acknowledged as prior art by virtue of their inclusion in this section.

[0002] Autonomous vehicles, or vehicles operating in autonomous mode, can use various sensors to detect their surroundings. For example, lidar, radar, and / or cameras can be used to identify objects in the environment surrounding the autonomous vehicle. Such sensors can be used for object detection and avoidance and / or navigation, for example. Summary of the Invention

[0003] The embodiments described herein relate to mitigating crosstalk between detection channels in a lidar device. Specifically, the example embodiments include performing two transmission cycles with two corresponding detection cycles. During the first cycle, all optical emitters within the lidar device emit optical signals, and photodetectors listen for reflected optical signals for a duration corresponding to a relatively long range (e.g., between 300m and 450m). However, during the second cycle, only a subset of the optical emitters within the optical device (possibly sequentially) emit optical signals, and the corresponding photodetectors listen for reflected optical signals for a duration corresponding to a relatively short range (e.g., between 45m and 75m). By comparing the range represented by the detected optical signals during the two cycles, signals corresponding to crosstalk can be identified and / or removed from the resulting dataset.

[0004] In a first aspect, a method is provided. The method includes emitting a first set of optical signals from a first set of optical emitters of a lidar device into an surrounding environment. The first set of optical signals corresponds to a first angular resolution relative to the surrounding environment. The method further includes detecting a first set of reflected optical signals from the surrounding environment by a first set of optical detectors of the lidar device during a first listening window. The first set of reflected optical signals corresponds to reflections of the first set of optical signals from objects in the surrounding environment. Additionally, the method includes emitting a second set of optical signals from a second set of optical emitters of the lidar device into the surrounding environment. The second set of optical emitters of the lidar device represents a subset of the first set of optical emitters of the lidar device. The second set of optical signals corresponds to a second angular resolution relative to the surrounding environment. The second angular resolution is lower than the first angular resolution. Furthermore, the method includes detecting a second set of reflected optical signals from the surrounding environment by a second set of optical detectors of the lidar device during a second listening window. The second set of optical detectors of the lidar device represents a subset of the first set of optical detectors of the lidar device. The second set of reflected optical signals corresponds to reflections of the second set of optical signals from objects in the surrounding environment. The duration of the second listening window is shorter than the duration of the first listening window. Furthermore, the method includes synthesizing a dataset that can be used to generate one or more point clouds by the controller of the lidar device. The dataset is based on a first set of detected reflected light signals and a second set of detected reflected light signals.

[0005] In a second aspect, a lidar (light detection and ranging) device is provided. The lidar device includes a first set of light emitters configured to emit a first set of light signals into the surrounding environment. The first set of light signals corresponds to a first angular resolution relative to the surrounding environment. The lidar device also includes a first set of light detectors configured to detect a first set of reflected light signals from the surrounding environment during a first listening window. The first set of reflected light signals corresponds to reflections of the first set of light signals from objects in the surrounding environment. Additionally, the lidar device includes a second set of light emitters configured to emit a second set of light signals into the surrounding environment. The second set of light emitters in the lidar device represents a subset of the first set of light emitters in the lidar device. The second set of light signals corresponds to a second angular resolution relative to the surrounding environment. The second angular resolution is lower than the first angular resolution. Furthermore, the lidar device includes a second set of light detectors configured to detect a second set of reflected light signals from the surrounding environment during a second listening window. The second set of light detectors in the lidar device represents a subset of the first set of light detectors in the lidar device. The second set of reflected light signals corresponds to reflections of the second set of light signals from objects in the surrounding environment. The duration of the second listening window is shorter than the duration of the first listening window. In addition, the lidar device includes a controller configured to synthesize a dataset that can be used to generate one or more point clouds. The dataset is based on a first set of detected reflected light signals and a second set of detected reflected light signals.

[0006] In a third aspect, a system is provided. The system includes a lidar (light detection and ranging) device. The lidar device includes a first set of light emitters configured to emit a first set of light signals into the surrounding environment. The first set of light signals corresponds to a first angular resolution relative to the surrounding environment. The lidar device also includes a first set of light detectors configured to detect a first set of reflected light signals from the surrounding environment during a first listening window. The first set of reflected light signals corresponds to reflections of the first set of light signals from objects in the surrounding environment. Additionally, the lidar device includes a second set of light emitters configured to emit a second set of light signals into the surrounding environment. The second set of light emitters in the lidar device represents a subset of the first set of light emitters in the lidar device. The second set of light signals corresponds to a second angular resolution relative to the surrounding environment. The second angular resolution is lower than the first angular resolution. Furthermore, the lidar device includes a second set of light detectors configured to detect a second set of reflected light signals from the surrounding environment during a second listening window. The second set of light detectors in the lidar device represents a subset of the first set of light detectors in the lidar device. The second set of reflected light signals corresponds to reflections of the second set of light signals from objects in the surrounding environment. The duration of the second listening window is shorter than the duration of the first listening window. Furthermore, the lidar device includes a lidar controller configured to synthesize a dataset that can be used to generate one or more point clouds. The dataset is based on a first set of detected reflected light signals and a second set of detected reflected light signals. The system also includes a system controller. The system controller is configured to receive the dataset from the lidar controller. The system controller is also configured to generate one or more point clouds based on the dataset.

[0007] These and other aspects, advantages, and alternatives will become clear to those skilled in the art by referring to the accompanying drawings and reading the following detailed description where appropriate. Attached Figure Description

[0008] Figure 1 This is a functional block diagram illustrating a vehicle according to an example embodiment.

[0009] Figure 2A This is a diagram illustrating the physical configuration of a vehicle according to an example embodiment.

[0010] Figure 2B This is a diagram illustrating the physical configuration of a vehicle according to an example embodiment.

[0011] Figure 2C This is a diagram illustrating the physical configuration of a vehicle according to an example embodiment.

[0012] Figure 2D This is a diagram illustrating the physical configuration of a vehicle according to an example embodiment.

[0013] Figure 2EThis is a diagram illustrating the physical configuration of a vehicle according to an example embodiment.

[0014] Figure 2F This is a diagram illustrating the physical configuration of a vehicle according to an example embodiment.

[0015] Figure 2G This is a diagram illustrating the physical configuration of a vehicle according to an example embodiment.

[0016] Figure 2H This is a diagram illustrating the physical configuration of a vehicle according to an example embodiment.

[0017] Figure 2I This is a diagram illustrating the physical configuration of a vehicle according to an example embodiment.

[0018] Figure 2J This is a diagram illustrating the field of view of various sensors according to an example embodiment.

[0019] Figure 2K This is an illustration of beam steering for a sensor according to an example embodiment.

[0020] Figure 3 This is a conceptual illustration of wireless communication between various computing systems associated with autonomous or semi-autonomous vehicles, according to an example embodiment.

[0021] Figure 4A This is a block diagram of a system including a lidar device according to an example embodiment.

[0022] Figure 4B This is a block diagram of a lidar device according to an example embodiment.

[0023] Figure 5A This is an illustration of a lidar device, according to an example embodiment, that can be used to emit a set of optical signals and detect a set of reflected optical signals.

[0024] Figure 5B This is an illustration of potential crosstalk within a lidar device according to an example embodiment.

[0025] Figure 6A This is an illustration of the first set of emitted light signals and the first set of reflected light signals during the first period according to an example embodiment.

[0026] Figure 6B This is an illustration of the second set of emitted light signals and the second set of reflected light signals during the second period according to an example embodiment.

[0027] Figure 6C This is an illustration of the firing sequence used during the first cycle according to an example embodiment.

[0028] Figure 6DThis is an illustration of the excitation sequence used during the second cycle according to an example embodiment.

[0029] Figure 6E This is an illustration of a pair of light emitters and light detectors used during the second period according to an example embodiment.

[0030] Figure 6F This is an illustration of a pair of light emitters and light detectors used during the second period according to an example embodiment.

[0031] Figure 6G This is an illustration of the excitation sequence used during the second cycle according to an example embodiment.

[0032] Figure 6H This is an illustration of the excitation sequence used during the second cycle according to an example embodiment.

[0033] Figure 7A It is an illustration of a lighting pattern having a first angular resolution according to an example embodiment.

[0034] Figure 7B This is an illustration of a lighting pattern with a second angular resolution according to an example embodiment.

[0035] Figure 8A This is a schematic diagram of the excitation circuit and associated light emitter according to an example embodiment.

[0036] Figure 8B This is a schematic diagram of the excitation circuit and associated light emitter according to an example embodiment.

[0037] Figure 9 This is an illustration of the second set of emitted light signals and the second set of reflected light signals during the second period according to an example embodiment.

[0038] Figure 10 This is a flowchart illustrating a method according to an example embodiment. Detailed Implementation

[0039] This document contemplates exemplary methods and systems. Any exemplary embodiments or features described herein are not necessarily to be construed as preferred or advantageous over other embodiments or features. Furthermore, the exemplary embodiments described herein are not intended to be limiting. It will be readily understood that certain aspects of the disclosed systems and methods can be arranged and combined in a wide variety of different configurations, all of which are contemplated herein. Additionally, the specific arrangements shown in the figures should not be considered limiting. It should be understood that other embodiments may include more or fewer of each element shown in a given figure. Furthermore, some of the elements shown may be combined or omitted. Moreover, exemplary embodiments may include elements not shown in the figures.

[0040] The lidar device described herein can include one or more light emitters and one or more detectors for detecting light emitted by the one or more light emitters and reflected by one or more objects in the environment surrounding the lidar device. As an example, the surrounding environment can include an internal or external environment, such as the interior or exterior of a building. Additionally or alternatively, the surrounding environment can include the interior of a vehicle. Furthermore, the surrounding environment can include the area around and / or on the road. Examples of objects in the surrounding environment include, but are not limited to, other vehicles, traffic signs, pedestrians, cyclists, road surfaces, buildings, terrain, etc. Additionally, the one or more light emitters can emit light into the local environment of the lidar itself. For example, light emitted from the one or more light emitters can interact with the lidar's housing and / or surfaces or structures coupled to the lidar. In some cases, the lidar can be mounted to a vehicle, in which case the one or more light emitters can be configured to emit light that interacts with objects near the vehicle. Furthermore, the light emitters can include fiber optic amplifiers, laser diodes, light-emitting diodes (LEDs), etc.

[0041] Throughout this disclosure, the term "subset" is used to describe a group of channels, photodetectors, light emitters, etc., within various devices and systems (e.g., lidar devices). As used herein, the term "subset" means "proper subset" or "strict subset" in mathematical terms. Furthermore, the term "subset" as used herein excludes the empty set. In other words, for the purposes of this disclosure, if a set contains n elements, then a "subset" of said set can contain any integer number of elements from 1 to n-1 (inclusive).

[0042] A lidar device can determine the distances to reflective environmental features while scanning a scene. These distances can then be aggregated into a "point cloud" (or other type of representation) indicating surfaces in the surrounding environment. The individual points in the point cloud can be determined, for example, by emitting a laser pulse and detecting any reflected pulses (if any) from objects in the surrounding environment, and then determining the distance to the object based on the time delay between the emission of the pulse and the reception of the reflected pulse. For example, the resulting point cloud can correspond to a three-dimensional map of points indicating the locations of reflective features in the surrounding environment.

[0043] In example embodiments, a lidar device may include one or more light emitters (e.g., laser diodes) and one or more light detectors (e.g., silicon photomultipliers (SiPMs), single-photon avalanche diodes (SPADs) / or avalanche photodiodes (APDs)). For example, a lidar device may include a channel array comprising light detectors and corresponding light emitters. Such an array can illuminate objects in a scene and receive reflected light from objects in the scene to collect data that can be used to generate a point cloud with a specific angular field of view relative to the lidar device. Furthermore, to generate a point cloud with an enhanced field of view (e.g., a complete 360° field of view), the light emitter array and the corresponding light detector array can transmit and receive light at predetermined times and / or locations within the enhanced field of view. For example, a lidar device may be able to include a channel array arranged around a vertical axis, such that light is simultaneously transmitted and received in multiple directions around the 360° field of view. As another example, a lidar device may scan around a central axis (e.g., by rotation or beam scanning using other mechanisms) to transmit / receive multiple datasets. The data can be used to form point clouds, which can be composited to generate an enhanced field of view.

[0044] Some lidar devices may be susceptible to noise generated by high-intensity returned signals. For example, if a light emitter emits a light pulse toward a highly reflective object (e.g., a retroreflector), the returned pulse from that object may be of high intensity. In some cases, if the intensity of the returned pulse is sufficiently large, it can cause crosstalk between channels of the lidar device. In other words, in addition to being detected by the photodetector corresponding to the light emitter that emitted the pulse, a high-intensity returned pulse can be detected by other photodetectors within the lidar device (e.g., photodetectors adjacent to the photodetector corresponding to the light emitter that emitted the pulse). This crosstalk may be a result of and / or amplified by one or more defects in the optical path between the photodetector and the object being detected. For example, the optical window of a lidar device may have rain, condensation, snow, dirt, mud, dust, ice, debris, etc. Such defects can reflect, refract, and / or scatter one or more reflected light signals from one or more objects in the surrounding environment, thus causing crosstalk.

[0045] Crosstalk can cause detection errors regardless of the cause. For example, when a detector detects a return pulse as a result of crosstalk, the computing device associated with the lidar device may incorrectly determine the presence of an object at a location in the surrounding environment, even if such an object does not actually exist (i.e., the lidar device generates a false positive detection). Additionally or alternatively, as a result of detecting a high-intensity crosstalk return pulse, a correct return pulse (e.g., at a lower intensity) may be incorrectly ignored. Therefore, the example embodiments disclosed herein can be used to mitigate and / or eliminate incorrect detections caused by noise sources. While crosstalk caused by highly reflective objects has been referenced throughout this disclosure, it will be understood that other noise sources are also possible and can also be mitigated using the techniques described herein. For example, one or more of the techniques described herein can also be used to mitigate interference on various photodetectors (e.g., from stray light sources, such as from different lidar devices, or from malicious light sources, such as someone flashing a laser pointer at a lidar device). Additionally or alternatively, the techniques described herein can also be used to mitigate electrical crosstalk. Electrical crosstalk can include, for example, electrical signals coupled to adjacent or nearby photodetectors when one photodetector experiences a large detection signal.

[0046] In some embodiments, a lidar device is provided. As described above, the lidar device may include a channel array. Each channel in the array may include a photodetector and a corresponding light emitter. For example, the light emitter in a given channel may be configured to emit a light pulse along a certain emission vector, and the corresponding photodetector may be configured to detect the light pulse reflected from an object in the surrounding environment along the path of the emission vector. Each photodetector in a different channel of the channel array may be positioned close to each other in the lidar device within a photodetector array. In this way, any high-intensity return pulse may affect the photodetector near the main photodetector that detects the high-intensity return pulse.

[0047] One way to mitigate this crosstalk is to identify which channel in the channel array has an emission vector that intersects with a high-reflectivity object in the surrounding environment that causes a high-intensity return. Then, upon identifying the responsible channel as the source of crosstalk, the light emitter in that channel can simply suppress the emission of light pulses in future emission cycles. However, it can be difficult to determine when (e.g., in which emission cycle) light pulses will resume being emitted from the light emitter corresponding to the high-intensity reflection. Similarly, another possible mitigation technique would be to simply ignore any detected pulses detected in future detection cycles by photodetectors in the array that are near the primary photodetector that detected the high-intensity reflection. However, this could result in several channels being essentially unused during one or more detection cycles. Clearly, the mitigation strategies described above could lead to the ignoring of multiple detected pulses, which may be unnecessary.

[0048] Therefore, this paper describes alternative noise reduction techniques that can be used in conjunction with or instead of the previously described mitigation techniques. Specifically, the techniques described herein can involve transmitting / detecting optical signals across two excitation cycles. The first cycle can involve exciting all channels of the lidar device and detecting all returns. This first cycle can seek to detect all possible returns, whether over a relatively long or short range. However, the second cycle can involve a series of interleaved transmissions / detections. This series of transmissions / detections can be performed by a subset of channels within the lidar device (e.g., a subset physically far enough apart that it is unlikely to be susceptible to crosstalk from each other). Furthermore, the transmissions / detections in the second cycle can correspond to transmissions / detections over a shorter range than those in the first cycle. Thus, the techniques described herein can take advantage of the fact that crosstalk may be a more significant problem at shorter distances (e.g., detection events in the second cycle can be used to detect objects at shorter distances, while detection events in the first cycle can be used to detect objects at longer distances). Finally, detection events from the second cycle can be combined with detection events from the first cycle to form a single dataset. The techniques described in this paper can represent a method for performing time-division multiple access on various channels of a lidar device (i.e., by separating detection events in time, it is possible to identify and ignore crosstalk).

[0049] A complete detection cycle can be performed as follows. First (i.e., during the first cycle), the light emitter in each channel of the lidar device can emit a light signal. Subsequently, the corresponding photodetector in the lidar device can detect reflections from objects in the surrounding environment during a first detection window. The duration (i.e., time length) of the first detection window can be relatively long (e.g., between 2.0 μs and 3.0 μs) to allow detection of objects over a relatively long range (e.g., up to a range between 300 m and 450 m). Detection events from the photodetectors during the first detection window can then be temporarily stored (e.g., in a memory such as volatile memory). For example, these detection events can be stored as complete waveforms (e.g., intensity waveforms from the corresponding photodetectors) and / or metadata (e.g., data corresponding to the detection time, detected intensity, and / or detected polarization).

[0050] Subsequently (i.e., during the second cycle), a subset of light emitters can be excited for shorter time intervals. For example, if the lidar device has 16 channels (e.g., labeled "Channel 0", "Channel 1", "Channel 2"... "Channel 15"), light emitters in a preselected subset of channels can be excited sequentially. For example, the light emitter of Channel 0 can be excited by itself for a portion of the second cycle (i.e., without exciting other light emitters). During this portion of the second cycle, the photodetector of Channel 0 can detect reflections from objects in the surrounding environment during a second detection window. The duration of the second detection window can be shorter than the duration of the first detection window. For example, the duration of the second detection window can be between 0.3 μs and 0.5 μs to detect objects within a relatively short range (e.g., a range up to 45 m and 75 m). The detection events(s) from this portion of the second cycle can then be temporarily stored (e.g., stored in a memory such as volatile memory). Similar to the detection events during the first cycle, these detection events can be stored as complete waveforms (e.g., intensity waveforms from the corresponding photodetector) and / or metadata (e.g., data corresponding to the detection time, detected intensity, and / or detected polarization).

[0051] Next, the aforementioned portion of the second cycle performed for channel 0 can be repeated individually for channels 2, 4, 6, 8, 10, 12, and 14 during the second cycle. It is evident from the fact that not all channels are used (e.g., channels 1, 3, 5, 7, 9, 11, 13, and 15 were not used in the previous example) that the angular resolution of the channels selected during the second cycle can be less than the angular resolution of all combined channels (e.g., the channels used during the first cycle). However, because the range probed during the second cycle can be shorter than during the first cycle, a lower angular resolution can be acceptable (e.g., if the surrounding environment is linearly over-resolved over a shorter range, making it sufficiently linearly resolvable over a longer range). In other words, even with reduced angular resolution, the data captured during the second cycle can still provide sufficient linear resolution (e.g., in points per inch) when considered within the shorter range involved during the second cycle (e.g., less than 75 m). The amount of reduction in angular resolution can be at least partially based on the total duration allocated to the second cycle. For example, if 5 μs is allocated to the second period, and the duration of each second detection window is 0.5 μs, then there can be 10 available excitation slots / parts during the second period. Therefore, if the channels are excited individually during the second period, the reduction in angular resolution can correspond to the total number of channels divided by the number of available excitation slots (e.g., 16 total channels / 10 excitation slots, or a 1.6-fold reduction in angular resolution).

[0052] It should be understood that the arrangement of channels excited during the second period described above is provided as an example, and other arrangements are possible and contemplated herein. Furthermore, while only a single channel of the light emitter can be excited during each portion of the second period as described above, other numbers of channels (e.g., pairs of channels, groups of three channels, groups of four channels, and / or groups of five channels) can be used during portions of the second period. For example, pairs of channels can be selected for simultaneous emission / detection in consecutive portions of the second period. In such embodiments, the pairs of channels selected during each portion of the second period can be chosen such that the channels in use (e.g., detectors in the channels in use) are physically far enough apart to prevent crosstalk between channels in each portion of the second period. In some embodiments, consecutive pairs of channels used across multiple portions of the second period can also represent staggered crossover across the lidar device / surrounding environment. Further, in some embodiments, the number of channels excited during a portion of the second period (e.g., pairs of channels) can differ from the number of channels excited during another portion of the second period (e.g., a group of three channels). Furthermore, although the first cycle described above is a longer-range, increased angular resolution cycle, and the second cycle described above is a shorter-range, decreased angular resolution cycle, it will be understood that the order of these cycles can be reversed (i.e., the first cycle is performed after the second cycle in time).

[0053] Additionally or alternatively, in some embodiments, previous detection data can be incorporated into the excitation scheme. For example, in some embodiments, highly reflective surfaces in the surrounding environment (e.g., retroreflectors) may have already been identified during a previous excitation cycle (e.g., based on high-intensity reflections detected by one or more photodetectors of the lidar device). Additionally, channels (e.g., light emitters of the channels) pointing towards the identified highly reflective surfaces can also be identified. Then, in subsequent excitation cycles (e.g., during both the first and second cycles described above), the light emitters of the channels pointing towards the retroreflectors can be completely avoided from excitation. This provides further robustness against accidental crosstalk.

[0054] Once all detection events from the first and second periods have been collected, these events can be synthesized to form a dataset that can be used to generate one or more point clouds. For example, data from the first period can be provided (e.g., by the controller of the lidar device to the computing device) as a dataset for generating a first point cloud, and data from the second period can be provided (e.g., by the controller of the lidar device to the computing device) as a dataset for generating a second point cloud. Alternatively, in some embodiments, detection events from the two periods can be combined in such a way that the resulting dataset can be used to generate a single point cloud. In such embodiments, detection events corresponding to a given channel during the first and second periods can be compared. For example, the distance to an object in the surrounding environment determined for a given channel (e.g., channel 1) during the first period can be compared with the distance to an object in the surrounding environment determined for the same channel (e.g., channel 1) during the second period. If the two distances are the same (or within a certain threshold difference), the measurement can be determined to be correct and does not indicate crosstalk. Therefore, one or both of the measured distances can be included in the dataset that can be used to generate a single point cloud. Furthermore, if a measurement during the second period does not produce a measured distance, but a measurement during the first period does, and the measurement during the first period is within a range exceeding that of the measurement during the second period, then the distance measured during the first period can also be included in the dataset that can be used to generate a single point cloud (e.g., because the measurement can be determined not to correspond to crosstalk). However, if the detection events detected during the second period and the first period are inconsistent (e.g., not within the threshold difference) and both correspond to the target range within the range of the measurement during the second period, then the determined distance may not be included in the dataset (e.g., because the measurement during the first period may be the result of crosstalk), or only the distance measured during the second period can be included in the dataset.

[0055] The following description and accompanying drawings will illustrate the features of various exemplary embodiments. The embodiments provided are illustrative and not intended to be limiting. Therefore, the dimensions of the drawings are not necessarily drawn to scale.

[0056] The example systems within the scope of this disclosure will now be described in more detail. The example systems can be implemented in or can take the form of automobiles. Additionally, the example systems can also be implemented in or take the form of various vehicles, such as automobiles, trucks (e.g., pickup trucks, vans, tractors and / or tractor-trailers), motorcycles, buses, airplanes, helicopters, drones, lawnmowers, bulldozers, boats, submarines, all-terrain vehicles, snowmobiles, aircraft, recreational vehicles, amusement park vehicles, farm equipment or vehicles, construction equipment or vehicles, warehouse equipment or vehicles, factory equipment or vehicles, trams, golf carts, trains, handcarts, sidewalk delivery vehicles, robotic equipment, etc. Other vehicles are also possible. Furthermore, in some embodiments, the example systems may not include a vehicle.

[0057] Now refer to the attached diagram, Figure 1 This is a functional block diagram illustrating an example vehicle 100, which can be configured to operate fully or partially in an autonomous mode. More specifically, vehicle 100 can operate in an autonomous mode without human interaction by receiving control commands from a computing system. As part of operating in autonomous mode, vehicle 100 can use sensors to detect and, possibly, identify objects in the surrounding environment for safe navigation. Additionally, example vehicle 100 can operate in a partially autonomous (i.e., semi-autonomous) mode, where some functions of vehicle 100 are controlled by a human driver and some functions are controlled by the computing system. For example, vehicle 100 may also include subsystems that enable the driver to control the operation of vehicle 100, such as steering, acceleration, and braking, while the computing system performs assistance functions, such as lane departure warning / lane keeping assist or adaptive cruise control, based on other objects in the surrounding environment (e.g., other vehicles).

[0058] As described in this paper, in partially autonomous driving modes, even when the vehicle assists with one or more driving operations (e.g., steering, braking, and / or acceleration to perform lane centering, adaptive cruise control, advanced driver assistance systems (ADAS), and / or emergency braking), the human driver is expected to be situationally aware of the vehicle's surroundings and supervise the assisted driving operations. Here, even if the vehicle can perform all driving tasks in certain situations, the human driver is expected to be responsible for taking control as needed.

[0059] Although various systems and methods are described below in conjunction with autonomous vehicles for the sake of brevity and simplicity, these or similar systems and methods can be used in various driver assistance systems (i.e., partially autonomous driving systems) that do not reach the level of fully autonomous driving. In the United States, the Society of Automotive Engineers (SAE) has defined different levels of automated driving operation to indicate the degree to which a vehicle controls driving; however, different organizations in the United States or other countries may classify levels differently. More specifically, the disclosed systems and methods can be used to implement SAE Level 2 driver assistance systems for steering, braking, acceleration, lane centering, adaptive cruise control, and other driver support. The disclosed systems and methods can be used in SAE Level 3 driver assistance systems capable of autonomous driving under limited (e.g., highway) conditions. Similarly, the disclosed systems and methods can be used in vehicles using SAE Level 4 automated driving systems, which operate autonomously in most normal driving situations and require only occasional human operator attention. In all such systems, accurate lane estimation can be performed automatically without driver input or control (e.g., while the vehicle is moving), leading to improvements in the reliability of vehicle positioning and navigation, as well as the overall safety of autonomous, semi-autonomous, and other driver assistance systems. As previously mentioned, other organizations in the U.S. or other countries may classify levels of automated driving operations differently than the way SAE classifies them. Without limitation, the systems and methods disclosed herein can be used in driver assistance systems defined by the automated driving operation levels of these other organizations.

[0060] like Figure 1 As shown, vehicle 100 can include various subsystems, such as a propulsion system 102, a sensor system 104, a control system 106, one or more peripheral devices 108, a power supply 110, a computer system 112 (which can also be referred to as a computing system) with a data storage device 114, and a user interface 116. In other examples, vehicle 100 can include more or fewer subsystems, each of which can include multiple elements. The subsystems and components of vehicle 100 can be interconnected in various ways. Furthermore, the functionality of vehicle 100 described herein can be divided into additional functional or physical components, or combined into fewer functional or physical components within an embodiment. For example, control system 106 and computer system 112 can be combined into a single system to operate vehicle 100 according to various operations.

[0061] The propulsion system 102 may include one or more components operable to provide powered motion to the vehicle 100, and may include an engine / motor 118, an energy source 119, a transmission 120, and wheels / tires 121, among other possible components. For example, the engine / motor 118 may be configured to convert the energy source 119 into mechanical energy, and may correspond to one or a combination of an internal combustion engine, an electric motor, a steam engine, or a Stirling engine, among other possible options. For example, in some embodiments, the propulsion system 102 may include multiple types of engines and / or motors, such as gasoline engines and electric motors.

[0062] Energy source 119 refers to an energy source that can supply power, in whole or in part, to one or more systems of vehicle 100 (e.g., engine / motor 118). For example, energy source 119 can correspond to gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and / or other power sources. In some embodiments, energy source 119 may include a combination of a fuel tank, battery, capacitor, and / or flywheel.

[0063] The transmission 120 is capable of transmitting mechanical power from the engine / motor 118 to the wheels / tires 121 and / or other possible systems of the vehicle 100. Thus, the transmission 120 may include a gearbox, clutch, differential, and drive shaft, as well as other possible components. The drive shaft may include an axle connected to one or more wheels / tires 121.

[0064] In the example embodiment, the wheels / tires 121 of the vehicle 100 can have various configurations. For example, the vehicle 100 can exist in the form of a unicycle, bicycle / motorcycle, tricycle, or four-wheeled car / truck, as well as other possible configurations. Thus, the wheels / tires 121 can be connected to the vehicle 100 in various ways and can be in different materials, such as metal and rubber.

[0065] Sensor system 104 can include various types of sensors, such as a Global Positioning System (GPS) 122, an Inertial Measurement Unit (IMU) 124, radar 126, lidar 128, camera 130, steering sensor 123, and throttle / brake sensor 125, as well as other possible sensors. In some embodiments, sensor system 104 may also include sensors configured to monitor the internal systems of vehicle 100 (e.g., O2 monitor, fuel gauge, engine oil temperature, and / or brake wear).

[0066] GPS 122 may include a transceiver operable to provide information about the position of vehicle 100 relative to the Earth. IMU 124 may be configured to use one or more accelerometers and / or gyroscopes and can sense changes in the position and orientation of vehicle 100 based on inertial acceleration. For example, IMU 124 can detect the pitch and yaw of vehicle 100 while vehicle 100 is stationary or in motion.

[0067] Radar 126 may represent one or more systems configured to use radio signals to sense objects (including the speed and direction of travel) within the surrounding environment of vehicle 100. Thus, radar 126 may include an antenna configured to transmit and receive radio signals. In some embodiments, radar 126 may correspond to an installable radar configured to obtain measurements of the surrounding environment of vehicle 100.

[0068] The lidar 128 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components, and may operate in a coherent mode (e.g., using heterodyne detection) or an incoherent detection mode (i.e., time-of-flight mode). In some embodiments, one or more detectors of the lidar 128 may include one or more photodetectors, which may be particularly sensitive detectors (e.g., avalanche photodiodes). In some examples, such photodetectors may be able to detect single photons (e.g., SPADs). Furthermore, such photodetectors can be arranged (e.g., via series electrical connections) in an array (e.g., as in a SiPM). In some examples, one or more photodetectors are Geiger-mode operating devices, and the lidar includes sub-components designed for such Geiger-mode operation.

[0069] Camera 130 may include one or more devices (e.g., still camera, video camera, thermal imaging camera, stereo camera and / or night vision camera) configured to capture images of the environment surrounding vehicle 100.

[0070] The steering sensor 123 is capable of sensing the steering angle of the vehicle 100, which may involve measuring the angle of the steering wheel or measuring an electrical signal representing the angle of the steering wheel. In some embodiments, the steering sensor 123 may measure the angle of the wheels of the vehicle 100, such as detecting the angle of the wheels relative to the forward axis of the vehicle 100. The steering sensor 123 may also be configured to measure a combination (or subset) of the steering wheel angle, an electrical signal representing the steering wheel angle, and the angles of the wheels of the vehicle 100.

[0071] Throttle / brake sensor 125 can detect the position of the throttle or brake of vehicle 100. For example, throttle / brake sensor 125 can measure the angle of both the throttle pedal (throttle) and the brake pedal, or it can measure an electrical signal that represents, for example, the angle of the accelerator pedal (throttle) and / or the angle of the brake pedal. Throttle / brake sensor 125 can also measure the angle of the throttle body of vehicle 100, which may include part of a regulating physical mechanism (e.g., a butterfly valve, carburetor) that provides energy source 119 to engine / motor 118. In addition, throttle / brake sensor 125 can measure the pressure of one or more brake pads on the rotor of vehicle 100, or a combination (or subset) of the angle of the accelerator pedal (throttle) and brake pedal, an electrical signal representing the angle of the accelerator pedal (throttle) and brake pedal, the angle of the throttle body, and the pressure exerted by at least one brake pad on the rotor of vehicle 100. In other embodiments, the throttle / brake sensor 125 may be configured to measure the pressure applied to a vehicle pedal (such as a throttle or brake pedal).

[0072] The control system 106 may include components configured to assist navigation of the vehicle 100, such as a steering unit 132, a throttle valve 134, a braking unit 136, a sensor fusion algorithm 138, a computer vision system 140, a navigation / path planning system 142, and an obstacle avoidance system 144. More specifically, the steering unit 132 may be operable to adjust the forward direction of the vehicle 100, and the throttle valve 134 may control the operating speed of the engine / motor 118 to control the acceleration of the vehicle 100. The braking unit 136 may decelerate the vehicle 100, which may involve using friction to slow down the wheels / tires 121. In some embodiments, the braking unit 136 may convert the kinetic energy of the wheels / tires 121 into electrical current for subsequent use by one or more systems of the vehicle 100.

[0073] Sensor fusion algorithm 138 may include Kalman filters, Bayesian networks, or other algorithms capable of processing data from sensor system 104. In some embodiments, sensor fusion algorithm 138 may provide evaluations based on incoming sensor data, such as evaluations of individual objects and / or features, evaluations of specific situations, and / or evaluations of potential impacts within a given situation.

[0074] Computer vision system 140 may include hardware and software (e.g., a general-purpose processor such as a central processing unit (CPU), a dedicated processor such as a graphics processing unit (GPU) or tensor processing unit (TPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), volatile memory, non-volatile memory, and / or one or more machine learning models) operable to process and analyze images in an effort to determine moving objects (e.g., other vehicles, pedestrians, cyclists, and / or animals) and stationary objects (e.g., traffic lights, road boundaries, speed bumps, and / or potholes). Thus, computer vision system 140 may use object recognition, structure-of-motion (SFM), video tracking, and other algorithms used in computer vision, such as to identify objects, map the environment, track objects, estimate object velocities, etc.

[0075] The navigation / path planning system 142 can determine the driving path of the vehicle 100, which may involve dynamically adjusting the navigation during operation. Thus, the navigation / path planning system 142 can navigate the vehicle 100 using data from sensor fusion algorithm 138, GPS 122, maps, and other sources. The obstacle avoidance system 144 can assess potential obstacles based on sensor data and enable the vehicle 100's systems to avoid or otherwise traverse potential obstacles.

[0076] like Figure 1 As shown, vehicle 100 may also include peripheral devices 108, such as a wireless communication system 146, a touchscreen 148, an internal microphone 150, and / or a speaker 152. Peripheral devices 108 can provide users with controls or other elements to interact with user interface 116. For example, touchscreen 148 can provide information to the user of vehicle 100. User interface 116 can also accept input from the user via touchscreen 148. Peripheral devices 108 can also enable vehicle 100 to communicate with devices such as other vehicle equipment.

[0077] The wireless communication system 146 can wirelessly communicate with one or more devices, either directly or via a communication network. For example, the wireless communication system 146 can use 3G cellular communication (such as Code Division Multiple Access (CDMA), Evolved Data Optimized (EVDO), Global System for Mobile Communications (GSM) / General Packet Radio Service (GPRS)) or cellular communication such as 4G Global Microwave Access Interoperability (WiMAX) or Long Term Evolution (LTE) or 5G. Alternatively, the wireless communication system 146 can use... Or other possible connections to communicate with a wireless local area network (WLAN). The wireless communication system 146 can also communicate directly with devices using, for example, an infrared link, Bluetooth, or ZigBee. Within the context of this disclosure, other wireless protocols, such as various vehicle communication systems, are possible. For example, the wireless communication system 146 may include one or more dedicated short-range communication (DSRC) devices capable of including public and / or private data communication between vehicles and / or roadside stations.

[0078] The carrier 100 may include a power supply 110 for powering components. In some embodiments, the power supply 110 may include a rechargeable lithium-ion or lead-acid battery. For example, the power supply 110 may include one or more batteries configured to provide power. The carrier 100 may also use other types of power supplies. In an example embodiment, the power supply 110 and the energy source 119 may be integrated into a single energy source.

[0079] The vehicle 100 may also include a computer system 112 to perform operations, such as those described herein. Thus, the computer system 112 may include at least one processor 113 (which may include at least one microprocessor) operable to execute instructions 115 stored in a non-transitory computer-readable medium (such as a data storage device 114). In some embodiments, the computer system 112 may represent multiple computing devices that can be used to control various components or subsystems of the vehicle 100 in a distributed manner.

[0080] In some embodiments, the data storage device 114 may include various functions of the vehicle 100 that can be executed by the processor 113 to perform (including those combined above). Figure 1 Instructions 115 (e.g., program logic) for the functions described. Data storage device 114 may also contain additional instructions, including instructions for sending data to, receiving data from, and interacting with and / or controlling one or more of the propulsion system 102, sensor system 104, control system 106, and peripheral devices 108.

[0081] In addition to command 115, data storage device 114 can store other information such as road maps and route information. This information can be used by vehicle 100 and computer system 112 while vehicle 100 is operating in autonomous, semi-autonomous, and / or manual modes.

[0082] Vehicle 100 may include a user interface 116 for providing information to or receiving input from a user of vehicle 100. User interface 116 may control or implement control over the content and / or layout of interactive images that can be displayed on touchscreen 148. Furthermore, user interface 116 may include one or more input / output devices within a set of peripheral devices 108, such as wireless communication system 146, touchscreen 148, microphone 150, and speaker 152.

[0083] Computer system 112 can control the functions of vehicle 100 based on inputs received from various subsystems (e.g., propulsion system 102, sensor system 104, and / or control system 106) and from user interface 116. For example, computer system 112 can utilize inputs from sensor system 104 to estimate the outputs generated by propulsion system 102 and control system 106. Depending on the embodiment, computer system 112 can be operable to monitor many aspects of vehicle 100 and its subsystems. In some embodiments, computer system 112 can disable some or all functions of vehicle 100 based on signals received from sensor system 104.

[0084] Components of vehicle 100 can be configured to operate in a manner interconnected with other components within or outside their respective systems. For example, in an example embodiment, camera 130 can capture multiple images representing information about the state of the environment surrounding vehicle 100, operating in autonomous or semi-autonomous mode. The state of the environment can include parameters of the road on which the vehicle is operating. For example, computer vision system 140 can be able to identify slopes (degrees) or other features based on multiple images of the road. Additionally, a combination of GPS 122 and features identified by computer vision system 140 can be used with map data stored in data storage device 114 to determine specific road parameters. Furthermore, radar 126 and / or lidar 128 and / or some other environmental mapping, ranging, and / or positioning sensor systems can also provide information about the vehicle's surroundings.

[0085] In other words, the combination of various sensors (which can be referred to as input indication and output indication sensors) and computer system 112 can interact to provide indications of the inputs provided to control the vehicle or indications of the vehicle's surroundings.

[0086] In some embodiments, computer system 112 can make determinations about various objects based on data provided by systems other than radio systems. For example, vehicle 100 may have lasers or other optical sensors configured to sense objects in the vehicle's field of view. Computer system 112 can use the outputs from various sensors to determine information about objects in the vehicle's field of view, and can determine distance and orientation information to various objects. Computer system 112 can also determine whether an object is desired or undesirable based on the outputs from various sensors.

[0087] although Figure 1 Various components of the vehicle 100 (i.e., the wireless communication system 146, the computer system 112, the data storage device 114, and the user interface 116) are shown as integrated into the vehicle 100; however, one or more of these components can be installed or associated separately from the vehicle 100. For example, the data storage device 114 can exist partially or wholly separate from the vehicle 100. Therefore, the vehicle 100 can be provided in the form of device elements that can be positioned individually or together. The device elements constituting the vehicle 100 can be communicatively coupled together in a wired and / or wireless manner.

[0088] Figures 2A-2E An example vehicle 200 (e.g., a fully autonomous vehicle, a semi-autonomous vehicle) is shown, which can include a reference vehicle. Figure 1 Vehicle 100 may contain some or all of the functions described. Although for illustrative purposes, vehicle 200 may contain some or all of the functions described. Figures 2A-2E The vehicle is shown as a truck with side mirrors, but this disclosure is not limited thereto. For example, vehicle 200 can represent a truck, car, semi-trailer, motorcycle, golf cart, off-road vehicle, agricultural vehicle, or any other vehicle described elsewhere herein (e.g., bus, boat, aircraft, helicopter, drone, lawnmower, bulldozer, submarine, all-terrain vehicle, snowmobile, aircraft, recreational vehicle, amusement park vehicle, farm equipment, construction equipment or vehicle, warehouse equipment or vehicle, factory equipment or vehicle, tram, train, handcart, sidewalk delivery vehicle, and / or robotic equipment).

[0089] Example vehicle 200 may include one or more sensor systems 202, 204, 206, 208, 210, 212, 214, and 218. In some embodiments, sensor systems 202, 204, 206, 208, 210, 212, 214, and / or 218 may represent one or more optical systems (e.g., cameras), one or more lidars, one or more radars, one or more inertial sensors, one or more humidity sensors, one or more acoustic sensors (e.g., microphones, sonar devices), or one or more other sensors configured to sense information about the environment surrounding vehicle 200. In other words, any sensor system now known or created hereafter may be coupled to vehicle 200 and / or may be used in conjunction with various operations of vehicle 200. As an example, lidar may be used for autonomous driving or other types of navigation, planning, perception, and / or mapping operations of vehicle 200. Furthermore, sensor systems 202, 204, 206, 208, 210, 212, 214 and / or 218 can represent combinations of sensors described herein (e.g., one or more lidars and radars; one or more lidars and cameras; one or more cameras and radars; one or more lidars, cameras and radars).

[0090] It should be noted that Figures 2A-2E The number, location, and type of sensor systems (e.g., 202, 204) depicted are intended as non-limiting examples of the location, number, and type of such sensor systems for autonomous or semi-autonomous vehicles. Alternative numbers, locations, types, and configurations of such sensors are possible (e.g., to suit specific environmental or application scenarios by conforming to vehicle size, shape, aerodynamics, fuel economy, aesthetics, or other conditions, and / or reducing costs). For example, sensor systems (e.g., 202, 204) can be positioned at various other locations on the vehicle (e.g., at location 216) and can have a field of view corresponding to the interior and / or surrounding environment of the vehicle 200.

[0091] Sensor system 202 can be mounted on top of vehicle 200 and can include one or more sensors configured to detect information about the environment around vehicle 200 and output indications of that information. For example, sensor system 202 can include any combination of cameras, radar, lidar, inertial sensors, humidity sensors, and acoustic sensors (e.g., microphones and / or sonar devices). Sensor system 202 can include one or more movable mounts operable to adjust the orientation of one or more sensors in sensor system 202. In one embodiment, the movable mount can include a rotating platform capable of scanning the sensors to obtain information from every direction around vehicle 200. In another embodiment, the movable mount of sensor system 202 can move in a scanning manner within a specific range of angles and / or azimuths and / or heights. Sensor system 202 can be mounted on the roof of a vehicle, but other mounting locations are also possible.

[0092] Furthermore, the sensors of sensor system 202 can be distributed at different locations and do not need to be juxtaposed at a single location. Additionally, each sensor of sensor system 202 can be configured to move or scan independently of the other sensors in sensor system 202. Alternatively or additionally, multiple sensors can be installed at one or more of sensor locations 202, 204, 206, 208, 210, 212, 214, and / or 218. For example, there may be two lidar devices installed at a sensor location and / or one lidar device and one radar device installed at a sensor location.

[0093] One or more sensor systems 202, 204, 206, 208, 210, 212, 214, and / or 218 can include one or more lidar sensors. For example, a lidar sensor can include multiple light emitter devices arranged within an angular range relative to a given plane (e.g., the xy plane). For example, one or more of sensor systems 202, 204, 206, 208, 210, 212, 214, and / or 218 can be configured to rotate or pivot about an axis perpendicular to the given plane (e.g., the z-axis) to illuminate the environment surrounding the vehicle 200 with light pulses. Information about the surrounding environment can be determined based on various aspects of the detected reflected light pulses (e.g., elapsed time of flight, polarization, and / or intensity).

[0094] In the example embodiment, sensor systems 202, 204, 206, 208, 210, 212, 214, and / or 218 can be configured to provide corresponding point cloud information that can be correlated with physical objects within the surrounding environment of vehicle 200. While vehicle 200 and sensor systems 202, 204, 206, 208, 210, 212, 214, and 218 are shown to include certain features, it will be understood that other types of sensor systems are contemplated within the scope of this disclosure. Furthermore, the example vehicle 200 is capable of including combinations of... Figure 1 Any component described in vehicle 100.

[0095] In the example configuration, one or more radars can be located on vehicle 200. Similar to radar 126 described above, one or more radars may include antennas configured to transmit and receive radio waves (e.g., electromagnetic waves with frequencies between 30 Hz and 300 GHz). Such radio waves can be used to determine the distance to one or more objects in the environment surrounding vehicle 200 and / or the speed of one or more objects in the environment surrounding vehicle 200. For example, one or more sensor systems 202, 204, 206, 208, 210, 212, 214 and / or 218 can include one or more radars. In some examples, one or more radars can be located near the rear of vehicle 200 (e.g., sensor systems 208, 210) to actively scan the environment near the rear of vehicle 200 to look for the presence of radio-reflecting objects. Similarly, one or more radars can be located near the front of vehicle 200 (e.g., sensor systems 212, 214) to actively scan the environment near the front of vehicle 200. The radar can be positioned, for example, in a location suitable for illuminating an area including the forward movement path of the vehicle 200 without being obstructed by other features of the vehicle 200. For example, the radar can be embedded in or near the front bumper, headlights, fairing, and / or hood. Furthermore, one or more additional radars can be positioned to actively scan the sides and / or rear of the vehicle 200 to detect the presence of radio-reflecting objects, such as by including such devices in or near the rear bumper, side panels, sill plates, and / or chassis.

[0096] Vehicle 200 may include one or more cameras. For example, one or more sensor systems 202, 204, 206, 208, 210, 212, 214 and / or 218 may include one or more cameras. The cameras may be photosensitive instruments, such as still cameras, video cameras, thermal imaging cameras, stereo cameras, night vision cameras, etc., configured to capture multiple images of the surrounding environment of vehicle 200. For this purpose, the cameras may be configured to detect visible light and may additionally or alternatively be configured to detect light from other parts of the spectrum, such as infrared or ultraviolet light. The cameras may be two-dimensional detectors and may optionally have sensitivity in a three-dimensional spatial range. In some embodiments, the camera may include, for example, a distance detector configured to generate a two-dimensional image indicating the distance from the camera to several points in the surrounding environment. For this purpose, the camera may use one or more distance detection techniques. For example, the camera may provide range information by using structured light technology, wherein vehicle 200 illuminates objects in the surrounding environment with a predetermined light pattern (such as a grid or checkerboard pattern), and the camera is used to detect reflections of the predetermined light pattern from the surrounding environment. Based on the distortion of the reflected light pattern, the vehicle 200 can determine the distance to a point on an object. The predetermined light pattern may include infrared light or radiation at other suitable wavelengths for such measurement. In some examples, the camera can be mounted inside the windshield of the vehicle 200. Specifically, the camera can be positioned to capture images from a forward-facing view relative to the vehicle 200. Other mounting positions and viewing angles of the camera are also possible, both inside and outside the vehicle 200. Furthermore, the camera can have associated optics operable to provide an adjustable field of view. Further still, the camera can be mounted to the vehicle 200 using a movable mount to vary the camera's pointing angle, for example, via a translation / tilt mechanism.

[0097] Vehicle 200 may also include one or more acoustic sensors for sensing the surrounding environment of vehicle 200 (e.g., one or more of sensor systems 202, 204, 206, 208, 210, 212, 214, 216, 218 may include one or more acoustic sensors). The acoustic sensors may include microphones (e.g., piezoelectric microphones, condenser microphones, ribbon microphones, and / or microelectromechanical systems (MEMS) microphones) for sensing sound waves (i.e., pressure differences) in a fluid (e.g., air) surrounding vehicle 200. Such acoustic sensors can be used to identify sounds in the surrounding environment (e.g., sirens, human voices, animal sounds, and / or alarms) upon which the control strategy of vehicle 200 may be based. For example, if the acoustic sensors detect a sirens (e.g., mobile sirens and / or fire truck sirens), vehicle 200 may decelerate and / or navigate to the edge of a road.

[0098] Although not in Figures 2A-2EAs shown, however, vehicle 200 may include a wireless communication system (e.g., similar to...). Figure 1 Wireless communication systems 146 and / or other than Figure 1 (In addition to the wireless communication system 146). The wireless communication system may include a wireless transmitter and receiver, which can be configured to communicate with devices external to or internal to the vehicle 200. Specifically, the wireless communication system may include transceivers configured to communicate with other vehicles and / or computing devices, such as in a vehicle communication system or road station. Examples of such vehicle communication systems include DSRC, radio frequency identification (RFID), and other proposed communication standards for intelligent transportation systems.

[0099] In addition to or in lieu of those shown, vehicle 200 may include one or more other components. These additional components may include electrical or mechanical functions.

[0100] The control system of vehicle 200 can be configured to control vehicle 200 according to one of a plurality of possible control strategies. The control system can be configured to receive information from sensors coupled to vehicle 200 (on or off vehicle 200), modify the control strategy (and associated driving behavior) based on that information, and control vehicle 200 according to the modified control strategy. The control system can also be configured to monitor information received from sensors and continuously evaluate driving conditions; and can also be configured to modify the control strategy and driving behavior based on changes in driving conditions. For example, the route taken by the vehicle from one destination to another can be modified based on driving conditions. Additionally or alternatively, speed, acceleration, turning angle, following distance (i.e., distance to the vehicle in front of the current vehicle), lane selection, etc., can all be modified in response to changes in driving conditions.

[0101] As described above, in some embodiments, the vehicle 200 may take the form of a truck, but alternative forms are also possible and are contemplated herein. Thus, Figures 2F-2I An embodiment of vehicle 250 in semi-truck form is shown. For example, Figure 2F A front view of vehicle 250 is shown, and Figure 2G An isometric view of vehicle 250 is shown. In an embodiment where vehicle 250 is a semi-trailer truck, vehicle 250 may include a tractor unit 260 and a trailer unit 270 (in... Figure 2G (as shown in the image). Figure 2H The tractor unit 260 is provided in both side and top views. It is similar to the vehicle 200 shown above. Figures 2F-2I The vehicle 250 shown may also include various sensor systems (e.g., similar to the reference 1000). Figures 2A-2ESensor systems 202, 206, 208, 210, 212, 214 are shown and described. In some embodiments, although Figures 2A-2E The vehicle 200 may include only a single copy of some sensor systems (e.g., sensor system 204), but Figures 2F-2I The vehicle 250 shown may also include multiple copies of the sensor system (e.g., sensor systems 204A and 204B as shown).

[0102] While the accompanying drawings and description may refer to a given form of vehicle (e.g., semi-truck vehicle 250 or van vehicle 200) throughout, it will be understood that the embodiments described herein are equally applicable to a variety of vehicle scenarios (e.g., with modifications taking into account the shape factor of the vehicle). For example, sensors and / or other components described or shown as part of van vehicle 200 can also be used in semi-truck vehicle 250 (e.g., for navigation and / or obstacle detection and avoidance).

[0103] Figure 2J The various sensor fields of view are shown (e.g., associated with the aforementioned vehicle 250). As described above, the vehicle 250 may contain multiple sensors / sensor units. For example, the positions of the various sensors may correspond to... Figures 2F-2I The location of the sensor is disclosed in the figures. However, in some cases, the sensor may have other locations. To simplify the figures, from... Figure 2J Sensor location labels are omitted from the attached diagram. For each sensor unit of vehicle 250, Figure 2J Representative fields of view are shown (e.g., fields of view labeled 252A, 252B, 252C, 252D, 254A, 254B, 256, 258A, 258B, and 258C). The sensor's field of view may include angular regions (e.g., azimuth and / or elevation regions) in which the sensor can detect objects.

[0104] Figure 2K An example embodiment is shown for use with a vehicle (e.g., reference). Figures 2F-2JThe beam steering of the sensors of the vehicle 250 is shown and described. In various embodiments, the sensor unit of the vehicle 250 may be radar, lidar, sonar, etc. Furthermore, in some embodiments, the sensor may be scanned within its field of view during operation. Various different scanning angles of the example sensor are shown as regions 272, each indicating the angular region on which the sensor is operating. The sensor may periodically or iteratively change the region on which it is operating. In some embodiments, the vehicle 250 may use multiple sensors to measure region 272. Additionally, other regions may be included in other examples. For example, one or more sensors may measure aspects of the trailer 270 of the vehicle 250 and / or the area directly in front of the vehicle 250.

[0105] At some angles, the sensor's operating area 275 may include the rear wheels 276A and 276B of the trailer 270. Therefore, the sensor can measure the rear wheels 276A and / or 276B during operation. For example, the rear wheels 276A and 276B may reflect lidar or radar signals transmitted by the sensor. The sensor can receive the reflected signals from the rear wheels 276A and 276. Therefore, the data collected by the sensor may include data from wheel reflections.

[0106] In some cases, such as when the sensor is radar, reflections from the rear wheels 276A and 276B can appear as noise in the received radar signal. Therefore, when the rear wheels 276A and 276B guide the radar signal away from the sensor, the radar can operate with an enhanced signal-to-noise ratio.

[0107] Figure 3 This is a conceptual illustration of wireless communication between various computing systems associated with an autonomous or semi-autonomous vehicle, according to an example embodiment. Specifically, wireless communication can occur between the remote computing system 302 and the vehicle 200 via network 304. Wireless communication can also occur between the server computing system 306 and the remote computing system 302, and between the server computing system 306 and the vehicle 200.

[0108] Vehicle 200 can correspond to various types of vehicles capable of transporting passengers or objects between locations, and can take any one or more forms of vehicles discussed above. In some cases, vehicle 200 can operate in an autonomous or semi-autonomous mode, which allows the control system to use sensor measurements to safely navigate vehicle 200 between destinations. When operating in autonomous or semi-autonomous mode, vehicle 200 can navigate with or without passengers. As a result, vehicle 200 can pick up and drop off passengers between desired destinations.

[0109] Remote computing system 302 can represent any type of device associated with remote assistance technology, including but not limited to those described herein. Within the example, remote computing system 302 can represent any type of device configured to (i) receive information related to vehicle 200, (ii) provide an interface through which a human operator can perceive the information and input a response related to the information, and (iii) send the response to vehicle 200 or other devices. Remote computing system 302 can take various forms, such as workstations, desktop computers, laptop computers, tablet computers, mobile phones (e.g., smartphones), and / or servers. In some examples, remote computing system 302 may include multiple computing devices operating together in a network configuration.

[0110] The remote computing system 302 may include one or more subsystems and components similar to or identical to those of the vehicle 200. At a minimum, the remote computing system 302 may include a processor configured to perform the various operations described herein. In some embodiments, the remote computing system 302 may also include a user interface including input / output devices such as a touchscreen and a speaker. Other examples are also possible.

[0111] Network 304 represents the infrastructure for enabling wireless communication between remote computing system 302 and vehicle 200. Network 304 also enables wireless communication between server computing system 306 and remote computing system 302, as well as between server computing system 306 and vehicle 200.

[0112] The location of the remote computing system 302 can vary within the example. For instance, the remote computing system 302 may have a remote location from the vehicle 200, having wireless communication via network 304. In another example, the remote computing system 302 may correspond to a computing device within the vehicle 200, which is detached from the vehicle 200, but can still be used by a human operator to interact with passengers or the driver of the vehicle 200. In some examples, the remote computing system 302 may be a computing device with a touchscreen operable by passengers of the vehicle 200.

[0113] In some embodiments, the operations described herein performed by the remote computing system 302 may additionally or alternatively be performed by the vehicle 200 (i.e., by any of the vehicle 200's systems or subsystems). In other words, the vehicle 200 may be configured to provide a remote assistance mechanism with which the vehicle's driver or passengers can interact.

[0114] Server computing system 306 can be configured to wirelessly communicate with remote computing system 302 and vehicle 200 via network 304 (or possibly directly with remote computing system 302 and / or vehicle 200). Server computing system 306 can represent any computing device configured to receive, store, determine, and / or transmit information related to vehicle 200 and its remote assistance. Thus, server computing system 306 can be configured to perform any of the operations(s) described herein as being performed by remote computing system 302 and / or vehicle 200, or portions thereof. Some embodiments of wireless communication related to remote assistance may utilize server computing system 306, while other embodiments may not.

[0115] Server computing system 306 may include one or more subsystems and components similar to or the same as those of remote computing system 302 and / or vehicle 200, such as processors configured to perform the various operations described herein, and wireless communication interfaces for receiving information from and providing information to remote computing system 302 and vehicle 200.

[0116] The various systems described above can perform a variety of operations. These operations and related characteristics will now be described.

[0117] Based on the above discussion, the computing system (e.g., remote computing system 302, server computing system 306, and / or the computing system local to vehicle 200) can operate to use cameras to capture images of the surrounding environment of the autonomous or semi-autonomous vehicle. Typically, at least one computing system will be able to analyze the images and, if possible, control the autonomous or semi-autonomous vehicle.

[0118] In some embodiments, to facilitate autonomous or semi-autonomous operation, a vehicle (e.g., vehicle 200) may receive data representing objects in the environment surrounding the vehicle (referred to herein as “environmental data”) in various ways. Sensor systems on the vehicle can provide environmental data representing objects in the surrounding environment. For example, the vehicle may have various sensors, including cameras, radar, lidar, microphones, radio units, and other sensors. Each of these sensors can transmit environmental data about the information received by each respective sensor to a processor within the vehicle.

[0119] In one example, the camera may be configured to capture still images and / or video. In some embodiments, the vehicle may have more than one camera positioned in different orientations. Moreover, in some embodiments, the camera may be able to move to capture images and / or video in different directions. The camera may be configured to store the captured images and video in memory for later processing by the vehicle's processing system. The captured images and / or video may be environmental data. Furthermore, the camera may include an image sensor as described herein.

[0120] In another example, the radar can be configured to emit electromagnetic signals reflected by various objects near the vehicle, and then capture the electromagnetic signals reflected from the objects. The captured reflected electromagnetic signals enable the radar (or processing system) to make various determinations about the objects reflecting the electromagnetic signals. For example, the distance to the various reflecting objects and the position of the various reflecting objects can be determined. In some embodiments, the vehicle may have more than one radar in different orientations. The radar can be configured to store the captured information in a memory for later processing by the vehicle's processing system. The information captured by the radar may be environmental data.

[0121] In another example, a lidar can be configured to transmit electromagnetic signals (e.g., infrared light, such as infrared light from a gas or diode laser or other possible light source) that will be reflected by a target object near the vehicle. The lidar can be able to capture the reflected electromagnetic (e.g., infrared light) signals. The captured reflected electromagnetic signals allow a ranging system (or processing system) to determine the distance to various objects. The lidar can also be able to determine the velocity or speed of the target object and store it as environmental data.

[0122] Additionally, in this example, the microphone can be configured to capture audio of the environment surrounding the vehicle. The sounds captured by the microphone can include emergency vehicle sirens and other vehicle sounds. For example, the microphone could capture the sirens of ambulances, fire trucks, or police cars. The processing system can then identify the captured audio signals to indicate an emergency vehicle. In another instance, the microphone could capture the exhaust sound of another vehicle, such as the exhaust sound of a motorcycle. The processing system can then identify the captured audio signals to indicate a motorcycle. The data captured by the microphone can form part of the environmental data.

[0123] In another example, the radio unit can be configured to transmit electromagnetic signals, which may take the form of Bluetooth signals, 802.11 signals, and / or other radio technology signals. The first electromagnetic radiation signal can be transmitted via one or more antennas located in the radio unit. Furthermore, the first electromagnetic radiation signal can be transmitted using one of many different radio signaling modes. However, in some embodiments, it is desirable to transmit the first electromagnetic radiation signal in a signaling mode that requests a response from a device located near the autonomous or semi-autonomous vehicle. The processing system can be able to detect nearby devices based on the responses transmitted back to the radio unit and use this transmitted information as part of environmental data.

[0124] In some embodiments, the processing system may be able to combine information from various sensors to further determine the vehicle's surroundings. For example, the processing system may combine data from both radar information and captured images to determine whether another vehicle or pedestrian is in front of the autonomous or semi-autonomous vehicle. In other embodiments, the processing system may use other combinations of sensor data to make determinations about the surrounding environment.

[0125] While operating in autonomous (or semi-autonomous) mode, the vehicle can control its operation with minimal human input. For example, a human operator can input an address into the vehicle, which can then drive to the designated destination without further human input (e.g., the human does not need to steer or touch the brake / accelerator pedal). Furthermore, while the vehicle operates autonomously or semi-autonomously, the sensor system can receive environmental data. The vehicle's processing system can modify the vehicle's control based on the environmental data received from various sensors. In some examples, the vehicle can change its speed in response to environmental data from various sensors. The vehicle can change its speed to avoid obstacles, comply with traffic regulations, etc. When the processing system in the vehicle identifies an object in its vicinity, the vehicle may be able to change its speed or otherwise alter its movement.

[0126] When a vehicle detects an object but is not highly confident in its detection, it can request a human operator (or a more powerful computer) to perform one or more remotely assisted tasks, such as (i) confirming whether the object actually exists in the surrounding environment (e.g., if a stop sign actually exists or if a stop sign does not actually exist), (ii) confirming whether the vehicle's identification of the object is correct, (iii) correcting the identification if incorrect, and / or (iv) providing supplementary instructions (or modifying current instructions) to an autonomous or semi-autonomous vehicle. Remotely assisted tasks may also include instructions provided by the human operator to control the vehicle's operation (e.g., instructing the vehicle to stop at the stop sign if the human operator determines the object is a stop sign), although in some scenarios, the vehicle itself may control its own operation based on feedback from the human operator related to the object's identification.

[0127] To facilitate this, the vehicle can analyze environmental data representing objects in the surrounding environment to identify at least one object with a detection confidence level below a threshold. A processor within the vehicle can be configured to detect various objects in the surrounding environment based on environmental data from various sensors. For example, in one embodiment, the processor can be configured to detect objects that would be important for vehicle identification. Such objects may include pedestrians, cyclists, street signs, other vehicles, indicator signals on other vehicles, and various other objects detected in the captured environmental data.

[0128] Detection confidence indicates the likelihood that a identified object is correctly identified or exists in the surrounding environment. For example, a processor may perform object detection on objects within image data in received environmental data, and determine that at least one object has a detection confidence below a threshold based on the premise that the object cannot be identified with a detection confidence above a threshold. If the result of object detection or object identification is uncertain, the detection confidence will be low or below the set threshold.

[0129] The vehicle can detect objects in the surrounding environment in various ways, depending on the source of the environmental data. In some embodiments, the environmental data may come from a camera and is image or video data. In other embodiments, the environmental data may come from lidar. The vehicle can analyze the captured image or video data to identify objects in the image or video data. The methods and apparatus can be configured to monitor image and / or video data for the presence of objects in the surrounding environment. In other embodiments, the environmental data may be radar, audio, or other data. The vehicle can be configured to identify objects in the surrounding environment based on radar, audio, or other data.

[0130] In some embodiments, the technology used by the vehicle to detect objects can be based on a set of known data. For example, data related to environmental objects can be stored in a memory located within the vehicle. The vehicle can compare the received data with the stored data to determine objects. In other embodiments, the vehicle can be configured to determine objects based on the context of the data. For example, street signs associated with buildings can typically be orange. Therefore, the vehicle can be configured to detect orange objects, located near the side of a road, as street signs associated with buildings. Additionally, when the vehicle's processing system detects an object in the captured data, it can also calculate a confidence level for each object.

[0131] Furthermore, the vehicle may also have a confidence threshold. The confidence threshold can vary depending on the type of object being detected. For example, for objects requiring a rapid response from the vehicle (such as brake lights on another vehicle), the confidence threshold may be lower. However, in other embodiments, the confidence threshold may be the same for all detected objects. When the confidence associated with a detected object is greater than the confidence threshold, the vehicle may assume the object has been correctly identified and adjust control of the vehicle accordingly based on that assumption.

[0132] The vehicle's actions can vary when the confidence level associated with a detected object is less than a confidence threshold. In some embodiments, the vehicle may react as if the detected object were present, even with a low confidence level. In other embodiments, the vehicle may react as if the detected object were not present.

[0133] When the vehicle detects an object in the surrounding environment, it can also calculate a confidence level associated with that specific detected object. Depending on the embodiment, the confidence level can be calculated in various ways. In one example, when an object in the surrounding environment is detected, the vehicle can compare environmental data with predetermined data associated with a known object. The closer the match between the environmental data and the predetermined data, the higher the confidence level. In other embodiments, the vehicle can use mathematical analysis of the environmental data to determine the confidence level associated with the object.

[0134] In response to determining that an object has a detection confidence level below a threshold, the vehicle may send a request to the telecomputing system for identification of the remotely assisted object. As mentioned above, the telecomputing system can take various forms. For example, the telecomputing system may be a computing device within the vehicle, separate from the vehicle but accessible to a human operator for interaction with passengers or the driver, such as a touchscreen interface for displaying remote assistance information. Additionally or alternatively, as another example, the telecomputing system may be a remote computer terminal or other device located not near the vehicle.

[0135] Requests for remote assistance may include environmental data, which may include objects such as image data, audio data, etc. The vehicle may transmit the environmental data to a remote computing system via a network (e.g., network 304) and, in some embodiments, via a server (e.g., server computing system 306). A human operator of the remote computing system may then use the environmental data as the basis for responding to the request.

[0136] In some embodiments, when an object is detected as having a confidence level below a confidence threshold, an initial object identification may be given, and the vehicle may be configured to adjust its operation in response to the initial identification. This operational adjustment may take the form of stopping the vehicle, switching the vehicle to manual control mode, changing the vehicle's speed (e.g., speed and / or direction), and other possible adjustments.

[0137] In other embodiments, even if the vehicle detects an object with a confidence level that meets or exceeds a threshold, the vehicle may act based on the detected object (e.g., stop if the object is identified as a stop sign with high confidence), but may be configured to request remote assistance while (or later) the vehicle is acting based on the detected object.

[0138] Figure 4A This is a block diagram of a system according to an example embodiment. Specifically, Figure 4A The system 400 is shown, which includes a system controller 402, a lidar device 410, multiple sensors 412, and multiple controllable components 414. The system controller 402 includes multiple processors 404, a memory 406, and instructions 408 stored in the memory 406 and executable by the multiple processors 404 to perform functions.

[0139] The (multiple) processors 404 can include one or more processors, such as one or more general-purpose microprocessors (e.g., having single or multiple cores) and / or one or more special-purpose microprocessors. One or more processors may include, for example, one or more central processing units (CPUs), one or more microcontrollers, one or more graphics processing units (GPUs), one or more tensor processing units (TPUs), one or more ASICs, and / or one or more field-programmable gate arrays (FPGAs). Other types of processors, computers, or devices configured to execute software instructions are also contemplated herein.

[0140] The memory 406 may include computer-readable media, such as non-transitory computer-readable media, which may include, but are not limited to, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), non-volatile random access memory (e.g., flash memory), solid-state drive (SSD), hard disk drive (HDD), compact disc (CD), digital video disc (DVD), digital magnetic tape, read / write (R / W) CD, R / W DVD, etc.

[0141] The lidar device 410, further described below, includes a plurality of light emitters configured to emit light (e.g., in the form of light pulses) and one or more photodetectors configured to detect the light (e.g., the reflected portion of the light pulse). The lidar device 410 can generate three-dimensional (3D) point cloud data from the output of the photodetectors(s) and provide the 3D point cloud data to a system controller 402. The system controller 402 can then perform operations on the 3D point cloud data to determine characteristics of the surrounding environment (e.g., relative positions of objects within the surrounding environment, edge detection, object detection, and / or proximity sensing).

[0142] Similarly, system controller 402 may use outputs from multiple sensors 412 to determine characteristics of system 400 and / or the surrounding environment. For example, sensors 412 may include one or more of GPS, IMU, image capture devices (e.g., cameras), light sensors, thermal sensors, and other sensors indicating parameters related to system 400 and / or the surrounding environment. For illustrative purposes, lidar device 410 is depicted as separate from sensor 412, and in some examples may be considered part of or considered as sensor 412.

[0143] Based on characteristics of system 400 and / or the surrounding environment determined by system controller 402 from outputs from lidar device 410 and sensor 412, system controller 402 can control controllable component 414 to perform one or more actions. For example, system 400 may correspond to a vehicle, in which case controllable component 414 may include the vehicle's braking system, turning system, and / or acceleration system, and system controller 402 may modify aspects of these controllable components based on characteristics determined from lidar device 410 and / or sensor 412 (e.g., when system controller 402 controls the vehicle in autonomous or semi-autonomous mode). In this example, lidar device 410 and sensor 412 may also be controlled by system controller 402.

[0144] Figure 4B This is a block diagram of a lidar device according to an example embodiment. Specifically, Figure 4BA lidar device 410 is shown, having a controller 416 configured to control a plurality of light emitters 424 and one or more photodetectors (e.g., a plurality of photodetectors 426). The lidar device 410 also includes excitation circuitry 428 configured to select a corresponding light emitter among the plurality of light emitters 424 and supply power to it; and may include selector circuitry 430 configured to select a corresponding photodetector among the plurality of photodetectors 426. The controller 416 includes processor(s) 418, memory 420, and instructions 422 stored in memory 420.

[0145] Similar to processor(s) 404, processor(s) 418 can include one or more processors, such as one or more general-purpose microprocessors and / or one or more special-purpose microprocessors. One or more processors may include, for example, one or more CPUs, one or more microcontrollers, one or more GPUs, one or more TPUs, one or more ASICs, and / or one or more FPGAs. Other types of processors, computers, or devices configured to execute software instructions are also envisioned herein.

[0146] Similar to memory 406, memory 420 may include computer-readable media, such as non-transitory computer-readable media, such as, but not limited to, ROM, PROM, EPROM, EEPROM, non-volatile random access memory (e.g., flash memory), SSD, HDD, CD, DVD, digital magnetic tape, R / W CD, R / W DVD, etc.

[0147] Instruction 422 is stored in memory 420 and can be executed by processor(s) 418 to perform functions associated with control excitation circuit 428 and selector circuit 430 for generating 3D point cloud data and for processing 3D point cloud data (or may facilitate processing of 3D point cloud data by another computing device, such as system controller 402).

[0148] The controller 416 can determine 3D point cloud data by emitting light pulses using light emitters 424. An emission time is established for each light emitter, and the relative position at the time of emission is also tracked. Various aspects of the environment surrounding the lidar device 410 (such as various objects) reflect the light pulses. For example, when the lidar device 410 is in an environment including roads, such objects may include vehicles, signs, pedestrians, road surfaces, building cones, etc. Some objects may be more reflective than others, such that the intensity of the reflected light can indicate the type of object reflecting the light pulse. Furthermore, the surface of an object may be in a different position relative to the lidar device 410, and therefore take more or less time to reflect a portion of the light pulse back to the lidar device 410. Therefore, the controller 416 can track the detection time when the reflected light pulse is detected by the photodetector and the relative position of the photodetector at the detection time. By measuring the time difference between the emission time and the detection time, the controller 416 can determine how far the light pulse travels before being received, and thus determine the relative distance to the corresponding object. By tracking the relative positions at the emission and detection times, the controller 416 is able to determine the orientation of the light pulses and reflected light pulses relative to the lidar device 410, and thus the relative orientation of the object. By tracking the intensity of the received light pulses, the controller 416 is able to determine how reflective the object is. Therefore, the 3D point cloud data determined based on this information can indicate the relative position of the detected reflected light pulses (e.g., in a coordinate system such as a Cartesian coordinate system) and the intensity of each reflected light pulse.

[0149] Excitation circuit 428 is used to select the light emitter for emitting light pulses. Selector circuit 430 is similarly used to sample the output from the photodetector.

[0150] Figure 5A This is an illustration of a lidar device, according to an example embodiment, that can be used to emit a set of optical signals and detect a set of reflected optical signals. For example, Figure 5A It can be used as a reference. Figure 4B The physical arrangement of the light emitter 424 and photodetector 426 within the lidar device 410 is shown and described. In some embodiments, such light emitters 424 and photodetectors 426 may be positioned (e.g., mounted to or fabricated on) on a substrate 500. Furthermore, the light emitters 424 and photodetectors 426 may be arranged as channels. Each channel may include a single light emitter 424 and a single corresponding photodetector 426. For example, as... Figure 5AAs shown, photodetector 426 can be positioned adjacent to its corresponding light emitter 424 on substrate 500 (e.g., above or below along the z-direction, as shown). However, it should be understood that other embodiments are possible and are contemplated herein. For example, multiple photodetectors may correspond to a single light emitter, multiple light emitters may correspond to a single photodetector, and / or photodetectors may not be positioned adjacent to their corresponding light emitters.

[0151] In addition, such as Figure 5A As illustrated, the light emitter 424 and the photodetector 426 can be connected (e.g., electrically connected) to the excitation circuit 428 and the selector circuit 430 (e.g., similar to...). Figure 4B For example, such a connection can be achieved using conductive trace 522. However, it will be understood that the techniques described herein remain broadly applicable, and Figure 5A The arrangement (including the combination of excitation circuit 428 and selector circuit 430) is provided only as an example.

[0152] The light emitter 424 in the array may include a light source such as a laser diode. In some embodiments, the light emitter 424 may include a pulsed light source. For example, the light source may include one or more pulsed lasers (e.g., Q-switched lasers). In alternative embodiments, a continuous wave (CW) light source may be used. In some embodiments, the light emitter 424 may include a fiber laser coupled to an optical amplifier. In particular, the fiber laser may be a laser in which the active gain medium (i.e., the optical gain source within the laser) is in an optical fiber. Furthermore, the fiber laser can be arranged in various ways within the lidar device 410 (e.g., partially or completely disposed on the substrate 500). However, in other embodiments, one or more light emitters 424 in the array may additionally or alternatively include LEDs, vertical-cavity surface-emitting lasers (VCSELs), organic light-emitting diodes (OLEDs), polymer light-emitting diodes (PLEDs), light-emitting polymers (LEPs), liquid crystal displays (LCDs), MEMS, and / or any other devices configured to selectively transmit, reflect, and / or emit light to provide emitted beams and / or pulses. The light emitter 424 can be configured to emit light signals toward objects in the surrounding environment. When reflected by such objects, the light signals can be detected by the light detector 426 to determine the distance between the lidar device 410 and the corresponding object.

[0153] The wavelength range emitted by the light emitter 424 can be, for example, in the ultraviolet, visible, and / or infrared portions of the electromagnetic spectrum. In some examples, the wavelength range can be a narrow wavelength range, such as that provided by a laser. In some embodiments, the wavelength range includes wavelengths of approximately 905 nm. Note that this wavelength is provided by way of example only and is not intended to be limiting.

[0154] Although not in Figure 5A As shown, but will be understood, the optical signals (e.g., light pulses) emitted by the light emitters 424 in the array can be transmitted to the surrounding environment via one or more lenses, mirrors, color filters, polarizers, waveguides, apertures, etc. For example, in some embodiments, the optical signals from the light emitters 424 can be redirected, focused, collimated, filtered, and / or otherwise adjusted before being transmitted to the surrounding environment. In some embodiments, the light emitters 424 can transmit the optical signals to the surrounding environment using shared optics (e.g., a single lens shared among all light emitters 424 or a group of light emitters 424) and / or using optics specific to a single light emitter 424 (e.g., a polarizer or color filter used only by that light emitter 424).

[0155] In some embodiments, for example, each of the light emitters 424 can transmit light signals to different areas of the surrounding environment to observe a field of view in the surrounding environment. The location within the surrounding environment where a given light emitter 424 can emit a light signal can depend on the position of the light emitter 424 (e.g., the (y, z) position of the light emitter 424 on the substrate 500), the angular orientation of the light emitter 424 relative to the surface of the substrate 500 (if any), and / or the position / or orientation of optics (e.g., mirrors and / or lenses) through which the light emitter 424 provides a light signal to the surrounding environment. In only one example, the light emitters 424 on the substrate 500 can emit light signals across a range of azimuth and / or elevation angles (e.g., to inquire about corresponding angular ranges within the surrounding environment) based on the position of the light emitters 424 on the substrate 500 relative to a shared telecentric lens assembly used by each light emitter 424 to provide light to the surrounding environment. Due to the shape of the shared telecentric lens assembly, the light signal can diffuse across the range of azimuth and / or elevation angles.

[0156] Photodetector 426 may include various types of detectors (e.g., single-photon detectors). For example, photodetector 426 may include a SPAD and / or a SiPM. The SPAD may employ avalanche breakdown within a reverse-biased pn junction (i.e., a diode) to increase the output current on the SPAD for a given incident illumination. Furthermore, the SPAD may be able to generate multiple electron-hole pairs for a single incident photon. In some embodiments, photodetector 426 may be biased above the avalanche breakdown voltage. Such a bias condition can create a positive feedback loop with a loop gain greater than 1. Additionally, a SPAD biased above a threshold avalanche breakdown voltage may be single-photon sensitive. In other examples, photodetector 426 may include a photoresistor, a charge-coupled device (CCD), a photovoltaic cell, and / or any other type of photodetector.

[0157] In some embodiments, the array of photodetectors 426 may include more than one type of photodetector across the array. For example, the photodetector array 426 can be configured to detect light of multiple predetermined wavelengths (e.g., in embodiments where light emitter 424 emits light of different wavelengths across the light emitter array 424). To this end, for example, the photodetector array 426 may include some SPADs sensitive to one wavelength range and other SPADs sensitive to different wavelength ranges. In some embodiments, the photodetector 426 may be sensitive to wavelengths (visible and / or infrared wavelengths) between 400 nm and 1.6 μm. Furthermore, the photodetector 426 may have various sizes and shapes. For example, the photodetector 426 may include SPADs having a package size of 1%, 0.1%, or 0.01% of the total area of ​​the substrate 500. Further, in some embodiments, one or more of the photodetectors 426 may include detector-specific optics. For example, each of the photodetectors 426 may include a microlens positioned above the photodetector 426 to enhance the amount of received light transmitted to the detection surface of the photodetector 426. Additionally or alternatively, one or more of the photodetectors 426 may include one or more optical filters (e.g., (multiple) neutral density filters, (multiple) polarization filters and / or (multiple) color filters).

[0158] As described above, each of the photodetectors 426 may correspond to a light emitter 424. In some embodiments, the photodetectors 426 may receive light from the surrounding scene via one or more optical devices (e.g., color filters, polarizers, lenses, mirrors, and / or waveguides). Such optical devices may be specific to one of the photodetectors 426 and / or shared by a group of photodetectors 426 (e.g., all photodetectors on the substrate 500). Furthermore, in some embodiments, in addition to being part of a receiving path for one or more of the photodetectors 426, one or more receiving optics may be part of a transmitting path for one or more of the light emitters 424. For example, a mirror may reflect light from one or more light emitters 424 into the surrounding environment and may also guide light received from the surrounding environment to one or more photodetectors 426.

[0159] As described above, the light emitter 424 can be configured to transmit optical signals to the surrounding environment across a range of azimuth and / or elevation angles (i.e., yaw and / or pitch angles). Similarly, based on the position of the light detector 426 in the lidar device 410, the light detector 426 can be arranged to receive optical signals reflected from objects in the environment surrounding the lidar device 410 across a corresponding range of azimuth and / or elevation angles (i.e., yaw and / or pitch angles).

[0160] The light emitter array 424 can be powered and / or controlled by the excitation circuit 428. Similarly, the photodetector 426 can be powered, controlled, and / or have a detection signal provided to the selector circuit 430 by the selector circuit 430. Figure 5A As shown, the excitation circuit 428 can be connected to one or more light emitters 424 via conductive traces 522 defined in the substrate 500, and the selector circuit 430 can be connected to one or more photodetectors 426 via conductive traces 522 defined in the substrate 500. Figure 5AA first conductive trace 522 connecting the excitation circuit 428 to the light emitter 424 and a second conductive trace 522 connecting the selector circuit 430 to the photodetector 426 are shown. It should be understood that this is provided by way of example only. In other embodiments, the excitation circuit 428 may be individually connected to each light emitter 424 via a separate conductive trace 522. Similarly, the selector circuit 430 may be individually connected to each photodetector 426 via a separate conductive trace 522. Alternatively, the excitation circuit 428 may be connected to a row of light emitters 424 via a single conductive trace 522, and / or the selector circuit 430 may be connected to a row of photodetectors 426 via a single conductive trace 522. For example, a group of four light emitters 424 may be connected to the excitation circuit 428 via a single conductive trace 522. In this way, a row of four light emitters 424 can be simultaneously excited by the excitation circuit 428. Other numbers of light emitters 424 or photodetectors 426 within a group are also possible.

[0161] In some embodiments, the excitation circuit 428 may include one or more capacitors. Such capacitors may be charged by one or more power sources. Then, in order for the light emitter 424 to emit a light signal (i.e., "excite"), the energy stored in the capacitors can be released through the light emitter 424. In some embodiments, the excitation circuit 428 may cause the light emitters 424 to emit light signals simultaneously with each other. In other embodiments, the excitation circuit 428 may cause the light emitters 424 to emit light signals sequentially. Other excitation modes, including random and pseudo-random excitation modes, are also possible and are contemplated herein.

[0162] Additionally, in some embodiments, the excitation circuit 428 may be controlled by a controller (e.g., reference...). Figure 4BThe controller 416 shown and described controls the light emitter 424. The controller 416 can selectively excite the light emitter 424 using the excitation circuit 428 via an excitation control signal (e.g., according to a predefined pattern). In some embodiments, the controller 416 can also be configured to control other functions of the lidar device 410. For example, the controller can control the movement of one or more movable stations associated with the lidar device 410 and / or generate a point cloud representation of the environment surrounding the lidar device 410 based on electronic signals received from the photodetector 426 in the lidar device 410, the electronic signals corresponding to light signals detected reflected from objects in the environment. In various embodiments, the generation of the point cloud representation can be performed based on the intensity of the detected signal compared to the intensity of the emitted signal and / or based on the timing of the detected signal compared to the timing of the emitted signal. In alternative embodiments, data regarding the detected light signal and / or the emitted light signal (e.g., timing data or intensity data) can be sent to a separate computing device (e.g., a remotely located server computing device; or a vehicle-mounted controller, such as a reference device). Figure 4A The system controller 402 is shown and described. A separate computing device can be configured to generate point cloud representations (e.g., and store the point cloud representations in a memory such as memory 406, and / or send the point cloud representations to the lidar controller).

[0163] Will understand, Figure 5A The arrangement shown is provided as an example, and other embodiments are possible and contemplated herein. For example, the lidar device 410 may alternatively include a plurality of substrates 500, each substrate 500 having a light emitter 424 and a photodetector 426 thereon. Additionally or alternatively, in some embodiments, the number of light emitters 424 on the substrates 500 may be [number missing]. Figure 5A The number of photodetectors 426 on the substrate 500 can vary depending on whether the number of light emitters 424 is more or less than sixteen. Figure 5A The arrangement of the light emitters 424 on the substrate 500 can differ from that shown (e.g., more or fewer than sixteen photodetectors 426). Figure 5A The arrangement of the photodetector 426 on the substrate 500 can differ from that shown. Figure 5A The positions and / or numbers of the conductive traces 522 shown may differ from those shown. Figure 5A As shown, the relative dimensions of one or more of the light emitters 424 can be different from those of the light emitters 424. Figure 5A The different, and / or the relative sizes of one or more of the photodetectors 426 shown may be related to Figure 5A The differences shown are not identical. Other differences are also possible and are envisioned in this paper.

[0164] Figure 5B This is a lidar device according to an example embodiment (e.g., reference). Figure 4A , Figure 4B and Figure 5A This illustration shows potential crosstalk between channels within a lidar device 410. For example, lidar device 410 may include a substrate 500, a channel array (e.g., each channel includes a light emitter 424 and a photodetector 426, such as...). Figure 5A (as shown), excitation circuit 428, selector circuit 430 and conductive trace 522.

[0165] As an example, lidar device 410 may include a first light emitter 502. The first light emitter 502 can emit light signals into the surrounding environment. Typically, when the emitted light signal is reflected by a surface with moderate reflectivity, a moderately intense reflected light signal 504 can be guided back to the lidar device. Figure 5B As shown, a moderately intense reflected light signal 504 can illuminate the corresponding first photodetector 514 within the lidar device 410. Furthermore, the magnitude of the moderately intense reflected light signal 504 may be insufficient to substantially and / or measurably illuminate the other photodetectors 426 within the lidar device. However, if the light signal emitted from the first light emitter 502 into the surrounding environment is reflected by a surface with high reflectivity (e.g., a retroreflector), the intensity of the reflected signal can be higher and / or occupy a larger detectable area when incident on the photodetector array 426. Figure 5B As shown, the high-intensity reflected light signal 506 can illuminate multiple photodetectors 426. For example, the high-intensity reflected light signal 506 can illuminate a first photodetector 514 and one or more second photodetectors 516. The second photodetector 516 may be referred to herein as a photodetector susceptible to crosstalk (e.g., crosstalk from reflected light signals generated by the emitted signal from the first light emitter 502), which means that the second photodetector 516 may undesirably detect light from the first channel (e.g., this may lead to noise or incorrect detection events based on the detection of the second photodetector 516).

[0166] It will be understood that which photodetectors 426 within the lidar device can detect a given reflected signal (e.g., which photodetectors 426 are susceptible to crosstalk) can depend on the intensity of the reflected signal (e.g., based on the reflectivity of surfaces in the surrounding environment), the sensitivity of the photodetectors 426, the position of the photodetectors 426 in the lidar device, the orientation of the photodetectors 426 in the lidar device (e.g., the azimuth / yaw angle and / or elevation / pitch angle orientation of the photodetectors 426), the distance to the reflecting surface in the surrounding environment, etc. For example, in some embodiments, the greater the distance from the lidar device 410 to the reflecting surface in the surrounding environment, the fewer photodetectors 426 may be affected by crosstalk (e.g., the greater the distance to the reflecting surface, the smaller the radius of the high-intensity reflected light signal 506). This could be a result of the reflected light signal attenuating / diverging as it propagates through the surrounding environment (e.g., due to dust, smoke, etc. in the surrounding environment), causing the intensity of the light signal to decrease as the distance between the lidar device and the reflecting surface increases.

[0167] In light of the foregoing, it will be understood that in various embodiments, reflected signals may be detected by the unintended photodetector 426 in the lidar device 410, resulting in crosstalk. The embodiments described herein may attempt to mitigate crosstalk within the lidar device 410 regardless of the cause (e.g., whether the crosstalk is caused by a high-reflectivity surface and / or, if the crosstalk is caused by a high-reflectivity surface, regardless of the spacing between the lidar device 410 and the high-reflectivity surface).

[0168] Figure 6A and Figure 6B The illustration shows techniques that can be used to mitigate crosstalk according to example embodiments. These techniques may include lidar devices (e.g., regarding...). Figure 4B , Figure 5A and Figure 5B Different channels of the lidar device 410 shown and described emit optical signals according to different emission patterns and / or emission sequences, and subsequently detect the reflected optical signals. For example, Figure 6A The light signal emitted by the light emitter 424 of the lidar device 410 during the first cycle and the reflected light signal detected by the photodetector 426 of the lidar device 410 can be shown. Figure 5AAs shown, the lidar device 410 may have 16 channels (e.g., numbered channel 0, channel 1, channel 2, ..., channel 15), each channel including a light emitter 424 and a corresponding photodetector 426. During the first cycle, the light emitter 424 of each of the 16 channels may emit a light signal, and if the emitted light signal is reflected from an object in the surrounding environment, the corresponding photodetector 426 may detect the reflected light signal. The photodetector 426 within the lidar device 410 may wait for the reflected light signal during a listening window of the first cycle. This listening window may have a sufficient duration to allow light signals reflected from distant objects (e.g., objects greater than 150m, 200m, 250m, 300m, 350m, 400m, 450m, or 500m) to still be detected by the photodetector 426.

[0169] on the other hand, Figure 6B The light signal emitted by the light emitter 424 of the lidar device 410 during the second period and the reflected light signal detected by the photodetector 426 of the lidar device 410 can be shown. Figure 6B As shown, during the second period, only a subset of channels can emit light (e.g., during the sequential listening window). This emission strategy prevents crosstalk caused by simultaneous emission / detection of adjacent channels. For example, as shown, only the light emitters 424 in even-numbered channels (e.g., channel 0, channel 2, channel 4) can emit light signals during the second period. Similar to the first period, the photodetector 426 within the lidar device 410 can wait for reflected light signals during the listening window of the second period. However, this second listening window can only have a sufficient duration to allow the photodetector 426 to detect light signals reflected from relatively close objects (e.g., objects less than 150m, less than 125m, less than 100m, less than 75m, or less than 50m away). Because the listening window during the second period can be shorter in duration than the listening window during the first period, the total duration of the second period can be less than the total duration of the first period. Alternatively (e.g., if the total duration of the second period is the same as or greater than the total duration of the first period), one or more emissions from optical emitter 424 during the second period can be staggered relative to each other in time. This can provide further robustness against crosstalk between channels because their emission / detection windows during the second period do not overlap (e.g., due to the temporal staggering).

[0170] By combining the longer-range detection capability of the first cycle with the shorter-range but more crosstalk-resistant detection capability of the second cycle, an enhanced dataset (e.g., one or more point clouds) can be generated. For example, a complete excitation cycle of the lidar device 410 may include a first cycle followed by a second cycle. During the excitation cycle, multiple emission / detection events from the first and second cycles can be recorded and combined to generate a dataset that mitigates the negative effects of crosstalk.

[0171] Figure 6C The first cycle is shown (e.g., reference). Figure 6A The excitation diagram for the first cycle is shown and described. "1" indicates that the corresponding channel is emitting a light signal at a specified time point, while "0" indicates that the corresponding channel is suppressing light signal emission. Therefore, as... Figure 6C As shown, each light emitter 424 in each channel of the lidar device 410 can be simultaneously excited. Subsequently, the photodetectors 426 of the channels of the lidar device 410 can use a single listening window with a duration sufficient for relatively distant objects. For example, a listening window between 2.0 μs and 3.0 μs can be used (e.g., 2.5 μs, which corresponds to a distance of 375 m). This can be achieved by simultaneously exciting groups 424 of light emitters in multiple channels of the lidar device 410 (e.g., by simultaneously exciting all light emitters 424 in all channels of the lidar device 410, such as...). Figure 6C As shown, the adverse effects of internal reflections within the lidar device 410 can be mitigated. For example, after the excitation of the light emitter 424, there may be a brief period during which reflections of internal components of the lidar device 410 are detected by one or more photodetectors 426. Such detected internal reflections can effectively prevent those photodetectors 426 from detecting signals reflected from the surrounding environment during this brief period (e.g., and in subsequent additional periods due to potential saturation of the photodetectors 426) (i.e., effectively blinding the photodetectors 426). By simultaneously exciting multiple light emitters 424, the time periods of internal reflections excited by those light emitters 424 can overlap, thereby reducing the total time spent blinding the photodetectors 426 within the lidar device 410 (e.g., when compared to an alternative sequential excitation sequence).

[0172] on the other hand, Figure 6D The second cycle is shown (e.g., reference). Figure 6B The excitation diagram of the second cycle is shown and described. (e.g.) Figure 6DAs shown, the second cycle can include multiple excitation times and multiple corresponding listening windows. During each excitation time, only a single channel can emit a light signal. For example, as shown, during the first excitation time of the second cycle, the light emitter 424 of channel 0 can emit a light signal. After channel 0 emits a light signal, the photodetector 426 of the lidar device 410 can attempt to detect the reflected signal within a listening window with a duration sufficient for a relatively close object. For example, a listening window between 0.3 μs and 0.7 μs can be used (e.g., 0.5 μs, which corresponds to a distance of 75 m). Subsequently, during the second excitation time of the second cycle, the light emitter 424 of channel 2 can emit a light signal. After channel 2 emits a light signal, the photodetector 426 of the lidar device 410 can again attempt to detect the reflected signal within a listening window (e.g., a listening window with the same duration as the previous listening window or a different duration). This process can continue to channel 4, then channel 6, then channel 8, then channel 10, then channel 12, and finally channel 14. This is in Figure 6D The middle is indicated by three points adjacent to the fifth listening window.

[0173] In an alternative embodiment, every two channels can be excited sequentially during the sequential excitation time until the total time allocated to the second cycle expires (e.g., if 3.0 μs is allocated to the second cycle, only six excitation times / 0.5 μs listening windows can be used). Channels can be selected such that the channels used during the second cycle are uniformly distributed among the photodetectors 426 of the lidar device 410. Additionally or alternatively, in some embodiments, channels (e.g., channels 0 through 15) may be excited in an alternating manner during the second cycle (e.g., channel 0, then channel 2, then channel 4, then channel 6, then channel 8, then channel 10, then channel 12, then channel 14, then channel 1, then channel 3, then channel 5, then channel 7, then channel 9, then channel 11, then channel 13, then channel 15 or channel 0, then channel 3, then channel 6, then channel 9, then channel 12, then channel 15, then channel 1, then channel 4, then channel 7, then channel 10, then channel 13, then channel 2, then channel 5, then channel 8, then channel 11, then channel 14). In other embodiments, the use of which channels during the second cycle may be determined based on the degree of contamination (e.g., the presence of condensation, rain, snow, ice, cracks, insect residue, and / or dust) on one or more optics of the lidar device 410. For example, if a crack exists in the optical window in front of the light emitter 424 or photodetector 426 in a given channel, that channel can be avoided during the second cycle. The degree of contamination can be based on previous measurements using lidar device 410 or different sensors and / or on ambient weather conditions near lidar device 410 (e.g., weather forecast, current temperature, and / or Doppler radar data). After the second cycle is completed, the detection results from the second cycle can be combined with the detection results from the first cycle to generate a dataset that can be used to produce one or more point clouds.

[0174] The channels used during the second cycle (e.g., iteratively passing through consecutive excitation / listening windows) can be selected using various methods. For example, one or more of the above arrangements (e.g., every two channels, every three channels, and / or alternately paired channels) can be stored in the memory of the lidar device's controller. Such a memory may include a predetermined list of which channels are excited in which order during the second cycle. In such embodiments, the channel sequence used may be fixed and repeated across emission cycles. In some embodiments, for example, channels can be excited in a round-robin manner during the second cycle of consecutive excitation cycles according to a stored predetermined list.

[0175] Alternatively, the selection of which channels to use during the second period can be based on one or more previous excitation periods and / or on the first period of the corresponding excitation period. For example, by analyzing the first period, high-intensity return signals can be identified. Such high-intensity return signals may indicate the presence of one or more highly reflective surfaces (e.g., retroreflectors) in the surrounding environment. Furthermore, such highly reflective surfaces may be more likely to cause crosstalk. Thus, channels that detected high-intensity returns during the first period (e.g., and channels within a predetermined angle of that channel in the lidar device) may not be used during the second period. However, in other embodiments (e.g., and perhaps more likely), those channels near the detected high-intensity return signals can be intentionally probed during the second period. Because channels near the channels receiving high-intensity returns during the first period are most likely to be affected by crosstalk during the first period, probing those channels individually during the second period would be most informative. Therefore, channels within a predetermined angular interval from the channels receiving high-intensity returns during the first period can be iterated during the second period (e.g., sequentially excited during the excitation slots of the second period).

[0176] In other embodiments, the channels used for excitation during the second period can be selected to provide the most robust coverage across the entire angular field of view during the second period. For example, a subset of channels (e.g., channel pairs, channel triplets, channel quadruplets) can be selected for each excitation slot in the second period. Each channel within a given subset can be selected such that a minimum predetermined angular resolution condition is satisfied. For example, channels can be selected such that each channel within each subset of channels excited during each excitation slot in the second period is separated by at least a predetermined degree in azimuth and / or elevation. The predetermined degree can be determined based on one or more optical components of the lidar device (e.g., aperture, lens, waveguide, mirror, and / or window).

[0177] Figure 6E This is a diagram of the channels in the lidar device 410. The lidar device 410 can be configured according to... Figure 6D The emission and detection of optical signals are shown and described during the second period of the excitation sequence for emitting and detecting optical signals. For example, in... Figure 6D The channels used during the second cycle are shown inside the boxes with dashed lines. (As shown...) Figure 6EAs shown, the photodetectors 426 used can be staggered relative to each other (e.g., perpendicularly along the z-direction) such that light reflected from objects in the surrounding environment (e.g., light reflected during a previous listening window) does not reach the detection plane of the lidar device 410 before a subsequent listening window, indicating that crosstalk may not be detected (e.g., because adjacent photodetectors 426 within the lidar device 410 are not used, even in sequential listening windows). It should be understood that other arrangements of other photodetectors 426 can be used during the second cycle (e.g., Figure 6F (The arrangement shown).

[0178] Furthermore, to understand, Figure 6D The excitation sequence shown is an example, and other excitation modes for the second cycle are possible and contemplated herein. For example, each channel (instead of every two channels) can be configured to emit light during a sequential excitation time / listening window (e.g., channel 0, then channel 1, then channel 2, then channel 3). Alternatively, every three channels (instead of every two channels) can be configured to emit light during a sequential excitation time / listening window (e.g., channel 0, then channel 3, then channel 6, then channel 9). The interval between channels excited during the sequential time window can be based on the physical spacing of the photodetectors 426 in the lidar device 410. For example, the proximity of the photodetectors 426 to each other can be used to determine how many subsequent channels are susceptible to crosstalk from adjacent channels, and only those channels less susceptible to crosstalk from each other can be selected for adjacent excitation time / listening windows.

[0179] In other embodiments, more than one channel may be used during each excitation time / listening window during the second cycle. For example, as... Figure 6GAs shown, two channels can be excited during each excitation time in the second cycle. In some embodiments, the two channels selected for simultaneous excitation can be as far apart from each other as possible within the lidar device 410 (e.g., by channel indexing and / or physical location within the lidar device 410). This prevents crosstalk between the two channels used during the associated listening window. For example, as shown, during the first excitation time / listening window of the second cycle, light emitters 424 of channels 0 and 8 can emit light signals, while during the second excitation time / listening window of the second cycle, light emitters 424 of channels 1 and 9 can emit light signals, and so on. This allows for a larger total number of channels to be used during the second cycle (e.g., thereby increasing the resolution of the resulting dataset) while still preventing the possibility of crosstalk. Furthermore, in some embodiments, the channels selected for simultaneous excitation during the second cycle can be chosen such that no crosstalk occurs between the channels even when a highly reflective object (e.g., a retroreflector) located at the maximum detectable distance is illuminated based on the duration of the second listening window. The phrase "so that no crosstalk occurs between the channels" is used throughout this disclosure. It will be understood that while embodiments in which no crosstalk is provided between channels are clearly envisioned, the phrase also envisions embodiments in which significantly reduced crosstalk is provided. For example, in some embodiments, there may be a minimum threshold strength for detection by the photodetectors(s) of the lidar device. Below this minimum threshold strength, detection events may not be registered (e.g., by the photodetectors(s) and / or by a computing device analyzing detection data from the photodetectors(s)). Then, in such embodiments, the phrase “so that no crosstalk occurs between channels” may correspond to a crosstalk level between channels that is less than the minimum threshold strength (e.g., while still maintaining a non-zero crosstalk signal between channels).

[0180] In other embodiments, three channels, four channels, five channels, etc., can be configured to emit / detect optical signals during each excitation time / listening window. Furthermore, in some embodiments, different numbers of channels can be used during different portions of the second cycle. For example, as... Figure 6HAs shown, channels 0, 7, and 14 can be used during the first excitation time / listening window of the second cycle; followed by channels 1, 5, 9, and 13 during the second excitation time / listening window of the second cycle; followed by channels 2, 8, and 15 during the third excitation time / listening window of the second cycle; and so on. It will be understood that other embodiments of the excitation sequence of the second cycle are also possible and are contemplated herein. For example, the duration of the sequential listening window of the second cycle can vary between listening windows (e.g., a 0.3 μs listening window, followed by a 0.5 μs listening window, followed by a 0.7 μs listening window, followed by a 0.5 μs listening window, followed by a 0.3 μs listening window).

[0181] The embodiments described herein can take advantage of the fact that when a nearby object is detected in the surrounding environment, the channels of the lidar device 410 can linearly over-resolution (i.e., have a resolution higher than necessary). This concept is based on... Figure 7A and 7B As shown in the image. Figure 7A The diagram shows that the lidar device 410 emits an optical signal with a first angular resolution during the first cycle, while Figure 7B The diagram shows the lidar device 410 emitting an optical signal with a second angular resolution during the second cycle. (See figure.) Figure 7A The first angular resolution (e.g., the number of light signals emitted per degree) is higher than Figure 7B The second angular resolution. Furthermore, the listening window used for the first period can correspond to a larger range than the listening window used for the second period. For example, as shown, the listening window used for the first period can correspond to 300m, while the listening window used for the second period can correspond to 50m (although the same principle shown will be applied to a different range than shown). As shown, although the second angular resolution of the emission pattern in the second period is lower, the linear resolution for both emission patterns (e.g., by along...) Figure 7A and Figure 7B The line representation in the y-direction corresponding to 1 cm can be the same. In other words, both emission patterns can be able to resolve objects as low as 1 cm within their respective ranges, even if they do not exhibit the same angular resolution. Therefore, even when some light signals are discarded from the emission pattern to prevent crosstalk, the result from the second period from a nearby object can still be used.

[0182] In some embodiments, in order to generate different excitation patterns used during the first and second cycles (e.g., such as...), Figure 6A and 6B As shown), the excitation circuit can be designed to adapt to both the first-cycle excitation sequence and the second-cycle excitation sequence. As an example, Figure 8AShowing a lidar device (e.g., reference) Figure 4B The excitation circuit 428 and associated light emitter 424 of the lidar device 410 (shown and described) are illustrated and described. Figure 8A As shown, each channel (e.g., channel 0 to channel 15) may have a corresponding signal line (e.g., CHG0 to CHG15) that can be used to select whether the channel will be excited during a given excitation time. These signal lines can be used to toggle a charging switch (e.g., a transistor) to charge the corresponding capacitor for the channel in question (e.g., charge it to the excitation voltage V). LASER Then, at the desired excitation time, a trigger control signal can be used to excite one or more light emitters 424 (e.g., laser diodes) by turning off the excitation switch (e.g., a transistor) so that current can flow from the capacitor through the respective light emitter 424.

[0183] Will understand, Figure 8A Provided only as an example, other excitation circuits 428 are possible and contemplated herein. As an alternative only, if the light emitters 424 are excited in groups of four (e.g., during the second cycle), then... Figure 8B The excitation circuit 428 and the light emitter 424 are arranged together. Figure 8A The excitation circuit of the 428 is different. Figure 8B The excitation circuit 428 may consist of only four signal lines (e.g., CHG0 to CHG3). Each signal line can be used to select whether a specific set of four channels will be excited during a given excitation time / associated listening window. Figure 8B As shown, the signal lines can be used to charge the four corresponding capacitors of the four corresponding channels (e.g., charge them to the excitation voltage V). LASER Then, at the desired excitation time, a trigger control (i.e., discharge) signal (e.g., DIS0 to DIS3) signal can be used to excite one or more groups of four light emitters 424 (e.g., laser diodes) by turning off four corresponding excitation switches (e.g., transistors) for a given group of light emitters 424, so as to allow current to flow from the charging capacitor through the corresponding light emitter 424 in that group.

[0184] The techniques described above for mitigating or eliminating crosstalk can be enhanced by additional or alternative techniques. As an example of an additional crosstalk mitigation technique that can be used in conjunction with the first / second cycle techniques described above, the identification of certain objects in the surrounding environment can be used to prevent crosstalk in future detection cycles. For example, highly reflective (e.g., retroreflective) objects in the surrounding environment can generate high-intensity reflections, which are likely to cause crosstalk. Thus, when a highly reflective object is identified in a given detection cycle (e.g., based on the intensity of the reflection detected from the object), the lidar device 410 can avoid emitting light signals toward that object in future detection cycles. For example, Figure 9 This shows an alternative set of potential signals emitted during the second cycle. Figure 9 The signal emitted during the second period (e.g., sequentially during different excitation times / listening windows) can be similar to that emitted during the second period. Figure 6B The signal was transmitted during the second cycle. However, Figure 6B and Figure 9 The traffic signs shown may have highly reflective components (e.g., the text of the sign may include reflectors). Therefore, to further mitigate crosstalk, Figure 9 The illustrated embodiment can also avoid emitting light signals toward highly reflective parts of the scene, including traffic signs (e.g., in...). Figure 9 In, with Figure 6B Unlike other light sources, the light emitter 424 of channel 10 can avoid emitting light signals during the second cycle. Highly reflective objects in the surrounding environment can be identified based on light signals emitted into the surrounding environment for the purpose of detection during one or more previous detection cycles. Additionally or alternatively, highly reflective objects in the surrounding environment can be identified based on calibration signals emitted into the surrounding environment by the light emitter 424 of lidar device 410, so that highly reflective objects are identified before runtime emission / detection will be performed.

[0185] In addition to, or instead of, modulating the listening window and / or angular resolution associated with the first / second cycle as described above, power modulation can also be performed to distinguish crosstalk signals from the detected signal. For example, a lower transmit power can be used for the optical signal emitted during the second cycle than for the optical signal emitted during the first cycle. This transmit power dichotomy can save energy (e.g., since the emitted optical signal does not need to travel as far during the second cycle because the listening window / range is shorter), reduce charging time (e.g., for capacitors used to excite the optical transmitter 424 based on the RC or RLC time constant of the charging circuit), reduce the likelihood of crosstalk in adjacent channels (e.g., lower intensity reflections due to the reduced transmit power), reduce the amount of dynamic range required by the photodetector / receiver of the lidar device, and / or prevent the photodetector from saturating during the second cycle. In some embodiments, for example, the transmit power used during the second cycle can be less than 75%, less than 50%, less than 25%, or less than 10% of the transmit power used during the first cycle.

[0186] Regardless of the differences employed between the first and second cycles, the embodiments described herein also include techniques for combining detection events from the first cycle with those from the second cycle. In some embodiments, the lidar device 410 may simply generate a dataset comprising two data segments; one segment may be used to generate a first point cloud (e.g., corresponding to the first cycle of the transmit / detect signal), and the other segment may be used to generate a second point cloud (e.g., corresponding to the second cycle of the transmit / detect signal). In other embodiments, the lidar device 410 (e.g., the controller 416 of the lidar device 410) may determine, for each channel used during both the first and second cycles, an apparent target distance based on detection events during the first cycle and an apparent target distance based on detection events during the second cycle. The lidar device 410 (e.g., the controller 416 of the lidar device 410) may then determine the difference between the two apparent distances for each channel. Then, for each channel, lidar device 410 (e.g., controller 416 of lidar device 410) can compare the difference in apparent distances with a threshold difference (e.g., between 0.1 and 5.0 m, such as 0.5 m, 1.0 m, or 2.5 m), and if the difference is less than the threshold difference, the apparent distance to one of the periods (e.g., the second period or the first period) is included in the dataset that can be used to generate a single point cloud representing a combination of detection events during the two periods. If the difference is greater than the threshold difference, lidar device 410 (e.g., controller 416 of lidar device 410) can alternatively exclude two apparent distances, include one of the apparent distances by default, calculate a blended distance representing some combination of the two measurements, and include the blended distance (e.g., with an associated confidence level based on the difference), etc.

[0187] Figure 10 This is a flowchart of method 1000 according to an example embodiment. In some embodiments, method 1000 may be performed to mitigate crosstalk from adjacent channels within a lidar device. In some embodiments, method 1000 may be performed by a device including a lidar device (e.g., Figure 4B and Figure 5A The system execution of the lidar device 410 shown.

[0188] At block 1002, method 1000 may include transmitting a first set of optical signals from a first set of optical emitters of a light detection and ranging (lidar) device into the surrounding environment. The first set of optical signals may correspond to a first angular resolution relative to the surrounding environment.

[0189] At box 1004, method 1000 may include detecting a first set of reflected light signals from the surrounding environment by a first set of photodetectors of the lidar device during a first listening window. The first set of reflected light signals may correspond to reflections of the first set of light signals from objects in the surrounding environment.

[0190] At block 1006, method 1000 may include transmitting a second set of optical signals from a second set of optical emitters of a lidar device into the surrounding environment. The second set of optical emitters of the lidar device may represent a subset of the first set of optical emitters of the lidar device. The second set of optical signals may correspond to a second angular resolution relative to the surrounding environment. The second angular resolution may be lower than the first angular resolution.

[0191] At block 1008, method 1000 may include detecting a second set of reflected light signals from the surrounding environment by a second set of photodetectors of the lidar device during a second listening window. The second set of photodetectors of the lidar device may represent a subset of the first set of photodetectors of the lidar device. The second set of reflected light signals may correspond to reflections of the second set of light signals from objects in the surrounding environment. The duration of the second listening window may be shorter than the duration of the first listening window.

[0192] At box 1010, method 1000 may include a dataset synthesized by the controller of the lidar device that can be used to generate one or more point clouds. The dataset may be based on a first set of detected reflected light signals and a second set of detected reflected light signals.

[0193] In some embodiments, method 1000 may further include emitting a third set of optical signals from a third set of optical emitters of the lidar device into the surrounding environment. The third set of optical emitters of the lidar device may represent a subset of the first set of optical emitters of the lidar device that differs from the second set of optical emitters. The third set of optical signals may correspond to a third triangular resolution relative to the surrounding environment. The third triangular resolution may be the same as the second triangular resolution. Method 1000 may further include detecting a third set of reflected optical signals from the surrounding environment by a third set of optical detectors of the lidar device during a third listening window. The third set of optical detectors of the lidar device may represent a subset of the first set of optical detectors of the lidar device that differs from the second set of optical detectors. The third set of reflected optical signals may correspond to reflections of the third set of optical signals from objects in the surrounding environment. The duration of the third listening window may be the same as the duration of the second listening window. The dataset may be based on the detected third set of reflected optical signals.

[0194] In some embodiments of method 1000, the third listening window may not overlap with the second listening window.

[0195] In some embodiments of method 1000, the second set of optical signals may include multiple optical signals. The third set of optical signals may include multiple optical signals. The second and third sets of optical signals may be interleaved relative to the surrounding environment.

[0196] In some embodiments of method 1000, the second set of photodetectors may include a plurality of photodetectors. The second set of photodetectors may be selected from the first set of photodetectors so as to be uniformly distributed across the first set of photodetectors.

[0197] In some embodiments of method 1000, the second set of photodetectors may be spatially distributed sufficiently sparsely across the first set of photodetectors such that no crosstalk occurs between the photodetectors within the second set when a retroreflector located at the maximum detectable distance is illuminated with a light signal from the second set of light signals. The maximum detectable distance may be based on the duration of the second listening window.

[0198] In some embodiments, method 1000 may further include transmitting calibration light signals from each optical emitter in the lidar device to the surrounding environment. Additionally, method 1000 may include detecting reflected calibration light signals from the surrounding environment by each photodetector of the lidar device during a calibration listening window. Each reflected calibration light signal may correspond to a reflection of one of the calibration light signals from an object in the surrounding environment. Furthermore, method 1000 may include identifying one or more optical emitters within the lidar device based on the detected reflected calibration light signals, wherein, for said one or more optical emitters, the corresponding calibration light signal is reflected from a retroreflector in the surrounding environment. Additionally, method 1000 may include selecting a first set of optical emitters from a set of all emitters in the lidar device. The first set of optical emitters may correspond to those optical emitters not identified as one or more optical emitters within the lidar device, wherein, for said one or more optical emitters, the corresponding calibration light signal is reflected from a retroreflector in the surrounding environment.

[0199] In some embodiments of method 1000, one or more light emitters within a lidar device may be identified based on the detected intensity of a corresponding detected reflected calibration light signal from the surrounding environment, wherein, for the one or more light emitters, the corresponding calibration light signal is reflected from a retroreflector in the surrounding environment.

[0200] In some embodiments of method 1000, transmitting the first set of optical signals into the surrounding environment may include transmitting the first set of optical signals into the surrounding environment using a first transmission power. Transmitting the second set of optical signals into the surrounding environment may include transmitting the second set of optical signals into the surrounding environment using a second transmission power. The second transmission power may be less than the first transmission power.

[0201] In some embodiments of method 1000, the second transmit power may be less than 25% of the first transmit power.

[0202] In some embodiments of method 1000, the dataset may be used to generate a first point cloud and a second point cloud. The first point cloud may include data related to a first set of detected reflected light signals. The second point cloud may include data related to a second set of detected reflected light signals.

[0203] In some embodiments of method 1000, the synthetic dataset may include: for each detected reflected light signal in the second set of detected reflected light signals, determining a second target distance based on the corresponding detected reflected light signal. The synthetic dataset may also include: for each detected reflected light signal in the second set of detected reflected light signals, determining a first target distance based on a corresponding detected reflected light signal in the first set of detected reflected light signals. The corresponding detected reflected light signals in the first set of detected reflected light signals may have already been detected by the same photodetector within the lidar device. Additionally, the synthetic dataset may include: for each detected reflected light signal in the second set of detected reflected light signals, determining the difference between the second target distance and the first target distance. Furthermore, the synthetic dataset may include: for each detected reflected light signal in the second set of detected reflected light signals, if the difference is less than a threshold difference, then including either the second target distance or the first target distance in the dataset.

[0204] In some embodiments of method 1000, the threshold difference can be between 0.1m and 5.0m.

[0205] In some embodiments of method 1000, the duration of the first listening window may be between 2.0 μs and 3.0 μs. The duration of the second listening window may be between 0.3 μs and 0.5 μs.

[0206] In some embodiments, method 1000 may further include determining which light emitters in the first group of light emitters are included in the second group of light emitters based on the degree of contamination of one or more optics of the lidar device.

[0207] In some embodiments of method 1000, the degree of contamination may be determined based on previous measurements using a lidar device or different sensors.

[0208] In some embodiments of method 1000, the degree of contamination can be determined based on ambient weather conditions near the lidar device.

[0209] In some embodiments of method 1000, the dataset may include multiple points associated with each detected reflected light signal in the second set of detected reflected light signals. Each of the multiple points may include a target distance. Each target distance may have an associated confidence level determined based on the detected reflected light signal in the second set of detected reflected light signals and the corresponding first detected reflected light signal in the first set of detected reflected light signals.

[0210] This disclosure is not limited to the specific embodiments described herein, which are intended to be illustrative of various aspects. Many modifications and variations are possible without departing from the spirit and scope of the invention, as will be apparent to those skilled in the art. In addition to those listed herein, functionally equivalent methods and apparatus within the scope of this disclosure will be apparent to those skilled in the art from the foregoing description. These modifications and variations are intended to fall within the scope of the appended claims.

[0211] The above detailed description, with reference to the accompanying drawings, illustrates various features and functions of the disclosed systems, devices, and methods. In the drawings, similar symbols generally identify similar components unless the context otherwise requires. The exemplary embodiments described herein and in the drawings are not intended to be limiting. Other embodiments and other changes can be utilized without departing from the scope of the subject matter presented herein. It will be readily understood that the aspects of this disclosure, as generally described herein and illustrated in the drawings, can be arranged, replaced, combined, separated, and designed in a variety of different configurations, all of which are expressly contemplated herein.

[0212] With respect to any or all of the message flow diagrams, scenarios, and flowcharts in the accompanying drawings and as discussed herein, each step, block, operation, and / or communication can represent the processing and / or transmission of information according to exemplary embodiments. Alternative embodiments are included within the scope of these exemplary embodiments. In these alternative embodiments, for example, operations described as steps, blocks, transmissions, communications, requests, responses, and / or messages can be performed not in the order shown or discussed, including substantially simultaneously or in reverse order, depending on the functionality involved. Furthermore, more or fewer blocks and / or operations can be associated with any message flow diagram, scenario, and flowchart discussed herein. Figure 1 They can be used together, and these message flow diagrams, scenarios, and flowcharts can be combined with each other, either partially or entirely.

[0213] The steps, blocks, or operations representing information processing can correspond to circuitry that can be configured to perform specific logical functions of the methods or techniques described herein. Alternatively or additionally, the steps or blocks representing information processing can correspond to modules, segments, or portions of program code (including associated data). The program code can include one or more instructions executable by a processor for implementing specific logical operations or actions in the method or technique. The program code and / or associated data can be stored on any type of computer-readable medium, such as storage devices including RAM, disk drives, solid-state drives, or other storage media.

[0214] Furthermore, steps, frames, or operations representing one or more information transfers can correspond to information transfers between software and / or hardware modules in the same physical device. However, other information transfers can occur between software and / or hardware modules in different physical devices.

[0215] The specific arrangements shown in the accompanying drawings should not be considered limiting. It should be understood that other embodiments are capable of including more or fewer of each element shown in a given figure. Furthermore, some of the elements shown can be combined or omitted. Additionally, the example embodiments are capable of including elements not shown in the drawings.

[0216] While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for illustrative purposes and not restrictive; the true scope is indicated by the appended claims.

Claims

1. A method for a lidar (light detection and ranging) device, comprising: A first set of optical signals is emitted from the first set of optical emitters of the lidar device into the surrounding environment, wherein the first set of optical signals corresponds to a first angular resolution relative to the surrounding environment; The first set of photodetectors of the lidar device detects a first set of reflected light signals from the surrounding environment during the first listening window, wherein the first set of reflected light signals corresponds to the reflection of the first set of light signals from objects in the surrounding environment; A second set of optical signals is emitted from a second set of optical emitters of the lidar device into the surrounding environment, wherein the second set of optical emitters of the lidar device represents a subset of the first set of optical emitters of the lidar device, wherein the second set of optical signals corresponds to a second angular resolution relative to the surrounding environment, and wherein the second angular resolution is lower than the first angular resolution; A second set of photodetectors from the lidar device detects a second set of reflected light signals from the surrounding environment during a second listening window, wherein the second set of photodetectors represents a subset of the first set of photodetectors from the lidar device, wherein the second set of reflected light signals corresponds to reflections of the second set of light signals from objects in the surrounding environment, and wherein the duration of the second listening window is shorter than the duration of the first listening window; and The controller of the lidar device synthesizes datasets that can be used to generate one or more point clouds, wherein the datasets are based on a first set of detected reflected light signals and a second set of detected reflected light signals. The synthetic dataset includes each detected reflected light signal from the second set of detected reflected light signals: The distance to the second target is determined based on the detected reflected light signals. The first target distance is determined based on the corresponding detected reflected light signal in the first group of detected reflected light signals, wherein the corresponding detected reflected light signal in the first group of detected reflected light signals is detected by the same photodetector in the lidar device; Determine the difference between the distance to the second target and the distance to the first target; and If the difference is less than a threshold difference, then the second target distance or the first target distance is included in the dataset.

2. The method according to claim 1, further comprising: A third set of optical signals is emitted from the third set of optical emitters of the lidar device into the surrounding environment, wherein the third set of optical emitters of the lidar device represents a subset of the first set of optical emitters that differs from the second set of optical emitters, wherein the third set of optical signals corresponds to a third angular resolution relative to the surrounding environment, and wherein the third angular resolution is the same as the second angular resolution; and A third set of photodetectors from the lidar device detects a third set of reflected light signals from the surrounding environment during a third listening window. This third set of photodetectors represents a subset of the first set of photodetectors from the second set. The third set of reflected light signals corresponds to reflections of light from objects in the surrounding environment. The duration of the third listening window is the same as the duration of the second listening window. The dataset is based on the detected third set of reflected light signals.

3. The method according to claim 2, wherein, The third listening window does not overlap with the second listening window.

4. The method according to claim 2, wherein, The second set of optical signals includes multiple optical signals, and the third set of optical signals includes multiple optical signals, wherein the second set of optical signals and the third set of optical signals are emitted alternately in space in the surrounding environment.

5. The method according to claim 1, wherein, The second set of photodetectors includes multiple photodetectors, wherein the second set of photodetectors is selected from the first set of photodetectors so as to be uniformly distributed across the first set of photodetectors.

6. The method according to claim 1, wherein, The second set of photodetectors is distributed across the first set of photodetectors in space such that when a retroreflector located at the maximum detectable distance is illuminated with a light signal from the second set of light signals, no crosstalk occurs between the photodetectors in the second set, wherein the maximum detectable distance is based on the duration of the second listening window.

7. The method according to claim 1, further comprising: Each optical transmitter in the lidar device emits a calibration optical signal into the surrounding environment; During the calibration listening window, each photodetector of the lidar device detects reflected calibration light signals from the surrounding environment, where each reflected calibration light signal corresponds to a reflection of one of the calibration light signals from an object in the surrounding environment; One or more light emitters within the lidar device are identified based on detected reflected calibration light signals, for which corresponding calibration light signals are reflected from a retroreflector in the surrounding environment; and Select the first group of optical transmitters of the lidar device from the set of all transmitters in the lidar device, wherein the first group of optical transmitters corresponds to those optical transmitters that are not identified as one or more optical transmitters within the lidar device, for which the corresponding calibration optical signal is reflected from a retroreflector in the surrounding environment.

8. The method according to claim 7, wherein, One or more light emitters within the lidar device are identified based on the intensity of the corresponding detected reflected calibration light signals from the surrounding environment, for which the corresponding calibration light signals are reflected from a retroreflector in the surrounding environment.

9. The method according to claim 1, wherein, Transmitting a first set of optical signals into the surrounding environment includes transmitting the first set of optical signals into the surrounding environment using a first transmission power, wherein transmitting a second set of optical signals into the surrounding environment includes transmitting the second set of optical signals into the surrounding environment using a second transmission power, and wherein the second transmission power is less than the first transmission power.

10. The method according to claim 9, wherein, The second transmission power is less than 25% of the first transmission power.

11. The method according to claim 1, wherein, The dataset can be used to generate a first point cloud and a second point cloud, wherein the first point cloud includes data related to a first set of detected reflected light signals, and wherein the second point cloud includes data related to a second set of detected reflected light signals.

12. The method according to claim 1, wherein, Transmitting a second set of light signals into the surrounding environment involves interleaving the second set of light signals with each other in time.

13. The method according to claim 12, wherein, The threshold difference is between 0.1m and 5.0m.

14. The method according to claim 1, wherein, The duration of the first listening window is between 2.0µs and 3.0µs, and the duration of the second listening window is between 0.3µs and 0.5µs.

15. The method of claim 1, further comprising determining which light emitters in the first group of light emitters are included in the second group of light emitters based on the degree of contamination of one or more optics of the lidar device.

16. The method according to claim 15, wherein, The degree of contamination is determined based on previous measurements using lidar devices or different sensors.

17. The method according to claim 15, wherein, The level of contamination is determined based on the ambient weather conditions near the lidar device.

18. The method according to claim 1, wherein, The dataset includes multiple points associated with each detected reflected light signal in the second set of detected reflected light signals, wherein each of the multiple points includes a target distance, and wherein each target distance has an associated confidence level, the associated confidence level being determined based on the detected reflected light signals in the second set of detected reflected light signals and the corresponding first detected reflected light signals in the first set of detected reflected light signals.

19. A lidar (light detection and ranging) device, comprising: The first set of optical transmitters is configured to transmit a first set of optical signals into the surrounding environment, wherein the first set of optical signals corresponds to a first angular resolution relative to the surrounding environment; The first set of light detectors is configured to detect a first set of reflected light signals from the surrounding environment during a first listening window, wherein the first set of reflected light signals corresponds to the reflection of the first set of light signals from objects in the surrounding environment; The second set of optical transmitters is configured to transmit a second set of optical signals into the surrounding environment, wherein the second set of optical transmitters of the lidar device represents a subset of the first set of optical transmitters of the lidar device, wherein the second set of optical signals corresponds to a second angular resolution relative to the surrounding environment, and wherein the second angular resolution is lower than the first angular resolution; A second set of photodetectors is configured to detect a second set of reflected light signals from the surrounding environment during a second listening window, wherein the second set of photodetectors of the lidar device represents a subset of the first set of photodetectors of the lidar device, wherein the second set of reflected light signals corresponds to reflections of the second set of light signals from objects in the surrounding environment, and wherein the duration of the second listening window is shorter than the duration of the first listening window; and The controller is configured to synthesize a dataset that can be used to generate one or more point clouds, wherein the dataset is based on a first set of detected reflected light signals and a second set of detected reflected light signals. The synthetic dataset includes each detected reflected light signal from the second set of detected reflected light signals: The distance to the second target is determined based on the detected reflected light signals. The first target distance is determined based on the corresponding detected reflected light signal in the first group of detected reflected light signals, wherein the corresponding detected reflected light signal in the first group of detected reflected light signals is detected by the same photodetector in the lidar device; Determine the difference between the distance to the second target and the distance to the first target; and If the difference is less than a threshold difference, then the second target distance or the first target distance is included in the dataset.

20. A system for generating one or more point clouds, comprising: Light detection and ranging (lidar) devices, including: The first set of optical transmitters is configured to transmit a first set of optical signals into the surrounding environment, wherein the first set of optical signals corresponds to a first angular resolution relative to the surrounding environment; The first set of light detectors is configured to detect a first set of reflected light signals from the surrounding environment during a first listening window, wherein the first set of reflected light signals corresponds to the reflection of the first set of light signals from objects in the surrounding environment; The second set of optical transmitters is configured to transmit a second set of optical signals into the surrounding environment, wherein the second set of optical transmitters of the lidar device represents a subset of the first set of optical transmitters of the lidar device, wherein the second set of optical signals corresponds to a second angular resolution relative to the surrounding environment, and wherein the second angular resolution is lower than the first angular resolution; A second set of photodetectors is configured to detect a second set of reflected light signals from the surrounding environment during a second listening window, wherein the second set of photodetectors of the lidar device represents a subset of the first set of photodetectors of the lidar device, wherein the second set of reflected light signals corresponds to reflections of the second set of light signals from objects in the surrounding environment, and wherein the duration of the second listening window is shorter than the duration of the first listening window; and A lidar controller is configured to synthesize a dataset that can be used to generate one or more point clouds, wherein the dataset is based on a first set of detected reflected light signals and a second set of detected reflected light signals; and The system controller is configured as follows: Receive dataset from lidar controller; and Generate one or more point clouds based on the dataset. The synthetic dataset includes each detected reflected light signal from the second set of detected reflected light signals: The distance to the second target is determined based on the detected reflected light signals. The first target distance is determined based on the corresponding detected reflected light signal in the first group of detected reflected light signals, wherein the corresponding detected reflected light signal in the first group of detected reflected light signals is detected by the same photodetector in the lidar device; Determine the difference between the distance to the second target and the distance to the first target; and If the difference is less than a threshold difference, then the second target distance or the first target distance is included in the dataset.

Images