Lidar based communication

Abstract

To describe systems and methods for performing operations based on LIDAR communications.SOLUTION: An example device may include one or more processors and a memory coupled to the one or more processors. The memory includes instructions that, when executed by the one or more processors, cause the device to receive data associated with a modulated optical signal emitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to a vehicle and the device, generate a rendering of an environment of the vehicle based on information from one or more LIDAR devices coupled to the vehicle, and update the rendering based on the received data. Updating the rendering includes updating an object rendering of an object in the environment of the vehicle. The instructions further cause the device to provide the updated rendering for display on a display coupled to the vehicle.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Patent Application No. 16 / 502,008, filed July 2, 2019, the contents of which are incorporated herein by reference in their entirety.

[0002] The present disclosure relates generally to LIDAR devices, and more particularly to using LIDAR as a communication channel in autonomous driving applications.

[0003] 2. Description of Related Art Light detection and ranging (LIDAR) devices may be used to detect objects in an environment. For example, a vehicle may include or be coupled to one or more LIDAR devices that detect vehicles, pedestrians, traffic lights, obstacles, etc. The LIDAR device emits light at a particular frequency (e.g., 800-1000 nm or 1550 nm), and the LIDAR device receives reflections of the emitted light. The LIDAR device then determines the time of flight (ToF) of the light as it scans the environment to estimate the distances of multiple reflective surfaces. The estimated distances may be used to generate a point cloud representative of the environment or otherwise used to render the environment or to assist in the operation of the vehicle.

[0004] For example, a vehicle can be configured to operate in an autonomous mode in which the vehicle navigates an environment with little or no input from the driver. An autonomous vehicle can include one or more LIDAR devices to determine the distance of objects in the environment, which can be used to control navigation of the vehicle. In another example, a vehicle can include one or more LIDAR devices to assist the driver, from performing adaptive cruise control, providing traffic alerts at intersections, providing lane departure warnings, etc., to performing all safe and critical driving functions and monitoring road conditions in a fully autonomous system. Summary of the Invention

[0005] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0006] Innovative aspects of the subject matter described in this disclosure may be implemented in an apparatus coupled to one or more LIDAR devices. In some implementations, an example apparatus includes one or more processors and a memory coupled to the one or more processors. The memory includes instructions that, when executed by the one or more processors, cause the apparatus to receive data associated with a modulated optical signal emitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to the vehicle, generate a rendering of an environment of the vehicle based on information from the one or more LIDAR devices coupled to the vehicle, and update the rendering based on the received data. Updating the rendering includes updating an object rendering of an object in the environment of the vehicle. The instructions further cause the apparatus to provide the updated rendering for display on a display coupled to the vehicle.

[0007] Innovative aspects of the subject matter described in this disclosure may be implemented in a computer-readable medium that stores instructions that, when executed by one or more processors of the device, cause the device to transmit data relating to modulated optical signals emitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to the vehicle. The instructions cause the device to receive data from one or more LIDAR devices coupled to the vehicle, generate a rendering of the vehicle's environment based on information from one or more LIDAR devices coupled to the vehicle, and update the rendering based on the received data. Updating the rendering includes updating object renderings of objects in the vehicle's environment. The instructions further cause the device to provide the updated rendering for display on a display coupled to the vehicle.

[0008] Innovative aspects of the subject matter described in this disclosure may be implemented as a method. An example method includes receiving, by the device, data associated with a modulated optical signal emitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to the vehicle and the device, generating a rendering of the vehicle's environment based on information from one or more LIDAR devices coupled to the vehicle, and updating the rendering based on the received data. Updating the rendering includes updating object renderings of objects in the vehicle's environment. The method includes providing the updated rendering for display on a display coupled to the vehicle.

[0009] Another example apparatus may include one or more processors and a memory coupled to the one or more processors. The memory includes instructions that, when executed by the one or more processors, cause the apparatus to receive data relating to an infrastructure (such as a toll booth, a road work area, a tunnel entrance, etc.) and a modulated optical signal received by a receiver of a second LIDAR device coupled to the apparatus. The apparatus may generate a rendering of the infrastructure's environment based on information from the one or more LIDAR devices coupled to the infrastructure. The apparatus may also update the rendering based on the received data. Updating the rendering may include updating an object rendering of an object in the infrastructure's environment. The apparatus may further provide the updated rendering for display. For example, the rendering may be displayed to a toll booth attendant, a central traffic authority accountant, a construction worker foreman at a construction site, etc.

[0010] The data may include a representation of an object in the environment, and updating the object rendering may include highlighting the displayed object rendering, adjusting a texture of the object rendering, including a representative image of the object in the rendering of the environment, and / or adjusting a dimension of the object rendering. Displaying the updated rendering may include informing the viewer that the object in the infrastructure environment is an emergency vehicle.

[0011] The device may also determine guidance behavior for one or more vehicles within the infrastructure environment based on the received data, and the device may provide coordinated guidance behavior to the one or more vehicles. The modulated optical signal may be received by a second LIDAR device operating in a communication mode, and objects within the infrastructure environment and line of sight of the second LIDAR device may be detected by the second LIDAR device operating in a detection mode.

[0012] The device may further receive second data associated with the second modulated optical signal received by the second LIDAR device and may update one or more entries in the local database based on the received second data. In some implementations, the device may transmit data regarding the updates to the other LIDAR devices to update the databases local to the other LIDAR devices.

[0013] The details of one or more implementations of the subject matter described in this disclosure are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages will become apparent from the description, drawings, and claims. Please note that the relative dimensions of the following figures may not be drawn to scale. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram of an example LIDAR device. [Figure 2A] 1 illustrates an example LIDAR device in an environment. [Figure 2B] 1 is an example timing diagram of waveforms corresponding to emitted and received light pulses of an example LIDAR device. [Diagram 3] 1 illustrates an example packet format for packets received from a LIDAR transmission based on an example communication protocol. [Figure 4] 1 illustrates an environment including an example parking space for an autonomous fleet vehicle to return to when not in use. [Diagram 5]1 illustrates an example environment for communicating the presence of an ambulance to other vehicles in the environment. [Figure 6A] 1 illustrates an example environment for a vehicle. [Figure 6B] 1 illustrates an example rendering of a vehicle's environment as sensed by a LIDAR device. [Figure 6C] 1 illustrates an example environment for vehicles, including emergency vehicles. [Figure 6D] 6D shows an example rendering of the environment of the vehicle of FIG. 6C as sensed by a LIDAR device. [Figure 7] 1 shows a flowchart depicting example operations for adjusting the rendering of an environment of a LIDAR device. [Figure 8] 6B shows an example adjusted rendering of the environment in FIG. 6A. [Figure 9A] 6B illustrates an example rendering of the environment of FIG. 6A where two vehicles in the environment belong to the same fleet. [Figure 9B] 6D illustrates an example rendering of the environment of FIG. 6C, where the environment includes an emergency vehicle. [Figure 10] 1 shows a flowchart depicting example operations for performing one or more actions based on data received in a transmission from a LIDAR transmitter.

[0015] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Efforts have been made to couple vehicles and infrastructure via cellular communications. For example, some vehicles include cellular modems that communicate with other vehicles or devices over the 5 GHz radio spectrum. In another example, a vehicle may be equipped with Dedicated Short Range Communications (DSRC) equipment that communicates with other vehicles or devices over the 5.9 GHz radio spectrum. One problem with cellular and wireless technologies for communication is that the vehicle may require a cellular connection or may require other vehicles or infrastructure to be equipped with consistent technology for communication. For example, some rural or saturated urban environments may not include an available cellular connection and the vehicle may not be able to communicate with other vehicles via a cellular modem. Another problem with cellular technologies for communication is the inherent delays associated with the communication. For example, cellular communication between vehicles may require communication through one or more base stations of the cellular network, which may delay the communication. Additionally, the wavelength of the wireless signals for communication limits the communication bandwidth. Additionally, cellular and wireless signals are typically transmitted in all directions, and transmitting multiple omnidirectional signals over the ground can quickly saturate the environment.

[0017] Many vehicles and infrastructure (e.g., toll booths, traffic lights, charging stations, etc.) may include or be coupled to one or more LIDAR devices. Furthermore, as autonomous vehicles become more common, equipped LIDAR devices are also becoming more common. In some embodiments, a LIDAR device may be configured to communicate with other LIDAR devices in addition to performing ToF measurements (to detect surfaces in the environment). Because the frequency of the emitted optical signals is higher than radio signals, the communication bandwidth for communication between LIDAR devices may be higher than cellular communication. Furthermore, communication between LIDAR devices may require the use of a cellular network. No network or other infrastructure is required. Additionally, optical signals (such as those with wavelengths close to 1000 nm) may be emitted in a focused-divergent pattern to prevent saturation and interference from multiple optical signals transmitted simultaneously over terrestrial waves.

[0018] Implementations of the subject matter described herein may enable a LIDAR device to communicate with another LIDAR device (referred to herein as "LIDAR communication"). LIDAR communication may be used for vehicle-to-vehicle (V2V) or vehicle-to-infrastructure (V2X) communication, and LIDAR communication may occur between any compatible vehicles and / or infrastructure (such as within a fleet of vehicles or between unrelated vehicles equipped with configured LIDAR devices). LIDAR communication may be used in a variety of situations and use cases, as described herein.

[0019] The following description is directed to a particular implementation for purposes of illustrating the innovative aspects of the present disclosure. However, one skilled in the art will readily recognize that the teachings herein can be applied in many different ways. The described implementations may be implemented in any device, system, or vehicle that includes or is coupled to one or more LIDAR devices. In some implementations, an "apparatus" for performing the operations described herein may refer to a vehicle and a controller or system coupled to one or more LIDAR devices, a vehicle that includes a controller or system and is coupled to one or more LIDAR devices, a controller or system coupled to infrastructure or another non-vehicle system, or another suitable implementation. Similarly, a "vehicle" may refer to a controller or system coupled to a vehicle, a vehicle separate from a controller or system coupled to a vehicle, a combination of a vehicle coupled to a controller or system, or another suitable implementation.

[0020] In the following description, numerous specific details are set forth, such as examples of specific components, systems, and processes, to provide a thorough understanding of the present disclosure. Also, for purposes of the following description and explanation, specific nomenclature and / or details are set forth to provide a thorough understanding of the exemplary embodiments. However, it will be apparent to one skilled in the art that these specific details may not be necessary to practice the exemplary embodiments. In other instances, well-known circuits, systems, and devices are shown in block diagram form to avoid obscuring the present disclosure. Any signals provided over the various buses described herein may be time multiplexed with other signals and provided over one or more common buses. Furthermore, the interconnections between components or software blocks may be depicted as buses or single signal lines. Each of the buses may alternatively be a single signal line, and each of the single signal lines may alternatively be a bus, and a single line or bus may represent any one or more of a myriad of physical or logical mechanisms for communication between components. Furthermore, the term "coupled" as used herein means directly coupled to or coupled through one or more intervening components or devices.

[0021] FIG. 1 is a block diagram of an example light detection and ranging (LIDAR) device 100. The LIDAR device 100 may be used to detect the surface of an object in an environment by emitting a pulse of light that illuminates the surface of the object and by detecting the light pulse reflected from the surface of the object. The LIDAR device 100 may determine the distance to the object based on the time delay between emitting the light pulse and receiving the corresponding light pulse reflected from the selected object. This time delay, sometimes referred to as the ToF of the light pulse, may be multiplied by the speed of light to determine the distance between the LIDAR device 100 and the object. Multiple pulses may be used to determine distance information for several points associated with the object in the environment. These points may be used to generate a point cloud or otherwise determine the position, size, shape, attitude, and motion of the object. In some implementations, information from the LIDAR device 100 may be used to control, for example, an autonomous vehicle, so that the autonomous vehicle can navigate the environment to reach a destination while avoiding obstacles. The LIDAR device 100 may also be used to measure distances for driver assistance operations.

[0022] The LIDAR device 100 is shown to include a transmitter 110, a receiver 120, and a LIDAR controller 130. The transmitter 110 may include a transmit controller 111, one or more light emitting elements 112, and a transmit aperture 113. The light emitting elements 112 may emit one or more light pulses 125 that can be used to detect objects in the surrounding environment. The light emitting elements 112 may include any number of suitable light sources, such as, but not limited to, laser diodes, light emitting diodes (LEDs), vertical cavity surface emitting lasers (VCSELs), organic light emitting diodes (OLEDs), polymer light emitting diodes (PLEDs), light emitting polymers (LEPs), liquid crystal displays (LCDs), microelectromechanical systems (MEMS), or any other device configured to selectively transmit or emit light pulses 125 of prescribed wavelengths. The light source wavelengths may include, for example, the ultraviolet, visible, and / or infrared portions of the electromagnetic spectrum. In some embodiments, the light emitting elements 112 may be disposed on one or more substrates (e.g., a printed circuit board (PCB), a flexible PCB, etc.). Although the light emitting elements 112 are described herein as emitting light pulses 115, one of ordinary skill in the art will readily appreciate that the light emitting elements 112 may transmit or emit light signals, light beams, photons, etc. Thus, the terms light pulses, light signals, light beams, and photons may be used interchangeably herein.

[0023] The transmit aperture 113 may include any suitable components (e.g., mirrors, lenses, gratings, exit apertures, etc.) coupled to the light emitting elements 112 and capable of focusing, directing, and / or conditioning the light pulses 115 for emission into the surrounding environment. In some implementations, the transmit aperture 113 may be configured to direct the light pulses 115 in one or more specified directions relative to the LIDAR device 100. The specified directions may span a range of directions, such that, for example, a distance between the LIDAR device 100 and a number of objects (e.g., vehicles, people, roads, traffic lights, traffic signs, obstacles, etc.) may be determined based on reflections of the light pulses 115 caused by the objects.

[0024] The transmit controller 111 may control the operation of the light emitting elements 112 and the transmit aperture 113 and may adjust multiple parameters or settings of the light emitting elements 112 and the transmit aperture 113, or both. In some implementations, the transmit controller 111 may be responsive to one or more control signals provided by the LIDAR controller 130. For example, the transmit controller 111 may adjust the width, timing, frequency, and / or amplitude (intensity) of the light pulses 115 emitted by the light emitting elements 112 based on the one or more control signals. In other implementations, the transmit controller 111 may be omitted or may be included within the LIDAR controller 130.

[0025] The receiver 120 may include a number of photodetectors 121, a detector circuit 122, and an analog-to-digital converter (ADC) 123. The photodetectors 121 may receive light pulses 125 (e.g., photons) from the surrounding environment. In some implementations, the received light pulses 125 may include components of the emitted light pulses 115 reflected from one or more objects in the surrounding environment. The photodetectors 121 may be configured to convert the received light pulses 125 into a photodetector signal (e.g., an analog current signal) indicative of an intensity level of the received light pulses 125. The photodetectors 121 may be any suitable component or element capable of receiving or detecting light, including, for example, a photodiode (such as an avalanche photodiode), a silicon photomultiplier tube (SiPM), a phototransistor, a camera (such as a CMOS sensor), an active pixel sensor (APS), a charge-coupled device (CCD), a cryogenic detector, and the like. In some implementations, the photodetectors 121 may, for example, generate a current flowing through each photodiode in response to receiving a light pulse such that the amount of current passing through each photodiode is proportional to the intensity of the light pulse received by that photodiode. It is a reverse-biased photodiode that produces a current.

[0026] Although not shown for simplicity, receiver 120 may include optics for filtering the wavelengths of the received light such that photodetector 121 receives primarily light corresponding to the wavelengths of the optical pulses 115 emitted by transmitter 110 (and receives weaker light corresponding to other wavelengths). For example, receiver 120 may include a bandpass filter to filter out optical signals outside of a wavelength range centered around the fundamental wavelength of the optical pulses emitted by transmitter 110.

[0027] The detector circuit 122 may use any suitable technique to sample the photodetector signal provided by the photodetector 121 to determine the intensity level of the received light pulse 125. In some implementations, the detector circuit 122 may sample the photodetector signal at multiple intervals or sampling times. In other implementations, the detector circuit 122 may continuously sample the photodetector signal. The detector circuit 122 may provide the determined intensity level to the ADC 123, for example, as an analog signal having a magnitude (e.g., a voltage magnitude or a current magnitude) indicative of the light information contained in the photodetector signal. In some aspects, the detector circuit 122 may amplify and / or filter the photodetector signal.

[0028] The ADC 123 may receive an analog signal indicative of the intensity level of the received light pulse 125 from the detector circuit 122 and convert the analog signal into digital data that can be processed by the LIDAR controller 130. The ADC 123 may be any suitable ADC, such as, but not limited to, a flash ADC, a successive approximation register (SAR) ADC, or a delta-sigma ADC. In some implementations, each photodetector 121 may correspond to a respective ADC. In other implementations, multiple photodetectors 121 may correspond to a single ADC (e.g., to reduce size, cost, and / or power consumption of the LIDAR device 100). In some other implementations, the ADC 123 may be omitted. Each of the photodetectors 121 (and corresponding ADC 123) may be associated with a particular light emitting element 112. In this manner, multiple ToFs may be measured, and thus multiple distances may be determined, during one scan of the LIDAR device 100.

[0029] The LIDAR controller 130 may include a processor 131, a memory 132, and a digital signal processor (DSP) 133. The DSP 133 may process digital data provided by one or more ADCs 123 to determine information about the light pulses received by any number of light detectors 121. In some implementations, the intensity and / or arrival time of the light pulses may be used to determine the size, shape, and location of multiple objects detected in the surrounding environment. For example, if the time of departure of the light pulse is known, the DSP 133 may use the arrival time to determine the ToF. In another example, if the intensity of the emitted light pulse is known, the DSP 133 may use the measured intensity to determine the energy loss of the reflected light pulse. Objects relatively closer to the LIDAR device 100 may reflect the emitted light pulse 115 before objects relatively farther from the LIDAR device 100. Additionally, light reflected from objects relatively closer to the LIDAR device 100 may experience less pulse spread than light reflected from objects relatively farther from the LIDAR device 100 (assuming similar surface reflectivities between objects at various distances). Thus, in some implementations, the distance between the LIDAR device 100 and an object may be estimated based on the rising and falling edges of the received light pulse 125.

[0030] Processor 131 may be one or more any suitable processors capable of executing scripts or instructions of one or more software programs stored in LIDAR device 100 (e.g., in memory 132). In some implementations, processor 131 may provide at least a portion of a machine-readable medium on which program instructions or scripts may be stored. In some implementations, the processor 131 may be or include one or more microprocessors and memory. In other implementations, the processor 131 may be an application specific integrated circuit (ASIC). In some other implementations, the processor 131 may be or include one or more field programmable gate arrays (FPGAs) or programmable logic devices (PLDs).

[0031] The memory 132 may store information about the transmitter 110, the receiver 120, the surrounding environment, or any combination thereof. The memory 132 may also include a non-transitory computer-readable medium (e.g., one or more non-volatile memory elements such as EPROM, EEPROM, flash memory, hard drive, etc.) that may store a number of software (SW) modules, each including instructions that, when executed by the processor 131, cause the LIDAR device 100 to perform all or a portion of the operations described herein. In some other implementations, the LIDAR controller 130 may be instructed to perform one or more operations related to emitting or receiving light pulses by one or more processors external to the LIDAR device 100 (e.g., included in a processing system of a vehicle coupled to the LIDAR device 100). For example, the LIDAR device 100 may be coupled to a vehicle processing hub or another processing system in the vehicle (such as via a Controller Area Network (CAN) bus), and the processing system may instruct the LIDAR device 100 to perform one or more operations and may receive information from the LIDAR device 100 in response (such as obtaining ToF or intensity information measured by the LIDAR device 100 and used by the processing system to generate a point cloud or other type of depth map or rendering of the environment).

[0032] FIG. 2A illustrates an example LIDAR device in an environment. In the example of FIG. 2A, the LIDAR device 100 is placed in an environment 200 that includes an automobile 201 and a canopy 202. In a simplified example, the LIDAR device 100 is shown to include three light emitters 112A-112C that each emit a light pulse 115A-115C into the environment 200. The LIDAR device 100 is also shown to include three light detectors 121A-121C that receive light pulses 125A-125C, respectively, that are reflected from surfaces of objects in the environment 200. A first light pulse 115A illuminates a surface 205A of the canopy 202, and the first light detector 121A receives a corresponding reflected light pulse 125A. A second light pulse 115B illuminates a surface 205B of the automobile 201, and the second light detector 121B receives a corresponding reflected light pulse 125B. A third light pulse 115C illuminates another surface 205C of the vehicle 201, and a third photodetector 121C receives a corresponding reflected light pulse 125C. The LIDAR device 100 can use one or more characteristics (e.g., timing, amplitude, pulse width, etc.) of the received light pulses 125A-125C to determine the distance between the LIDAR device 100 and each surface 205A-205C in the environment 200.

[0033] 2B is an example timing diagram 210 of waveforms corresponding to emitted and received light pulses of an example LIDAR device. Transmit waveforms 215A-215C may represent the intensity levels of each of the emitted light pulses 115A-115C from the LIDAR device 100 of FIG. 2A, and receive waveforms 225A-225C may represent the intensity levels of each of the received light pulses 125A-125C by the LIDAR device 100 of FIG. 2A. The light pulses 115A-115C are emitted from the LIDAR device 100 at the same time t0 (or at least substantially simultaneously), and the reflected light pulses 125A-125C are 、 At different times t A ~t CThe transmitted waveforms 215A-215C include pulses 216A-216C, respectively, that represent the time t0 when the corresponding reflected light pulses 125A-125C are received by the LIDAR device 100. The received waveforms 225A-225C include pulses 216A-216C, respectively, that represent the time t0 when the corresponding reflected light pulses 125A-125C are received by the LIDAR device 100. A ~t C The pulses 216A-216C each represent a pulse A peak detection technique (e.g., determining the peak amplitude, determining the center, determining the average time between threshold crossings, etc.) is used to determine the reception time t A ~t C can be obtained from the pulses 226A to 226C. A ~~t C can be used to determine the respective distances between the LIDAR device 100 and the surfaces 205A-205C of the environment 200.

[0034] Many LIDAR devices are capable of adjusting one or more characteristics of the emitted light pulses. For example, the LIDAR device 100 (FIG. 1) may configure the transmitter 110 to adjust one or more of the timing of the light pulses of the emitted light 115, the frequency of the light pulses, or the intensity of the light pulses. In some examples, the LIDAR device 100 may dither the light pulses of the emitted light 115 to control the timing. Some LIDAR devices may be capable of adjusting the power provided to the light emitting elements 112 (such as LEDs) to control the intensity. Some LIDAR devices may be capable of adjusting a reference signal frequency (such as a light source) to adjust the frequency of the light pulses of the emitted light 115. The receiver 120 may also be configured to detect differences in the timing, frequency, or intensity of the light pulses of the received light 125. For example, the sensitivity of the photodiode array of the photodetector 121 may be sufficient to determine differences in intensity, and the sampling rate of the photodiode array and / or ADC may be sufficient to determine differences in the timing or frequency of the light pulses of the received light 125.

[0035] In some aspects, the LIDAR device 100 (such as the LIDAR controller 130) may be configured to encode information into the emitted light 115 via frequency adjustment, intensity adjustment, and / or time dithering of the emitted pulses, and the transmitter 110 may be configured to communicate such encoded information via the emitted light 115. For example, the LIDAR device 100 may receive a transmitted data signal, and the LIDAR device 100 may use the emitted light (sometimes referred to herein as an optical signal) as a carrier signal. In this manner, the LIDAR device 100 may modulate an optical signal to include the data signal, and the LIDAR device 100 may transmit the modulated optical signal. In the case of a first LIDAR device, the receiver 120 may be configured to receive a modulated optical signal transmitted by the transmitter 110 of the second LIDAR device. The modulated optical signal may be encoded with a data signal from the second LIDAR device, and the LIDAR controller 130 of the first LIDAR device may be configured to extract the data signal from the received modulated optical signal. For example, the LIDAR controller 130 may demodulate the received optical signal to generate the data signal. Such communication between LIDAR devices is point-to-point (and thus does not require a central network) and the frequency of the signals is higher than that of cellular communication, so that the throughput may be higher and the delay may be lower than that of conventional cellular communication. In this specification, the "modulated optical signal" refers to an optical signal (such as the light emitted by the LIDAR device 100) modulated to include a data signal.

[0036] In some implementations, the LIDAR device 100 may be configured to switch between a distance measurement mode (which may be referred to herein as a "detection mode") and a communication mode for transmitting a modulated optical signal including a data signal. For example, the LIDAR controller 130 may determine when the LIDAR device 100 transmits or receives information via the modulated optical signal and when the LIDAR device 100 operates to detect the surface of an object in an environment. For example, the LIDAR controller 130 may determine to place the LIDAR device 100 in a communication mode during a first portion of time and to place the LIDAR device 100 in a detection mode during a second portion of time. Thus, the LIDAR device 100 may switch between a communication mode and a detection mode. In some other implementations, the emitted light 115 may be used to communicate information to another receiver (such as by adjusting the frequency) and may also be used to detect the surface of an object (e.g., by sensing a difference in the intensity of the received light 125), and the LIDAR device 100 may be configured to perform both modes simultaneously. For example, the LIDAR device 100 may emit a modulated optical signal (including a data signal). The LIDAR device 100 may receive a reflection of the modulated optical signal and use the reflection to determine the depth of the object from the LIDAR device 100. Additionally, a second LIDAR device may receive the modulated optical signal and demodulate the optical signal to generate the included data signal from the LIDAR device 100.

[0037] An apparatus, such as a vehicle or infrastructure, may include or be coupled to one or more LIDAR devices (such as LIDAR device 100) configured to communicate data signals (e.g., provided by the vehicle or infrastructure) via emitted light 115. In this manner, the apparatus may communicate with the vehicle or infrastructure using the configured LIDAR device. In some embodiments, the LIDAR device 100 may be configured to use a communication protocol adopted by other vehicles and infrastructure that include the LIDAR device. The communication protocol may be ad-hoc or managed, and any suitable packetization of information may be used for the communication protocol. For example, LIDAR communication within a fleet of vehicles may be based on a specific protocol. In some implementations, a standardized protocol (or a protocol adopted by multiple parties) may more easily enable integration of LIDAR communication between vehicles and infrastructure. Such a protocol may include a defined packet format for transmitting and receiving information. Protocols from other communication mediums, such as cellular communication, Wi-Fi communication, digital subscriber line (DSL) communication, fiber optic communication, etc., may be utilized to create a protocol for communication between LIDAR devices.

[0038] FIG. 3 illustrates an example packet format of a number of received packets 300 based on an example communication protocol. In some implementations, the LIDAR device 100 may include a buffer or other suitable memory (such as memory 132) for ordering one or more received packets 300 (such as packets 1-6). The LIDAR device 100 may process the buffered packets for the vehicle or infrastructure to perform one or more operations. In another example, the buffered packets may be provided to a processing system of the device for processing. As shown, the format of packets 1-6 includes fields 302-308 including a transmitter ID 302, a location 304, a payload type 306, and a payload 308. Any suitable packet format may be used, therefore additional fields, fewer fields, different fields, or different organization of fields may be included in the packets. For example, packets 1-6 may include a cyclic redundancy check (CRC) field after the payload 308 to correct errors in the received packets.

[0039] The transmitter ID 302 may indicate the device that is transmitting the packet. In some implementations, each vehicle may include a unique identifier to identify that particular vehicle. For example, if a fleet of 200 autonomous vehicles communicate with each other, each vehicle may include an identifier (e.g., “vehicle_1” through “vehicle_200”) that is different from the other vehicles in the fleet. If a LIDAR device equipped on a vehicle associated with “Vehicle_100” transmits packets 1 and 2 to the LIDAR device 100, the transmitter ID 302 of the packet may include the unique identifier “Vehicle_100”. Similarly, if a LIDAR device equipped on a vehicle associated with “Vehicle_102” transmits packet 3 to the LIDAR device 100, the transmitter ID 302 of the packet may include the unique identifier “Vehicle_102”. Some vehicles may not include a unique identifier or may not otherwise be identified, but the vehicles may still communicate the packet to the LIDAR device 100. In one example, the Sender ID 302 may include a Sender ID value unique to a previously unidentified vehicle or equipment (shown as "Unidentified_Vehicle" for packet 4). The Sender ID value may be filled with zeros, a null value field, or otherwise appropriately populated to indicate that the vehicle or infrastructure associated with the packet does not have a unique identifier.

[0040] Infrastructure (such as a toll booth, a tunnel, an entrance gate for a high-occupancy vehicle (HOV) lane, etc.) may include or be coupled to one or more LIDAR devices to transmit and receive packets to and from LIDAR device 100. For example, packet 5 may be transmitted by a transmitter located at the entrance to a tunnel (which may be uniquely identified as "Tunnel_40"), and packet 6 may be transmitted by a transmitter located at a toll booth (which may be uniquely identified as "Toll Booth_30"). Any suitable vehicle or infrastructure may include a LIDAR device and may include a unique transmitter ID for LIDAR communications.

[0041] In some implementations, emergency vehicles, construction zones, and other devices associated with priority transmissions may include a transmitter ID to indicate that the packet is for priority transmission. When the LIDAR device 100 sequentially decodes packets, the LIDAR device 100 may process the transmitter ID 302 first and determine that the packet is a priority transmission. In this manner, the LIDAR device 100 may determine which packets are prioritized before completing processing of the remainder of the packet. Other suitable implementations of the transmitter ID 302 may be used, and the disclosure is not limited to the examples provided. For example, the transmitter ID may be configured to distinguish between fleets, types of vehicles (private cars, commercial trucks, school buses, etc.), and the like.

[0042] Location 304 may indicate the location of a vehicle or infrastructure relative to the transmitting device. For example, the vehicle or infrastructure may include a Global Positioning System (GPS) receiver to determine latitude and longitude. The latitude and longitude may then be provided in location 304, as shown in packets 1-6. Alternatively, location 304 may include location information related to the LIDAR device 100, as determined by another vehicle or infrastructure via its own LIDAR device. For example, the transmitting LIDAR device may determine the relative distance and location between the transmitting vehicle and the receiving vehicle or infrastructure, and the distance and location of the device may be transmitted to location 304. In some implementations, if location information cannot be transmitted (e.g., the vehicle does not include a GPS receiver), location 304 may be filled with zeros or otherwise solid to indicate that no location is provided.

[0043] The payload type 306 may indicate the type of information provided in the payload 308 of the packet. In the example of packets 1-6, the payload type 306 of packet 1 may indicate "HEARTBEAT" or "BEAT", which may be a signal transmitted periodically by "Vehicle_100". In some implementations, the "HEARTBEAT" or "BEAT" may indicate that the packet is similar to a beacon, and basic information may be provided in the payload 308. In some examples providing basic information of the vehicle or infrastructure, Payload1 may include the dither rate or other operating parameters of the transmitter, and / or trajectory information or other status information of the vehicle. Alternatively, no information may be provided in the payload 308 of such a packet (e.g., the payload is filled with zeros and the packet only provides location information via position 304 and notifies other devices of the presence of the transmitter).

[0044] The payload type 306 for Packet 2 is "RENDER_DESCRIPTION". As explained in the use case below, the LIDAR device 100 can measure surfaces within the line of sight (LoS) of the LIDAR device 100, but surfaces that are not within the LoS of the LIDAR device 100 may not be sensed and detected. In this way, a rendering of the environment based on measurements from the LIDAR device 100 may not include portions of the vehicle that are not within the field of view of the LIDAR device 100. "RENDER_DESCRIPTION" may indicate that basic information regarding the rendering of a vehicle or infrastructure may be provided in the payload 308. For example, Payload2 may provide information regarding the "Vehicle_100" being rendered. The dimensions may include the dimensions of one or more shapes associated with the vehicle. In one implementation, the dimensions may include the orientation and dimensions of a geometric shape (such as a rectangle for a two-dimensional rendering or a rectangular prism for a three-dimensional rendering). Other suitable rendering information may include texture, contour, or other features of the rendering of the object shown by the payload 308. For example, an emergency vehicle or road work zone may be highlighted in the rendering displayed by the vehicle to the driver and / or passengers. Thus, the payload 308 may show a texture to highlight the portion of the rendering related to the emergency vehicle or work zone. In another example implementation, similar textures may be used for vehicles in the same fleet. In this way, the driver and / or passengers may easily identify the fleet vehicles in the displayed rendering. However, any suitable rendering description may be used, and the disclosure is not limited to the examples presented.

[0045] In addition to or as an alternative to "RENDER_DESCRIPTION", the payload type 306 may indicate "RENDER_IDENTIFICATION", such as packet 4. In some implementations, a list of standard vehicles (or infrastructure) and their associated rendering information may be stored in a database (such as a look-up table or another suitable data set organized for access). For example, if the vehicle sending the packet is a 1994 GeoMetro, then the payload 308 (such as Payload4) may indicate "1994 GeoMetro". The database may be used in determining the rendering details of a particular vehicle (such as the dimensions and texture of the rendering), and existing renderings may be extended with additional rendering details. The database may also include information about a particular vehicle, such as length, acceleration capabilities, etc. The term "rendering" herein may mean a line drawing, a point cloud, a depth map, an image, a texture, shading, or other information that may be visualized or displayed.

[0046] The payload type of Packet 5 is “GATED_ENTRY”, which may indicate that Payload 5 indicates the type of vehicles that are permitted to enter “Tunnel_40”. For example, “Tunnel_40” may be restricted to multi-occupancy vehicles or autonomous vehicles, which may be indicated in Packet 5. Thus, a vehicle including the LIDAR device 100 may determine whether it is permitted to enter “Tunnel_40” based on the information in Payload 5.

[0047] The payload type of Packet 6 is “FEE_CHARGE”, which may indicate that Payload 6 indicates the fee charged by “Toll Booth_30”. For example, if the bridge toll is $5, Payload 6 may indicate that $5 will be automatically collected from the driver or fleet account when the vehicle passes “Toll Booth_30”. In some implementations, Payload 308 may include a combination of the fee to be charged and restrictions on vehicle entry. For example, a transmitter for a parking garage space may indicate the parking fee and the vehicles allowed in the space (e.g., designated handicapped parking, reserved for electric vehicle space, etc.). While the vehicle is parking in the space, the transmitter may update the parking fee based on the rate so that the driver may be aware of the current parking fee.

[0048] The payload type of Packet 3 is “VEHICLE_LOCATOR”, which may indicate that Payload 3 indicates the vehicle being found. For example, if “Vehicle_005” is offline, a vehicle in the fleet may send a packet attempting to locate “Vehicle_005”. Thus, Payload 3 may include a “Vehicle_005” identifier to indicate which vehicle should be located. In some implementations, the LIDAR device 100 may send multiple packets of the received packet. In some implementations, the LIDAR device 100 may persist the transmitter ID. Thus, the LIDAR device 100 may be configured to search for the persisted transmitter ID in the received packet to determine whether "Vehicle_005" has communicated with the LIDAR device 100. The LIDAR device 100 may then indicate this to "Vehicle_102." For example, the LIDAR device 100 may transmit the location received in the packet from "Vehicle_005" to "Vehicle_102." In some other implementations, the LIDAR device 100 may be configured to propagate a message to other vehicles or infrastructure indicating that "Vehicle_005" should be located. In this way, if "Vehicle_005" receives a propagated message from another vehicle or infrastructure, "Vehicle_005" may decide to communicate its location to the fleet.

[0049] There may be other suitable payload types and payloads (such as in different use cases of LIDAR communication as described herein), and the disclosure is not limited to the examples provided. In some example implementations, a vehicle may include multiple sensors to gather information for packet generation. For example, the payload may include information regarding the number of occupants, the vehicle's operating mode, etc., and sensors such as pressure sensors, occupancy sensors, engine sensors to detect the operating mode, etc. may be used to gather such information.

[0050] A wide range of information may be provided and received via LIDAR communications, which may be applied to a variety of use cases as described herein.

[0051] Implementing LIDAR communication Vehicle Locator The vehicle may operate in an area where there is no cellular network for communication. For example, if the vehicle traverses a rural or sparsely populated area, the vehicle may not have a stable cellular connection or may not otherwise be able to communicate with a base station. In another example, the vehicle's cellular modem may be inoperable. If the vehicle is part of a fleet of vehicles (such as a taxi or rental car fleet), the dispatcher may not be able to determine the location of the vehicle and the dispatcher may wish to locate the vehicle.

[0052] In some implementations, searching for a missing, stranded, or offline vehicle may be a proactive measure, where a vehicle or infrastructure that finds the offline vehicle reports that the offline vehicle has been found to a dispatcher. In some other implementations, searching for an offline vehicle may be a reactive measure, where a vehicle records the last known location of the offline vehicle. In this way, when the vehicle syncs with a dispatcher or the rest of the fleet (such as when the vehicle is charging or otherwise not operating), the vehicle reports its results.

[0053] Other vehicles in the fleet may be instructed to transmit vehicle locator messages via LIDAR communications. For example, each of the fleet vehicles may be instructed to periodically transmit packets with a payload type of "Vehicle_Locator," thereby identifying offline vehicles in the packet payload. The dispatcher may instruct fleet-owned infrastructure, such as electric vehicle charging stations and parking spaces that include or are coupled to LIDAR devices, to transmit vehicle locator messages. As noted above, other vehicles may also propagate such messages received to increase the number of transmitters transmitting vehicle locator messages. In this manner, a vehicle that may be offline to its primary communication method (e.g., cellular) may receive a message from a passing vehicle or infrastructure, which may decide to communicate its location to the dispatcher via LIDAR communications. In another example, a vehicle or infrastructure that finds an offline vehicle may communicate the location of the offline vehicle to the dispatcher.

[0054] For example, if another fleet vehicle is in LIDAR communication with the offline vehicle, the offline vehicle may send a message via the LIDAR communication to the other fleet vehicle that its cellular modem is inoperative and to contact a dispatcher, for example, via the other vehicle's cellular modem. In this manner, the dispatcher may communicate with the offline vehicle via the other fleet vehicle.

[0055] In addition to or instead of dispatchers finding offline fleet vehicles, vehicle locator messages may be used in emergency situations such as stolen vehicles. The payload of a vehicle locator packet may include a description of the vehicle to be located, and such packets may be propagated to other vehicles and infrastructure to find the stolen vehicle. In some implementations, the vehicle may display a notification to the driver in response to receiving the packet. In some other implementations, the stolen vehicle may be automatically identified based on the dimensions or other characteristics of the stolen vehicle. For example, the detected surfaces of other vehicles using a LIDAR device may be used to identify vehicles of the same make and model based on similar dimensions or other characteristics of the stolen vehicle. In another example, a visible light camera may be used to capture an image of the license plate after detecting the missing vehicle, and the image may be analyzed to identify the missing vehicle based on the license plate number. The vehicle may report the location of the missing vehicle to a central office (e.g., via a cellular modem).

[0056] In some other implementations, the vehicle locator message may be initiated by a vehicle that is offline or disabled. For example, a fleet vehicle's cellular modem may become inoperable, causing the fleet vehicle to be unable to communicate with the fleet via a cellular network. If the dispatcher provides information about where the fleet vehicle should go via cellular communication, the fleet vehicle may not be able to determine where the dispatcher wants to send the vehicle. In the example of a taxi fleet of autonomous vehicles, the dispatcher may want to send a vehicle to pick up a customer at a particular address and take the customer to a particular destination. However, the vehicle is unable to receive instructions from the dispatcher. LIDAR communication may be used to provide messages from the vehicle (such as a vehicle with a disabled cellular modem) to the dispatcher. For example, the vehicle may send a message to the dispatcher via LIDAR communication to other fleet vehicles. The other fleet vehicles may then send the message to the dispatcher. In this way, the dispatcher is notified that the vehicle may not be able to receive messages from the dispatcher, and the dispatcher may update the processing request without including the vehicle in the fleet. The dispatcher may also provide instructions to the vehicle via LIDAR communications (such as via other fleet vehicles) to wait for service at a designated location, remove itself from fleet operation, or other appropriate operations.

[0057] In some implementations, the vehicle may have a home location away from the taxi fleet dispatcher. For example, if the vehicle has no place to go (e.g., due to a passenger communicated by the dispatcher), the vehicle may return to a designated location within the service area, such as a parking space for a particular vehicle within the service area. The space may be associated with a LIDAR device that can communicate with the vehicle (e.g., allowing the dispatcher to communicate with the vehicle). When the vehicle is fully operational and back in standard communication with the dispatcher, the designated location may be a waiting area for the vehicle until the vehicle is activated for a passenger. If the vehicle cannot communicate with the dispatcher and is not being used for a passenger, the vehicle may also return to a designated location (e.g., a parking space). References herein to offline vehicles may refer to vehicles that are stranded, missing, or other vehicles that are not operating or cannot be expected to operate within a default, typical, or standard mode of operation.

[0058] FIG. 4 illustrates an example parking space for an autonomous fleet vehicle 402 to return to when not in use. 4 illustrates an environment 400 including a parking space 410. An autonomous vehicle 402 includes one or more LIDAR devices, such as LIDAR device 404. A stand 408 may be located near the parking space 410. Examples of stands 408 include parking meters, charging stations, and taxi stands for pedestrian clients. In some implementations, the stand 408 includes a LIDAR device for communicating with the vehicle 402 via LIDAR communications 406. The stand 408 also includes a wired or wireless backhaul (such as a cellular modem or fiber optic connection). A vehicle 402 that cannot directly communicate with a dispatcher may communicate with the stand 408 via LIDAR communications, and the stand 408 may communicate with the dispatcher via the backhaul. Although the vehicle 402 is described as communicating with the stand 408 via LIDAR communications, in some other embodiments, additional or alternative communication systems may be used. Other systems may include wireless local area network systems (such as IEEE 802.11 based systems), Bluetooth® systems, cellular systems (such as 3G, 4G, 5G, etc.), visible light communication systems, near field communication (NFC) systems, and the like.

[0059] When communicating with the stand 408, the vehicle 402 may provide a vehicle locator packet to the stand 408. In some implementations, sending a vehicle locator packet may indicate to the stand 408 and the dispatcher that the vehicle 402 is unable to communicate directly with the dispatcher (such as via a cellular modem). In some examples, the dispatcher may remove the vehicle 402 from service, send maintenance personnel to the vehicle 402, tell the vehicle 402 where to go for maintenance, or communicate new passengers to be handled by the vehicle 402. In some implementations, the stand 408 may also communicate with the vehicles 402 passing by on the road. If the vehicle 402 is unable to communicate directly with the dispatcher, the stand 408 may update the dispatcher with the location of the vehicle 402.

[0060] Vehicle Summoning In addition to locating the vehicle, the LIDAR communication can be used to summon the vehicle. Returning to FIG. 4 , stand 408 may be a pedestrian taxi stand. A person can walk up to stand 408 and input a desired destination (e.g., via a smartphone app, a graphical user interface of stand 408, or another suitable interface with stand 408). Multiple automated vehicles may line up at a location away from stand 408 (e.g., a parking lot around the corner) and communicate with another LIDAR transmitter (e.g., one or more stands in the parking lot). Parking space 410 may be a location to summon the next vehicle in the queue, or parking space 410 may be a location where a user enters a vehicle and boards the requested ride.

[0061] The stand 408 can handle multiple ride requests simultaneously. As a result, the stand 408 can page multiple vehicles (such as by using one or more LIDAR transmitters in a staging area for lined-up vehicles). Each paged vehicle may be assigned a particular client or ride, and as the vehicle approaches the parking space 410, the vehicle may indicate its assigned ride to the stand 408 (via LIDAR communication 406). In this manner, the stand 408 may notify customers of rides assigned to vehicles in the parking space 410 to expedite boarding and coordinate rides for multiple customers from the same parking space 410.

[0062] notification LIDAR communication can be used to provide notifications to vehicles or drivers. For example, vehicles may be notified via LIDAR communication of approaching emergency vehicles, road hazards, school routes during school hours, speed limit changes, construction zones, etc. In the case of emergency vehicles, emergency broadcasts are communicated by the infrastructure and vehicles that have previously received the broadcast, and the emergency vehicle is notified via LIDAR communication. Their presence can be signalled to other vehicles.

[0063] 5 illustrates an example environment 500 for communicating the presence of an ambulance 502 to a vehicle 506. The vehicle 508 may detect the ambulance 502. For example, the ambulance 502 may broadcast an emergency signal indicating the presence of the ambulance 502. In another example, a LIDAR device or other sensor on the vehicle 508 may detect the ambulance 502, and the vehicle 508 may identify the ambulance 502 as an emergency vehicle.

[0064] In some examples other than the ambulance 502, the vehicle 508 may detect or be notified of a person in the environment that may cause other vehicles to change course. For example, the vehicle 508 may detect one or more bicyclists in a lane, children playing nearby the street, children or other pedestrians crossing the street (e.g., before or after work), etc. In other examples, the vehicle 508 may detect a stalled car, a minor traffic accident (a "minor crash"), a road closure, or other traffic situation that may cause other vehicles to change course or receive notification of such an accident. In such scenarios, the vehicle may slow down in the area and adjust its scan of the environment using one or more LIDAR devices to focus on a particular area, ensuring, for example, by changing lanes to provide more space for the detected person, situation, etc.

[0065] The vehicle 508 may not be within LIDAR communication range with the vehicle 506. For example, the LIDAR communication may be based on LoS, and the vehicle 506 may be blocked or otherwise obstructed by a building, preventing the vehicle 508 from directly communicating with the vehicle 506. In some implementations, the vehicle 508 may use one or more vehicles and / or infrastructure to perform LIDAR communication with the vehicle 506 (e.g., communicate "around the corner"). For example, the vehicle 508 may use LIDAR communication 510 with the vehicle 504 to indicate the presence of the ambulance 502. Additionally or alternatively, the vehicle 508 may communicate with the stand 514 via LIDAR communication 516 to indicate the presence of the ambulance 502.

[0066] While FIG. 5 illustrates direct LoS communication between the LIDAR devices (510, 512, 516, and 518), LoS communication between the LIDAR devices may include one or more reflections of the signal from a surface. For example, a focused beam of optical signals from the transmitting LIDAR device may be intentionally reflected off one or more building walls, signs, or other objects in the environment 500, and the reflections may be received by the receiving LIDAR device. If the beam is sufficiently focused, the transmission includes sufficient transmit power, and one or more surfaces have sufficient reflectivity, the reflections received by the receiving LIDAR device may be processed to determine a communication transmitted by the transmitting LIDAR device (such as an indication of the presence of the ambulance 502). In this manner, the LIDAR devices may communicate with each other without a direct LoS. In this specification, LoS may include a direct LoS or an indirect LoS (which may include one or more reflections).

[0067] Vehicle 504 or stand 514 may be configured to indicate the presence of ambulance 502 or other traffic conditions to other vehicles and infrastructure within LIDAR communication range. For example, vehicle 504 may indicate the presence of ambulance 502 using LIDAR communication 512. In another example, stand 514 may indicate the presence of ambulance 502 using LIDAR communication 518. In this example, indicating the presence of ambulance 502 may involve two hops to get from vehicle 508 to vehicle 506 (e.g., via LIDAR communications 510 and 512 or via LIDAR communications 516 and 518).

[0068] In some implementations, the indication of an emergency vehicle (or other notification) may be propagated a defined number of hops or a defined distance. In this way, notification of the presence of the emergency vehicle to other vehicles is limited to the area surrounding the emergency vehicle or traffic situation. In some examples, the vehicle Both 508 may also determine the trajectory, route, or other information of the ambulance 502 and indicate such information to the vehicle 504 or stand 514. The distance or number of hops for propagating the instructions may, for example, increase in the direction of the route (or, conversely, decrease in the opposite direction of the route). If the vehicle 506 is an autonomous vehicle, the vehicle 506 may proactively pull over, stop, change lanes, or change route in response to receiving an indication of the presence of the ambulance 502 or other traffic conditions. In some other implementations, the vehicle 506 may provide visual or audio notifications to the driver (and / or passengers) to indicate the presence of the ambulance 502 or traffic conditions. For example, a speaker or display on the vehicle may notify the driver and / or passengers of the presence of the ambulance 502 or traffic conditions.

[0069] Another example notification is regarding available parking spaces in a parking lot or garage. In some implementations, the entrance gate of the parking garage may communicate the available spaces to an entering vehicle via LIDAR communication. If the vehicle is an autonomous vehicle, the vehicle may be assigned a parking space and automatically proceed to park in the space. In this way, the parking garage or parking lot may efficiently manage the parking of entering and exiting vehicles. In some other implementations, the entrance gate may indicate available parking spaces to the vehicle, and the vehicle may notify the driver and / or passengers of the location of the available spaces. For example, the vehicle may display a map of the parking garage and indicate the location of the available spaces on the displayed map. The driver and / or passengers may be notified before the vehicle enters the parking garage (e.g., as the vehicle approaches the garage), and the driver and / or passengers may select a preferred space.

[0070] As noted above, further examples of notifications may be regarding traffic accidents or other obstructions to traffic. The notification may be propagated via LIDAR communications to a vehicle preceding the obstruction, which may determine an alternate route and / or notify a driver and / or passengers of a possible delay based on the received notification. Other suitable examples of notifications may also be transmitted via LIDAR communications, and the disclosure is not limited to the above examples.

[0071] Access area restrictions Another use case for LIDAR communication is to restrict access to defined areas. Some areas may be restricted to certain vehicles or vehicle types. For example, an ambulance entrance at a hospital may be restricted to ambulances. In another example, a handicapped parking space may be restricted to vehicles that include a handicapped tag. In a further example, an HOV lane may be restricted to vehicles that include, for example, three or more occupants. In another example, a portion of a city center may be restricted to autonomous vehicles, low or zero emission vehicles, taxis, or other specific types of vehicles. In a further example, a corporate parking garage may be restricted to employee vehicles or vehicles with security clearance.

[0072] LIDAR communication may be used to indicate restrictions for certain areas and to allow certain vehicles access to such areas. In some implementations, a LIDAR device in an area (such as an entrance gate, stand, or other infrastructure) may transmit the restrictions in that area to an approaching vehicle. The vehicle then communicates the necessary authorization (via its own LIDAR device) to the LIDAR device, indicating that the vehicle is authorized for the area. The vehicle may then enter the area (such as a gate opening, vehicles otherwise allowed to access the area) and be allowed to drive. In some other implementations, the vehicle may store the necessary credentials for the area. In this way, the vehicle may preferentially communicate its credentials to the LIDAR device in order to gain access. For example, when an ambulance approaches the ambulance entrance of a hospital, the ambulance may use the LIDAR device to identify itself as an ambulance authorized to enter. It may be identified as an express vehicle.

[0073] Access restrictions to an area may change over time. For example, HOV lanes may be restricted only during designated congestion times. In another example, city centers may be restricted to zero-emission vehicles if smog levels are above a threshold. In a further example, areas around accidents or where police or emergency responders are needed may be restricted to such police or emergency response vehicles. Because LIDAR communications do not require a centralized network, changes in access restrictions may be implemented and provided to vehicles more quickly than, for example, remotely adjusting and establishing access parameters for an area via other communications. In some examples, a vehicle may notify the driver and / or passengers that the vehicle is about to enter a restricted area. In this manner, the driver, passengers, or the vehicle itself may adjust their route to travel around the restricted area.

[0074] In another implementation, the area may include different tolls or fees based on time of day, day of week, how busy the area is, special events, etc. Such tolls or fees may be communicated to the vehicle using LIDAR communication before the vehicle enters the area. Then, based on the information, the driver and / or passengers may decide to enter the area (or the vehicle may automatically decide). For example, some roads have hot lanes, and the price to enter such lanes is based on the time of day and how busy the road is. The driver and / or passengers may be notified of the price to enter the hot lane and may then decide whether to enter. Alternatively, the vehicle may be configured to enter the hot lane based, for example, on whether the price is lower than a threshold amount or a ride priority. For example, if a pregnant couple is traveling to a hospital, a ride priority may be set to override the price constraint on the hot lane. There may be other suitable use cases for restrictions, and the disclosure is not limited to the examples provided.

[0075] Visualization or rendering of objects Another use case of LIDAR communication is to provide rendering or visualization information. As described with respect to FIG. 3, the packets may include “RENDER_DESCRIPTION” or “RENDER_IDENTIFICATION” information for rendering or visualizing a vehicle (or infrastructure such as stand 408 in FIG. 4). The rendering or visualization of objects in the vehicle's environment may be limited to the LoS of the LIDAR device connected to the vehicle. As a result, surfaces that are not in the LoS of the LIDAR device may not be detected and rendered.

[0076] 6A illustrates an example environment 600 of a vehicle 602. The environment 600 includes buildings 604 and 606 and vehicles 608 and 610. As illustrated, the view of the vehicle 608 from the LIDAR device 612 of the vehicle 602 includes the front and driver's side surfaces of the vehicle 608. The view of the vehicle 610 from the LIDAR device 612 is obstructed by the building 606. As a result, the view of the vehicle 610 includes only a portion of the front and driver's side surfaces. Furthermore, the view of the buildings 604 and 606 from the LIDAR device 612 are surfaces facing the vehicle 602. The LIDAR device 612 may collect information regarding the depth of surfaces within the field of view of the LIDAR device 612, and the vehicle 602 may visualize or render the environment 600 including the detected surfaces. As used herein, a vehicle that visualizes or renders an environment may include one or more of a vehicle or infrastructure that provides instructions to a display coupled to the vehicle or infrastructure (such as a tablet communicatively coupled to the vehicle's processing system, an in-dash entertainment system coupled to the vehicle's processing system, a display off the vehicle) to render at least a portion of the environment on the display, or a vehicle that renders at least a portion of the environment on an integrated display (such as via an integrated entertainment system in the center console of the vehicle). A vehicle or infrastructure may refer to a device or system coupled to the vehicle or infrastructure that performs an operation (such as generating renderings to be displayed and providing the renderings to a display coupled to the vehicle or infrastructure).

[0077] 6B illustrates an example rendering 650 of the environment 600 as sensed by the LIDAR device 612. As described above, the example rendering 650 may be displayed on a display integrated into the vehicle or coupled to a processing system of the vehicle. The rendering 650 includes renderings of a surface 654 corresponding to a building 604, a surface 656 corresponding to a building 606, a surface 658 corresponding to a vehicle 608, and a surface 660 corresponding to a vehicle 610. The example rendering 650 also includes a representation 652 of the vehicle 602 to provide perspective. For example, the representation may be an icon, stock image, block, or other stored representation of the vehicle used in the rendering.

[0078] The example rendering 650 is simplified to illustrate aspects of the disclosure, and the rendering 650 may include additional information or details. For example, road lines and markers may be reflective and therefore detected via LIDAR. In another example, additional features of buildings 604 and 606 and vehicles 608 and 610 within the LoS of the LIDAR device 612 may be detected via LIDAR. Although the example rendering 650 is shown as two-dimensional from a bird's-eye view, the rendering may be from any suitable orientation or any suitable perspective. For example, the rendering may be a three-dimensional point cloud as seen by the LIDAR device 612, a two-dimensional view of the environment as seen by the LIDAR device 612, or any other suitable rendering.

[0079] The example rendering 650 may be provided to the driver and / or passengers by the vehicle 602 via a vehicle display. For example, the example rendering 650 may be displayed on a display integrated into the vehicle, or the vehicle may provide instructions for rendering to a display coupled to the vehicle's processing system. As shown in the example rendering 650, depth information may not be determined for occluded surfaces of objects and buildings. For example, the rendering 650 does not include information for the passenger side and rear surfaces of the vehicle 608, does not include information for the passenger side, rear, and part of the driver side surfaces of the vehicle 610, and does not include information for the sides of the buildings 604 and 606 that are not facing the vehicle 602.

[0080] In some implementations, another LIDAR device may transmit rendering information of one or more objects in the environment 600 to the vehicle 602. For example, the LIDAR device 614 of the vehicle 608 may transmit information about the vehicle 608 for rendering, and the LIDAR device 616 of the vehicle 610 may transmit information about the vehicle 610 for rendering. In some implementations, dimensions of a shape for rendering the vehicle may be transmitted. For example, dimensions of one or more rectangles for rendering the vehicle 608 in the rendering 650 may be transmitted from the LIDAR device 614 to the LIDAR device 612. Dimensions of one or more rectangles for rendering the vehicle 610 in the rendering 650 may be transmitted from the LIDAR device 616 to the LIDAR device 612. Other features related to the rendering of the vehicle 608 and the vehicle 610 may also be transmitted. Exemplary features include textures such as color, shading, highlights, etc. for the vehicle rendering.

[0081] In some other implementations, the LIDAR devices 614 and 616 may transmit identification information for their respective vehicles. For example, the LIDAR device 614 may transmit information about the make and model of the vehicle 608, the vehicle identification number of the vehicle 608, the license plate number of the vehicle 608, whether the vehicle is part of a particular vehicle fleet (such as the same vehicle fleet as the vehicle 602), a unique identifier within the vehicle fleet, or other identification information.

[0082] The vehicle 602 (or a memory coupled to a display for displaying the rendering) may store rendering information for a particular vehicle, and the vehicle 602 may use the vehicle identification information to retrieve the stored rendering information for a particular vehicle. For example, the vehicle memory (or other memory) may store rendering information for multiple makes and models of vehicles (including vehicle 608 and vehicle 610), and such rendering information may be retrieved and used to enhance the rendering 650 for the environment 600. When the rendering 650 is displayed on a display coupled to the vehicle 602, the vehicle 602 may provide the rendering information retrieved from the vehicle memory, or the vehicle 602 may provide information to identify the rendering information in an external memory (such as providing information about the make and model to an entertainment system connected to the vehicle). Exemplary rendering information may include a stock image or drawing of the particular vehicle. In some examples, the rendering information may be used to replace or combine corresponding portions of the rendering 650. In some other examples, adjusting the rendering may include highlighting the object rendering. For example, the rendering may be updated to inform the driver or passengers of the condition of the emergency vehicle or the vehicle's environment.

[0083] FIG. 6C illustrates an example environment 670 for a vehicle 672. The example environment 670 includes buildings 684 and 686 and vehicles 674 and 676. The environment 670 also includes an emergency vehicle 678. FIG. 6D illustrates an example top-down rendering 690 of the environment 670 for the vehicle 672 of FIG. 6C. The rendering 690 includes a rendering of a surface 694 corresponding to the building 684, a surface 696 corresponding to the building 686, and a surface 698 corresponding to the vehicle 674. The example rendering 690 also includes a representation 692 of the vehicle 672 to provide a sense of perspective. The vehicle 676 and the emergency vehicle 678 may not be within the LoS of the LIDAR device coupled to the vehicle 672. As a result, the rendering 690 may not include a rendering of the vehicle 676 or a rendering of the emergency vehicle 678.

[0084] FIG. 7 shows a flow chart depicting an example operation 700 for adjusting the rendering of an environment, such as a vehicle, a LIDAR device, infrastructure, etc. The example operation 700 is described below with respect to vehicles 602 and 672 of FIGS. 6A and 6C for illustrative purposes only. Those skilled in the art will recognize that the example operation 700 may be performed by any suitable device (such as a controller coupled to a vehicle), infrastructure, and / or vehicle according to various implementations, and that the example operation 700 described herein may be performed with additional steps, fewer steps, steps in a different order, steps in parallel, or any combination thereof. As used herein, a vehicle 602 performing one or more steps may refer to a processing system coupled to the vehicle performing one or more steps, a processing system integrated into the vehicle 602 performing one or more steps, or other suitable embodiment for performing the described method. For example, a controller may be located separately from the vehicle or otherwise coupled to the vehicle. Such a controller may also be coupled to one or more LIDAR devices, to a display (for rendering or displaying other information), and / or to one or more other input / output components (such as a speaker, a keypad, etc.) Vehicle 602 performing one or more steps may, in some implementations, mean that a controller performs the one or more steps.

[0085] Beginning at 702, the vehicle 602 may receive data associated with a modulated optical signal emitted by a transmitter of a first LIDAR device. For example, a device may be coupled to the LIDAR device 612, and a modulated optical signal may be transmitted from the LIDAR device 614 or the LIDAR device 616 to the LIDAR device 612. The modulated optical signal includes an optical carrier signal modulated to include a data signal. The LIDAR device 612 or a device coupled to the LIDAR device 612 may extract the data signal and determine the data included in the data signal.

[0086] The vehicle 602 may also generate a rendering of the vehicle's environment based on information from one or more LIDAR devices coupled to the vehicle (704). For example, the vehicle 602 may include a processing system for receiving measurements from the LIDAR device 612 when the LIDAR device 612 is in detection mode. The measurements may be used to generate a rendering, such as a point cloud, a depth map, or other suitable representation of the environment 600. For example, the processing system may generate an example rendering 650 of the environment 600 (or an example rendering 690 of the environment 670) based on measurements received from the LIDAR device 612 or other LIDAR devices coupled to the vehicle, where the rendering may include a representation of surfaces detected in the environment 600 (or the environment 670).

[0087] Based on the received data, the vehicle 602 may update the rendering (706). For example, the vehicle 602 may update the rendering of objects in the environment (708). In some example implementations, the vehicle 602 may receive an object identifier in the received data. For example, the LIDAR device 612 in a communication mode may receive a packet including an identifier of the vehicle 608 from the LIDAR device 614 via a modulated optical signal. The identifier may be a specific vehicle ID in the packet, a vehicle identification number, a make and model of the vehicle, or another suitable identifier.

[0088] The vehicle 602 may determine rendering information for the object based on the identifier. Exemplary rendering information may include dimensions of one or more shapes in which the object is rendered (e.g., one or more rectangles or other shapes in which a vehicle is rendered), a representative image of the object (e.g., a representative drawing of the vehicle based on the type of vehicle, the make and model of the vehicle, etc.), a stock image of the object (e.g., a stock photo of the vehicle), or a texture to be applied to the rendering of the object (e.g., a texture to be applied to the rendering of the vehicle). The rendering information may also indicate whether to highlight or adjust (e.g., change size, dimensions, stretch, etc.) the rendering of the object. In some implementations, when the vehicle 602 receives an identifier (e.g., make and model) of the vehicle 608, the vehicle 602 may determine rendering information for the vehicle 608 (e.g., a stored image of the make and model of the vehicle 608) by scanning a database indexed by identifiers (e.g., make and model) associated with the rendering information (e.g., a reference image of the vehicle, stored dimensions of the vehicle, highlighting information if an emergency vehicle, etc.). The vehicle 602 may then update the object rendering based on the determined rendering information. For example, the rendered surface 658 in the rendering 650 may be combined with images or other rendering information stored for the vehicle 608 to update the rendering 650. In another example, the rendered surface may be highlighted. In another example of updating the object rendering, the object rendering may be added to a rendering of the environment. For example, a rendering of the vehicle 676 and / or a rendering of the emergency vehicle 678 may be added to the rendering 690.

[0089] In some other implementations, the LIDAR device may be configured to transmit a data signal that includes a specific description of how an object should be rendered. For example, the LIDAR device 614 of the vehicle 608 may transmit a rendering model, texturing, and any other characteristics for rendering the vehicle. In this manner, memory including lookup tables, databases, etc. may not be used to determine rendering information for objects when updating the rendering of the environment.

[0090] 7, after updating the rendering, the vehicle 602 may provide the updated rendering for display (710). For example, a controller coupled to the vehicle 602 may provide the updated rendering to a display coupled to the vehicle 602, which may display the updated rendering to the driver and / or passengers. In another example, the rendering may be displayed to inform others about the vehicle 602's environment. may be provided to a remote display.

[0091] FIG. 8 illustrates an example rendering 800 of the environment 600 of FIG. 6. The rendering 800 may be an example of a rendering that adjusts the rendering 650 of FIG. 6B based on rendering information for the vehicles 608 and 610. For example, the vehicle 602 may store representative images (stock images, line drawings, etc.) or other suitable rendering information for the received identifiers of the vehicles 608 and 610. The LIDAR devices 614 and 616 may transmit vehicle identifiers to the LIDAR device 612, and the vehicle 602 may use the received vehicle identifiers to determine rendering information for the vehicles 608 and 610 and adjust the rendering 650.

[0092] The rendered surface 658 associated with the vehicle 608 may be combined with determined rendering information of the vehicle 608 (such as a stock image or stored drawing of the vehicle 608). For example, the image of the vehicle 608 may be aligned with the rendered surface 658 (such as by scaling and / or orienting the image of the vehicle 608 to align with the rendered surface 658). In the example rendering 800, the object 808 corresponds to the adjusted rendering of the vehicle 608. Similarly, the object 810 of the rendering 800 corresponds to the adjusted rendering of the vehicle 610.

[0093] Although not shown, other objects that may be represented in rendering 650 may include infrastructure, buildings, road markings, pedestrians, or other non-vehicles for which the rendering may be adjusted. In some examples, a LIDAR device coupled to the vehicle may transmit information about objects in the environment other than the vehicle. For example, one or more LIDAR devices (such as LIDAR device 614) of vehicle 608 may be used to determine the depth of objects in the vehicle's environment, and one or more LIDAR devices (such as LIDAR device 616) of vehicle 610 may be used to determine the depth of objects in the vehicle's environment. In one example, each vehicle 602, 608, and 610 may generate a rendering based on measurements provided by the LIDAR device of the respective vehicle. In some implementations, vehicles 602, 608, and 610 may share LIDAR measurements with each other (via LIDAR communication), and the combined LIDAR measurements may be used by vehicle 602 to generate a more comprehensive rendering of environment 600 than rendering 650.

[0094] In some implementations, any suitable updated rendering may be displayed on the display of the vehicle 602. For example, updated renderings may be generated for areas of the environment that are not yet within range or LoS of the LIDAR device of the vehicle 602. The vehicle 610 may detect objects near the street behind the vehicle 610 (e.g., using the LIDAR device 616), which may be used to render the environment 600 that is not within the field of view of the vehicle 612. The vehicle 610 may transmit rendering information of at least a portion of its environment (e.g., rendering information of pedestrians, bicyclists, parked cars, traffic cones, obstructions, obstacles, etc.) to the vehicle 602. In this manner, the vehicle 602 may use the rendering information to include additional information about the environment 600 (which is blocked by the building 606 from the field of view of the LIDAR device 612 of the vehicle 602) on the right side of the rendering 650. In some aspects, rendering information may be shared among multiple vehicles and infrastructure, and a rendering of a particular vehicle (such as vehicle 602) may include an area well beyond the vehicle. For example, rendering information from vehicle 610 regarding the vehicle's environment over multiple hops may be transmitted over hops to vehicle 610 and then to vehicle 602. The rendering information may then be used by vehicle 602 to expand the area covered by the rendering generated by vehicle 602.

[0095] The received data is used to update renderings as well as to guide autonomous vehicle navigation or may be used for other automated operations. For example, the updated rendering may show additional obstructions or obstacles, and the vehicle may navigate (update its navigation, etc.) to avoid the additional obstacles during operation. Referring back to renderings 650 and 800 of FIG. 6B and FIG. 8, respectively, rendering 650 does not show the rear of vehicle 610, while updated rendering 800 shows all of vehicle 610. If the route of vehicle 602 is determined to collide with the rear of vehicle 610 determined based on the received data, the navigation of vehicle 602 may be updated to avoid vehicle 610. In another example, the driver and / or passengers may be notified of a possible collision or obstruction based on the updated rendering. For example, an updated rendering including the obstruction or obstacle may be displayed to the driver and / or passengers, or the driver and / or passengers may be otherwise notified of the obstruction or obstacle. In this manner, the vehicle's occupant may be informed of route changes, vehicle slowdowns, or other changes in the autonomous vehicle's navigation before the obstruction or obstacle comes into the occupant's field of view. Additionally, or alternatively, a dispatcher or central office of the fleet that includes the vehicle may be notified of the obstruction or obstacle to provide a reason to adjust the vehicle's navigation.

[0096] In addition to updating rendering 650 to include a make or other representative image of a vehicle (or another object in the environment), rendering 650 may be updated to include a texture for that make. For example, renderings of vehicles in the same fleet may be coordinated to have similar textures to indicate that they belong to the same fleet.

[0097] FIG. 9A illustrates an example rendering 900 of the environment 600 of FIG. 6A, in which vehicle 608 and vehicle 602 belong to the same fleet. As illustrated, objects 902 and 908 in the rendering 900 may be textured (e.g., shaded) similarly. Object 910, which corresponds to vehicle 610 that is not part of the same fleet, may have a different texture than objects 902 and 908. For example, object 910 may include a different shading than objects 902 and 908. Objects such as infrastructure, buildings, roads, vehicles, etc. may have different textures depending on the type of object. For example, stands may be rendered using the same texture, emergency vehicles may be rendered using the same texture, etc. In this manner, a driver and / or passengers may easily identify similar objects in the environment based on the textures of the displayed rendering. In some implementations, updating the rendering may include highlighting the rendering of the object. For example, the rendering of the emergency vehicle may be highlighted to notify the vehicle occupants of the presence of the emergency vehicle. Similarly, construction zones, safety zones, or other areas of the environment may be highlighted to inform vehicle occupants of their presence within the environment.

[0098] 9B shows an updated example rendering 950 of the environment 670 (FIG. 6C). The updated rendering 950 includes a rendering 952 of an emergency vehicle 678. The rendering 952 may be highlighted to indicate that the vehicle 678 is an emergency vehicle. The rendering 952 may also include a representative image 954 included with the vehicle 674 and a representative image 956 included with the vehicle 676, which was not originally included in the rendering 670.

[0099] 6C , the vehicle 676 and the emergency vehicle 678 may be blocked from the LoS of the LIDAR device coupled to the vehicle 672. As a result, the rendering 690 of FIG. 6D (based on measurements from the LIDAR device) does not include renderings of the vehicle 676 and the emergency vehicle 678. In some implementations, the LIDAR device coupled to the vehicle 672 may receive a reflection of a modulated optical signal from the LIDAR device coupled to the vehicle 676 or the emergency vehicle 678. The modulated optical signal may include data indicative of the presence of the emergency vehicle 678. The data may also include information such as the type of emergency vehicle, whether the emergency vehicle 678 is responding to an emergency, and / or other information regarding the emergency vehicle, such as the path of travel of the emergency vehicle, its speed, etc. The data may also include information about other vehicles or objects. For example, the data may indicate the presence of vehicle 676.

[0100] In some other implementations, the vehicle 672 may receive data regarding the emergency vehicle 678 from an intermediate device. The vehicle 676 may detect the presence of the emergency vehicle 678 (e.g., via a LIDAR device coupled to the vehicle 676). The vehicle 676 may transmit data regarding the emergency vehicle 678 to the vehicle 674 in the LoS via a modulated optical signal 680. The vehicle 674 may then transmit data regarding the emergency vehicle 678 to the vehicle 672 in the LoS via a modulated optical signal 682. Although one intermediate node is shown for transmitting the data, any number of hops may be performed by the data. Once the vehicle 672 receives the data regarding the emergency vehicle 678 (and the data regarding the vehicle 676), the vehicle 672 may update the rendering 690 to highlight or otherwise indicate the presence of the emergency vehicle 678 as well as fill in the blockage caused by the building 684. In addition to or instead of highlighting the emergency vehicle, the vehicle 672 may otherwise notify the driver and / or crew by, for example, flashing the display, providing an audio notification, applying a special texture to the rendering, and / or providing a text notification on the display. Although the examples of Figures 6C, 6D, and 9B are described with respect to an emergency vehicle (such as an ambulance), the actions may be applied to any vehicle, object, or area of ​​interest (such as a construction zone, a school zone, an accident area, etc.).

[0101] In some implementations, the rendering information for objects in the environment is generated by software The rendering information may be provided via a SaaS (Software as a Service). For example, a unique identifier for the object may be received by the vehicle 602. The vehicle 602 may then communicate with the service (e.g., via a cellular modem or other communication means) to obtain rendering information for the object. For example, the vehicle 602 may receive a vehicle identification number for the vehicle 608, and the vehicle 602 may communicate with a remote server that stores rendering information for multiple objects, including the vehicle 608. The remote server may provide the rendering information for the vehicle 608 in response to a request, and the rendering information may be used by the vehicle 602 to update the rendering.

[0102] In some other implementations, the vehicle 602 may store rendering information for objects, and a remote server may provide rendering information for objects not stored locally for the vehicle 602. Additionally or instead, the remote server may be used to update the locally stored rendering information. For example, if a new make and model of a vehicle is released, rendering information for the vehicle may be created but not yet stored in the memory of the vehicle 602. Using LIDAR communications, the locally stored rendering information of the vehicle 602 may be updated to include the new rendering information. In some implementations, the vehicle 602 may update its stored rendering information when coupled to a stand with a dedicated backhaul (such as stand 408 of FIG. 4).

[0103] Other use cases for LIDAR communication exist, and the disclosure is not limited to the above examples. For example, a device (such as a vehicle) may include software or firmware that periodically requires updating. A LIDAR device may be used to receive software updates from other devices. For example, a software update may be provided to a first subset of vehicles, including rendering information for new make and model of vehicles. These vehicles may transmit the update data to other vehicles in the environment via LIDAR communication. In this way, the update data may be distributed quickly, without all vehicles needing to connect to a central location (such as a server that stores the updates). For example, communication between devices (such as vehicles and infrastructure) may resemble a peer-to-peer network, where update data may be propagated between devices, without the need for a central repository to download the update data. Another Exemplary Use Case may include audio or video transmission between devices (eg, voice calls, messaging, video conferencing, etc.).

[0104] In general, one or more actions may be based on data received via modulated optical signals for LIDAR communications. As described herein, LIDAR communications may be used to provide data to, for example, affect navigation of an autonomous vehicle, provide notifications to a driver or passengers, coordinate access to locations, etc.

[0105] FIG. 10 illustrates a flow chart showing example operations 1000 for performing one or more operations based on data received in a transmission from a LIDAR transmitter. FIG. 10 is discussed in terms of an apparatus performing one or more operations for purposes of describing aspects of the disclosure. The operations 1000 may be performed by a vehicle, an infrastructure, a dispatcher or central office coordinating a fleet of vehicles, a processing system (such as an entertainment system) coupled to the vehicle, or a suitable control system (such as a control system coupled to the vehicle for level 4 autonomy). The example operations 1000 are not limited to being performed by or performing operations of a particular apparatus.

[0106] Beginning at 1002, a device may receive data associated with a modulated optical signal emitted by a transmitting LIDAR device. For example, a device (such as a vehicle, a processing system, etc.) may be coupled to a LIDAR device configured to receive a modulated optical signal from a separate transmitting LIDAR device. A receiver of the receiving LIDAR device may receive the modulated optical signal, which may include an optical carrier signal modulated to carry a data signal including data. The carrier signal may be a signal typically used for detection and ranging. The receiving LIDAR device (or a device coupled to the receiving LIDAR device) may demodulate the modulated optical signal to generate and provide the data provided in the modulated optical signal. In this manner, the device (such as a processing system or a vehicle) may receive data associated with the modulated optical signal. In some implementations, the data may include one or more packets. Referring back to FIG. 3, an example packet may include information specific to, for example, locating the vehicle, using the vehicle to access a particular area or location, object rendering, etc.

[0107] Referring back to FIG. 10, the device may determine one or more actions to perform based on the received data (1004). For example, the device may determine one or more actions based on the received one or more packets. In some implementations, the device may determine one or more vehicle locator actions (1006). As an example, if the vehicle receives information in the data that it should locate its vehicle, the vehicle may determine to return to a specified location (such as a parking space that includes a stand as shown in FIG. 4) and communicate the vehicle's location and current status to the stand (via LIDAR communication). In another example, if the device receives information about a missing vehicle and the device has recently communicated with the missing vehicle, the device may communicate about the last received location of the missing vehicle and other information about the missing vehicle, such as vehicle, infrastructure, dispatch, etc. Additionally or alternatively, the device may propagate the missing vehicle message to other vehicles and devices via LIDAR communication.

[0108] In some other implementations, the device may determine a hailing action for one or more vehicles (1008). Referring back to FIG. 4, if the stand 408 is a pedestrian stand for a fleet of automated taxis, the stand 408 may communicate to the vehicle via LIDAR communication that the vehicle has been assigned to a particular client and ride. The vehicle may receive data from the stand 408 via LIDAR communication between a LIDAR device coupled to the vehicle and a LIDAR device coupled to the stand and located in another section where the vehicles are lined up. Based on the received data, the vehicle may determine that the vehicle is near the stand 408. The vehicle or device may approach the space and communicate information regarding the ride or client assigned to the vehicle to the stand 408. The stand 408 may notify the client that the vehicle has arrived so that the client can board. In another example, the vehicle or device may determine that a driver (such as a taxi driver) should be notified about the client and / or ride assigned to the vehicle. The taxi driver may approach the client based on the notification to initiate the ride.

[0109] Referring back to FIG. 10, in some further implementations, the device may determine one or more notification actions (1010). In some examples, referring back to FIG. 5, the vehicle 506 may receive data that an ambulance 502 or other emergency vehicle is nearby. The vehicle 506 may pull up to a curb, stop, or otherwise adjust the operation or travel of the vehicle to allow the ambulance 502 to pass safely. Additionally or alternatively, the vehicle 506 or another device may determine to notify the driver and / or passengers of the presence of the ambulance 502. In some other examples, the vehicle 506 or device may determine to indicate the presence of the ambulance 502 to other vehicles, infrastructure, or other devices via LIDAR communication (thereby propagating the indication to other vehicles and infrastructure in the area of ​​the ambulance 502).

[0110] In another example of a notification operation, a vehicle may receive data regarding available parking spaces in a parking garage. Thus, the vehicle may move to and park in one of the available spaces. The vehicle or device may decide to notify the driver and / or passengers of the available spaces, etc. (e.g., by showing the available spaces on a displayed map of the parking garage). Other example notification operations may include updating the route based on a notification of an accident or traffic along the current route, notifying other vehicles of the accident or traffic via LIDAR communications, notifying the driver and / or passengers of changes in route or traffic conditions, etc.

[0111] Referring back to FIG. 10, in some other implementations, the device may determine (1012) to operate in one or more restricted access areas. For example, if the city center is restricted to autonomous vehicles, zero-emission vehicles, or other specific types of vehicles, the vehicle may receive data related to the access restriction from a LIDAR device coupled to the vehicle. The LIDAR device may receive data in a modulated optical signal transmitted by a LIDAR device coupled to the infrastructure or another suitable device. For example, a gate, a toll booth, a stand, etc. may include a LIDAR device for transmitting an indication of the access restriction to the vehicle. The vehicle may determine whether the vehicle is authorized to enter the area based on the indication. If the vehicle is authorized, the vehicle may also determine one or more navigation operations to access the area. For example, if access is restricted to autonomous vehicles, the autonomous vehicle may enter the city center. Additionally or alternatively, the vehicle may determine to notify the driver and / or passengers of the approach to the restricted access area and the access restriction. In some examples, the driver may decide whether to access the area based on the notification. In some further examples, the vehicle may be notified of a toll or fee for accessing a particular area. For example, a bridge, lane, or area of ​​a city may be associated with an access fee. Before accessing infrastructure in a limited area (such as providing payment at a toll booth before crossing a bridge), the vehicle may provide account or other payment information via LIDAR communication. In some examples, the fee may be notified to the driver and / or passengers.

[0112] As restrictions to an area may change based on current conditions (time of day, day of week, current congestion, smog levels, conditions requiring area restrictions to be restricted to emergency vehicles, etc.), changes to such restrictions may be communicated. In response, the vehicle may, for example, communicate changes to access restrictions to the area. The vehicle may decide to notify the driver and / or passengers, adjust its route to travel around the newly restricted area, or notify other vehicles of the change via LIDAR communications.

[0113] Referring back to FIG. 10, in some further implementations, the device may determine one or more visualization or rendering operations (1014). For example, the vehicle's environment, the LIDAR device, the infrastructure, or other suitable device may be rendered (e.g., generating a point cloud, mapping the environment, a depth map, etc.), and the rendering may be displayed to the driver and / or passengers. Referring back to FIGS. 6A and 6B, the vehicle 602 (or another suitable device coupled to the vehicle 602, such as an entertainment system, a processing system, etc.) may generate a rendering 650 from the data provided by the LIDAR device 612. The vehicle 602 (or other suitable device) may also receive one or more identifiers of the vehicles 608 and 610 in the transmissions from the LIDAR devices 614 and 616, respectively. Thus, the vehicle 602 (or other suitable device) may determine rendering information for the vehicles 608 and 610 based on the received identifiers (looking up information in a vehicle or device memory, communicating with a remote service to provide rendering information for one or more objects, determining rendering information encoded in transmissions from the respective LIDAR devices, etc.). Thus, the rendering operation may include updating the rendering 650 using the determined rendering information. For example, the rendering 650 of FIG. 6B may be updated to the rendering 800 of FIG. 8 based on a representative image (e.g., make) of the vehicles 608 and 610 used to update the rendering. Other example rendering operations may include applying a defined texture to emergency vehicles, applying similar textures to similar types of objects, applying a unique texture that is specific to each vehicle or object in the environment and associated with a unique identifier (such as the unique transmitter ID 302 of FIG. 3), etc. In some examples, the vehicle 602 or other suitable device may determine that an updated rendering of the environment should be displayed to the driver and / or passengers.The rendering operations may also include contacting a remote service to update a stored database of rendering information, updating the database based on receiving new rendering information (such as a new make or model), or propagating rendering information updates via LIDAR communications to other vehicles or devices in the environment.

[0114] Next, referring back to FIG. 10, the device may instruct one or more actions to be performed (1016). For example, if one or more rendering actions are determined, the device may instruct a display or another device coupled to the display to display the rendering and display the updated rendering. In another example, if the device is a vehicle and the one or more actions are operations in a restricted access area, the vehicle may instruct one or more vehicle components to guide the vehicle into or out of the area. The device may also provide instructions to the driver and / or passengers to display information about the restricted access area.

[0115] Although some of the example processing of the example operations 1000 of FIG. 10 are described from the perspective of a vehicle, any suitable device may perform one or more associated operations. For example, a station (or a processing system coupled to a station) may perform a vehicle locator operation (e.g., contacting a dispatcher) in response to receiving a LIDAR transmission from a missing vehicle. In another example, a station may perform a vehicle paging operation (e.g., notifying a client that a vehicle has arrived) in response to receiving a LIDAR transmission from a called vehicle. In another example, a station may perform a notification operation (e.g., sending a notification of an ambulance in the area) in response to receiving a notification in the LIDAR transmission. In a further example, a toll booth or gate may perform an operation in a restricted access area (e.g., changing access fees or access restrictions in the area) in response to receiving a traffic congestion update from a LIDAR device for infrastructure or vehicles in the area. As discussed, the example operations 1000 are not limited to being performed by a particular device (e.g., a vehicle).

[0116] Additionally, although example actions (e.g., call a ride, restrict access, updated renderings, etc.) are described individually, the device may be configured to perform any number and combination of actions. For example, the device may be configured to update renderings regarding an emergency vehicle, notify a vehicle occupant of the presence of an emergency vehicle, update areas to which the vehicle is restricted based on the presence of the emergency vehicle, and update navigation of the vehicle to avoid the restricted areas.

[0117] Those skilled in the art will appreciate that information and signals may be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof. Furthermore, those skilled in the art will appreciate that examples are provided for illustrative purposes in describing aspects of the present disclosure. Aspects of the present disclosure may be implemented by any suitable device (such as an autonomous vehicle, a fleet of autonomous vehicles, infrastructure, a control system for a vehicle or infrastructure, an entertainment system coupled to a vehicle, or other device), and are not limited to the specific examples herein.

[0118] Further, those skilled in the art will appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the aspects disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, the various components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in various ways for each particular application, and such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure. For example, a processing system of a vehicle or other suitable device may include one or more processors and a memory coupled to the one or more processors. The memory may include instructions executed by the one or more processors to cause the device to perform operations as described herein. The processing system may also be coupled to one or more LIDAR devices for LIDAR communication and surface detection and ranging. In some aspects, the processing system may include dedicated hardware, such as one or more integrated circuits, configured to perform one or more operations.

[0119] Thus, the methods, sequences, or algorithms described in connection with the aspects disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled (e.g., directly or remotely) to one or more processors such that the one or more processors can read information from and write information to the storage medium. Alternatively, the storage medium may be integral to the one or more processors.

[0120] In the foregoing specification, the example embodiments have been described with reference to certain example embodiments thereof. However, it will be apparent that various modifications and changes may be made thereto without departing from the broader scope of the implementation of the present disclosure, as set forth in the appended claims. For example, while the vehicle is generally shown as a car, any suitable vehicle may be used, such as a motorcycle, a drone, an aircraft, a watercraft, a helicopter, etc. Additionally, a vehicle may include one or more LIDAR devices, and the vehicle may be coupled to a LIDAR device manufactured separately from the vehicle. Also, the vehicle may include one or more processing systems, entertainment systems, control systems, etc., and the vehicle may be coupled to such processing systems, entertainment systems, control systems manufactured separately from the vehicle. For example, one or more LIDAR devices may be mounted on various parts of a car to ensure coverage of the environment around the car. The LIDAR devices may be coupled to a processing system (or a control system coupled to the vehicle) of the car to provide data transmitted by other LIDAR devices for processing, or transmit data provided by the processing system to other LIDAR devices via modulated optical signals.

[0121] The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

Claims

1. An apparatus comprising: one or more processors; and a memory coupled to the one or more processors and including instructions that, when executed by the one or more processors, cause the apparatus to: receiving data associated with a modulated optical signal emitted by a transmitter of a first Light Detection and Ranging (LIDAR) device and received by a receiver of a second LIDAR device coupled to the vehicle and the device, the modulated optical signal including an optical carrier signal modulated to include a data signal including the data; generating a rendering of the vehicle's environment based on information from one or more LIDAR devices coupled to the vehicle; updating the rendering based on the received data, where updating the rendering includes updating object renderings of objects in the environment of the vehicle; and and providing the updated rendering for display on a display coupled to the vehicle.

2. the data includes a representation of the object in the environment, and updating the object rendering; highlighting the object rendering during display; adjusting a texture of the object rendering; including a representative image of the object in the rendering of the environment; and and adjusting a dimension of the object rendering.

3. The apparatus of claim 2 , wherein displaying the updated rendering includes notifying at least one of a driver or a passenger of the vehicle that the object in the environment is an emergency vehicle.

4. Execution of the instructions causes the device to: determining one or more guidance actions for the vehicle that are adjusted based on the emergency vehicle in the environment; 4. The apparatus of claim 3, further comprising: providing the one or more coordinated guidance actions for execution by the vehicle.

5. 4. The apparatus of claim 3, wherein the data originates from a second device not coupled to the first LIDAR device.

6. Execution of the instructions causes the device to: searching for the representation of the object in a database indexed by vehicle identifier, the representation of the object including a vehicle identifier; 3. The apparatus of claim 2, further comprising: identifying, in the database, an indication of the update to be performed on the object rendering associated with the vehicle identifier, wherein the updating of the object rendering is based on the identified indication of the update to be performed.

7. Execution of the instructions causes the device to: receiving second data associated with a second modulated optical signal received by the second LIDAR device; 7. The apparatus of claim 6, further comprising: updating one or more entries in the database based on the received second data.

8. the one or more LIDAR devices coupled to the vehicle include the second LIDAR device operating in a detection mode; 10. The device of claim 1, wherein the second LIDAR device operates in a communications mode when receiving the modulated optical signal.

9. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a device, cause the device to: receiving data associated with a modulated optical signal emitted by a transmitter of a first Light Detection and Ranging (LIDAR) device and received by a receiver of a second LIDAR device coupled to the vehicle, the modulated optical signal including an optical carrier signal modulated to include a data signal including the data; generating a rendering of the vehicle's environment based on information from one or more LIDAR devices coupled to the vehicle; updating the rendering based on the received data, where updating the rendering includes updating object renderings of objects in the environment of the vehicle; and and providing the updated rendering for display on a display coupled to the vehicle.

10. the data includes a representation of the object in the environment, and updating the object rendering; highlighting the object rendering during display; adjusting a texture of the object rendering; including a representative image of the object in the rendering of the environment; and and adjusting a dimension of the object rendering.

11. 11. The computer-readable medium of claim 10, wherein displaying the updated rendering includes notifying at least one of a driver or a passenger of the vehicle that the object in the environment is an emergency vehicle.

12. Execution of the instructions causes the device to: determining one or more guidance actions for the vehicle that are adjusted based on the emergency vehicle in the environment; 12. The computer-readable medium of claim 11, further comprising: providing the one or more coordinated guidance actions for execution by the vehicle.

13. 12. The computer-readable medium of claim 11, wherein the data originates from a second device not coupled to the first LIDAR device.

14. Execution of the instructions causes the device to: searching for the representation of the object in a database indexed by vehicle identifier, the representation of the object including a vehicle identifier; 11. The computer-readable medium of claim 10, further comprising: identifying, in the database, an indication of the update to be performed to the object rendering associated with the vehicle identifier, wherein the updating of the object rendering is based on the identified indication of the update to be performed.

15. Execution of the instructions causes the device to: receiving second data associated with a second modulated optical signal received by the second LIDAR device; 15. The computer-readable medium of claim 14, further comprising: updating one or more entries in the database based on the received second data.

16. 1. A method comprising: receiving, by one or more processors of a first Light Detection and Ranging (LIDAR) device, data associated with a modulated optical signal emitted by a transmitter of the first LIDAR device and received by a receiver of a second LIDAR device coupled to the vehicle and the device, the modulated optical signal including an optical carrier signal modulated to include a data signal including the data; generating, by the one or more processors, a rendering of the vehicle's environment based on information from one or more LIDAR devices coupled to the vehicle; updating the rendering based on the received data, where updating the rendering includes updating object renderings of objects in the environment of the vehicle; and and providing the updated rendering for display on a display coupled to the vehicle.

17. the data includes a representation of the object in the environment, and updating the object rendering; highlighting said object rendering during display; adjusting a texture of the object rendering; including a representative image of the object in the rendering of the environment; and and adjusting a dimension of the object rendering.

18. 20. The method of claim 17, wherein displaying the updated rendering includes notifying at least one of a driver or a passenger of the vehicle that the object in the environment is an emergency vehicle.

19. 20. The method of claim 17, wherein the data originates from a second device not coupled to the first LIDAR device.

20. searching for the representation of the object in a database indexed by vehicle identifier, the representation of the object including a vehicle identifier; identifying, in the database, an indication of the update to be performed on the object rendering associated with the vehicle identifier, where the updating of the object rendering is based on the identified indication of the update to be performed; receiving second data associated with a second modulated optical signal received by the second LIDAR device; 20. The method of claim 17, further comprising: updating one or more entries in the database based on the received second data.

Images