LIDAR-based communications

Abstract

To describe systems and methods for performing operations based on LIDAR communications.SOLUTION: An example device may include one or more processors and a memory coupled to the one or more processors. The memory includes instructions that, when executed by the one or more processors, cause the device to receive data associated with a modulated optical signal emitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to a vehicle and the device, generate a rendering of an environment of the vehicle based on information from one or more LIDAR devices coupled to the vehicle, and update the rendering based on the received data. Updating the rendering includes updating an object rendering of an object in the environment of the vehicle. The instructions further cause the device to provide the updated rendering for display on a display coupled to the vehicle.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Patent Application No. 16 / 502,008, filed July 2, 2019, the entire contents of which are incorporated herein by reference.

[0002] The present disclosure relates generally to LIDAR devices, and more particularly to using LIDAR as a communication channel in autonomous driving applications.

[0003] 2. Description of Related Art Light detection and ranging (LIDAR) devices can be used to detect objects in an environment. For example, a vehicle may include or be coupled to one or more LIDAR devices that detect vehicles, pedestrians, traffic lights, obstacles, etc. The LIDAR devices emit light at specific frequencies (e.g., 800-1000 nm or 1550 nm), and the LIDAR devices receive reflections of the emitted light. The LIDAR devices then determine the time of flight (ToF) of the light as they scan the environment to estimate the distances of multiple reflective surfaces. The estimated distances can be used to generate a point cloud representing the environment or otherwise render the environment or to assist in the operation of the vehicle.

[0004] For example, a vehicle can be configured to operate in an autonomous mode in which the vehicle navigates an environment with little or no input from the driver. An autonomous vehicle may include one or more LIDAR devices to determine the distance of objects in the environment, which may be used to control the navigation of the vehicle. In another example, a vehicle may include one or more LIDAR devices to assist the driver, from performing adaptive cruise control, providing traffic alerts at intersections, providing lane departure warnings, etc., to performing all safe and critical driving functions and monitoring road conditions in a fully autonomous system. Summary of the Invention

[0005] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0006] Innovative aspects of the subject matter described in this disclosure may be implemented in an apparatus coupled to one or more LIDAR devices. In some implementations, an example apparatus includes one or more processors and a memory coupled to the one or more processors. The memory includes instructions that, when executed by the one or more processors, cause the apparatus to receive data associated with a modulated optical signal emitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to the vehicle, generate a rendering of the vehicle's environment based on information from the one or more LIDAR devices coupled to the vehicle, and update the rendering based on the received data. Updating the rendering includes updating object renderings of objects in the vehicle's environment. The instructions further cause the apparatus to provide the updated rendering for display on a display coupled to the vehicle.

[0007] Innovative aspects of the subject matter described in this disclosure may be implemented in a computer-readable medium that stores instructions that, when executed by one or more processors of a device, cause the device to receive data related to modulated optical signals emitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to a vehicle. The instructions cause the device to receive data, generate a rendering of the vehicle's environment based on information from one or more LIDAR devices coupled to the vehicle, and update the rendering based on the received data. Updating the rendering includes updating object renderings of objects in the vehicle's environment. The instructions further cause the device to provide the updated rendering for display on a display coupled to the vehicle.

[0008] Innovative aspects of the subject matter described in this disclosure can be implemented as a method. An example method includes receiving, by an apparatus, data related to a modulated optical signal emitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to the vehicle and the apparatus, generating a rendering of the vehicle's environment based on information from one or more LIDAR devices coupled to the vehicle, and updating the rendering based on the received data. Updating the rendering includes updating object renderings of objects in the vehicle's environment. The method includes providing the updated rendering for display on a display coupled to the vehicle.

[0009] Another exemplary apparatus may include one or more processors and a memory coupled to the one or more processors. The memory includes instructions that, when executed by the one or more processors, cause the apparatus to receive data related to infrastructure (e.g., a toll booth, a road construction area, a tunnel entrance, etc.) and a modulated optical signal emitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to the apparatus. The apparatus may generate a rendering of the infrastructure's environment based on information from the one or more LIDAR devices coupled to the infrastructure. The apparatus may also update the rendering based on the received data. Updating the rendering may include updating object renderings of objects in the infrastructure's environment. The apparatus may further provide the updated rendering for display. For example, the rendering may be displayed to a toll booth attendant, a central transit authority accountant, a construction worker foreman at a construction site, etc.

[0010] The data may include a representation of an object in the environment, and updating the object rendering may include highlighting the displayed object rendering, adjusting the texture of the object rendering, including a representative image of the object in the rendering of the environment, and / or adjusting the dimensions of the object rendering. Displaying the updated rendering may include informing the viewer that the object in the infrastructure environment is an emergency vehicle.

[0011] The device may also determine guidance behavior for one or more vehicles within the infrastructure environment based on the received data, and the device may provide coordinated guidance behavior to the one or more vehicles. The modulated optical signal may be received by a second LIDAR device operating in a communication mode, and objects within the infrastructure environment and line of sight of the second LIDAR device may be detected by the second LIDAR device operating in a detection mode.

[0012] The device may further receive second data associated with the second modulated optical signal received by the second LIDAR device and may update one or more entries in the local database based on the received second data. In some implementations, the device may transmit data regarding the update to the other LIDAR device to update the database local to the other LIDAR device.

[0013] The details of one or more implementations of the subject matter described in this disclosure are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, drawings, and claims. Please note that the relative dimensions of the following figures may not be drawn to scale. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram of an example LIDAR device. [Figure 2A] 1 illustrates an example LIDAR device within an environment. [Figure 2B] 1 is an example timing diagram of waveforms corresponding to emitted and received light pulses of an example LIDAR device. FIG. [Figure 3] 1 illustrates an example packet format for multiple packets received from a LIDAR transmission according to an example communication protocol. [Figure 4] 1 illustrates an environment including exemplary parking spaces for autonomous fleet vehicles to return to when not in use. [Figure 5]1 illustrates an exemplary environment for communicating the presence of an ambulance to other vehicles in the environment. [Figure 6A] 1 illustrates an exemplary environment for a vehicle. [Figure 6B] 1 shows an example rendering of a vehicle's environment as sensed by a LIDAR device. [Figure 6C] 1 illustrates an exemplary environment for vehicles, including emergency vehicles. [Figure 6D] 6D shows an example rendering of the vehicle's environment of FIG. 6C as sensed by a LIDAR device. [Figure 7] 1 shows a flowchart depicting example operations for adjusting the rendering of an environment of a LIDAR device. [Figure 8] 6B shows an example adjusted rendering of the environment in FIG. 6A. [Figure 9A] 6B illustrates an example rendering of the environment of FIG. 6A in which two vehicles in the environment belong to the same fleet. [Figure 9B] 6D illustrates an example rendering of the environment of FIG. 6C, where the environment includes an emergency vehicle. [Figure 10] 1 shows a flowchart depicting example operations for performing one or more actions based on data received in a transmission from a LIDAR transmitter.

[0015] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION OF THE INVENTION

[0016] Efforts have been made to connect vehicles to infrastructure via cellular communications. For example, some vehicles include cellular modems that communicate with other vehicles or devices over the 5 GHz radio spectrum. In another example, a vehicle may be equipped with dedicated short-range communications (DSRC) equipment that communicates with other vehicles or devices over the 5.9 GHz radio spectrum. One issue with cellular and wireless technologies for communication is that vehicles require a cellular connection or may require other vehicles or infrastructure to be equipped with consistent technology for communication. For example, some rural or saturated urban environments may not include available cellular connections, and vehicles may not be able to communicate with other vehicles via cellular modems. Another issue with cellular technologies for communication is the inherent delays associated with communication. For example, cellular communication between vehicles may require communication through one or more base stations of the cellular network, which can delay communication. Furthermore, the wavelength of the radio signals used for communication limits communication bandwidth. Furthermore, cellular and wireless signals are typically transmitted omnidirectionally, and transmitting multiple omnidirectional signals over the ground can quickly saturate the environment.

[0017] Many vehicles and infrastructure (e.g., toll booths, traffic lights, charging stations, etc.) may include one or more LIDAR devices or may be coupled to one or more LIDAR devices. Furthermore, as autonomous vehicles become more common, equipped LIDAR devices also become more common. In some embodiments, LIDAR devices may be configured to communicate with other LIDAR devices in addition to performing ToF measurements (to detect surfaces in the environment). Because the frequency of the emitted optical signals is higher than that of radio signals, the communication bandwidth between LIDAR devices may be higher than that of cellular communications. Furthermore, communication between LIDAR devices may involve the use of a cellular network. No network or other infrastructure is required. Furthermore, optical signals (such as those with wavelengths close to 1000 nm) may be emitted in a focused-divergent pattern to prevent saturation and interference from multiple optical signals transmitted simultaneously over the airwaves.

[0018] Implementations of the subject matter described herein may enable a LIDAR device to communicate with another LIDAR device (referred to herein as "LIDAR communication"). LIDAR communication may be used for vehicle-to-vehicle (V2V) communication or vehicle-to-infrastructure (V2X) communication, and LIDAR communication may occur between any compatible vehicles and / or infrastructure (such as within a fleet of vehicles or between unrelated vehicles equipped with configured LIDAR devices). LIDAR communication may be used in a variety of situations and use cases, as described herein.

[0019] The following description is directed to a particular implementation for purposes of illustrating innovative aspects of the present disclosure. However, those skilled in the art will readily recognize that the teachings herein can be applied in many different ways. The described implementations may be implemented in any device, system, or vehicle that includes or is coupled to one or more LIDAR devices. In some implementations, an "device" for performing the operations described herein may refer to a vehicle and a controller or system coupled to one or more LIDAR devices, a vehicle that includes a controller or system and is coupled to one or more LIDAR devices, a controller or system coupled to infrastructure or another non-vehicle system, or another suitable implementation. Similarly, a "vehicle" may refer to a controller or system coupled to a vehicle, a vehicle separate from a controller or system coupled to a vehicle, a combination of a vehicle coupled to a controller or system, or another suitable implementation.

[0020] In the following description, numerous specific details are set forth, such as examples of specific components, systems, and processes, to provide a thorough understanding of the present disclosure. Also, for purposes of the following description and explanation, specific terminology and / or details are set forth to provide a thorough understanding of the example embodiments. However, it will be apparent to those skilled in the art that these specific details may not be necessary to practice the example embodiments. In other instances, well-known circuits, systems, and devices are shown in block diagram form to avoid obscuring the disclosure. Any signals provided over various buses described herein may be time-multiplexed with other signals and provided over one or more common buses. Furthermore, interconnections between components or software blocks may be depicted as buses or single signal lines. Each bus may alternatively be a single signal line, and each single signal line may alternatively be a bus, and a single line or bus may represent any one or more of numerous physical or logical mechanisms for communication between components. Furthermore, as used herein, the term "coupled" means directly coupled to or coupled through one or more intervening components or devices.

[0021] FIG. 1 is a block diagram of an example light detection and ranging (LIDAR) device 100. The LIDAR device 100 can be used to detect the surface of an object in an environment by emitting pulses of light that illuminate the surface of the object and detecting the light pulses reflected from the object's surface. The LIDAR device 100 can determine the distance to the object based on the time delay between emitting the light pulse and receiving the corresponding light pulse reflected from the selected object. This time delay, sometimes referred to as the ToF of the light pulse, can be multiplied by the speed of light to determine the distance between the LIDAR device 100 and the object. Multiple pulses can be used to determine distance information for several points associated with the object in the environment. These points can be used to generate a point cloud or otherwise determine the object's position, size, shape, attitude, and motion. In some implementations, information from the LIDAR device 100 can be used to control, for example, an autonomous vehicle, so that the autonomous vehicle can navigate an environment to reach a destination while avoiding obstacles. The LIDAR device 100 may also be used to measure distance for driver assistance operations.

[0022] The LIDAR device 100 is shown to include a transmitter 110, a receiver 120, and a LIDAR controller 130. The transmitter 110 may include a transmit controller 111, one or more light emitting elements 112, and a transmit aperture 113. The light emitting elements 112 may emit one or more light pulses 125 that can be used to detect objects in the surrounding environment. The light emitting elements 112 may include any number of suitable light sources, such as, but not limited to, laser diodes, light emitting diodes (LEDs), vertical cavity surface emitting lasers (VCSELs), organic light emitting diodes (OLEDs), polymer light emitting diodes (PLEDs), light emitting polymers (LEPs), liquid crystal displays (LCDs), microelectromechanical systems (MEMS), or any other device configured to selectively transmit or emit light pulses 125 of prescribed wavelengths. The light source wavelengths may include, for example, the ultraviolet, visible, and / or infrared portions of the electromagnetic spectrum. In some embodiments, the light emitting elements 112 may be disposed on one or more substrates (e.g., printed circuit boards (PCBs), flexible PCBs, etc.). Although the light emitting elements 112 are described herein as emitting light pulses 115, one skilled in the art will readily understand that the light emitting elements 112 may transmit or emit light signals, light beams, photons, etc. Thus, the terms light pulses, light signals, light beams, and photons may be used interchangeably herein.

[0023] The transmit aperture 113 may include any suitable components (e.g., mirrors, lenses, diffraction gratings, exit apertures, etc.) coupled to the light emitting elements 112 and capable of focusing, directing, and / or conditioning the light pulses 115 for emission into the surrounding environment. In some implementations, the transmit aperture 113 may be configured to direct the light pulses 115 in one or more specified directions relative to the LIDAR device 100. The specified directions may span a range of directions, for example, such that the distance between the LIDAR device 100 and multiple objects (e.g., vehicles, people, roads, traffic lights, traffic signs, obstacles, etc.) can be determined based on reflections of the light pulses 115 caused by the objects.

[0024] The transmit controller 111 may control the operation of the light emitting elements 112 and the transmit aperture 113 and may adjust multiple parameters or settings of the light emitting elements 112 and the transmit aperture 113, or both. In some implementations, the transmit controller 111 may be responsive to one or more control signals provided by the LIDAR controller 130. For example, the transmit controller 111 may adjust the width, timing, frequency, and / or amplitude (intensity) of the light pulses 115 emitted by the light emitting elements 112 based on the one or more control signals. In other implementations, the transmit controller 111 may be omitted or may be included within the LIDAR controller 130.

[0025] The receiver 120 may include multiple photodetectors 121, detector circuitry 122, and an analog-to-digital converter (ADC) 123. The photodetectors 121 may receive light pulses 125 (e.g., photons) from the surrounding environment. In some implementations, the received light pulses 125 may include components of the emitted light pulses 115 that have been reflected from one or more objects in the surrounding environment. The photodetectors 121 may be configured to convert the received light pulses 125 into a photodetector signal (e.g., an analog current signal) indicative of the intensity level of the received light pulses 125. The photodetectors 121 may be any suitable component or element capable of receiving or detecting light, including, for example, a photodiode (e.g., an avalanche photodiode), a silicon photomultiplier tube (SiPM), a phototransistor, a camera (e.g., a CMOS sensor), an active pixel sensor (APS), a charge-coupled device (CCD), a cryogenic detector, etc. In some implementations, the photodetectors 121 may be configured to generate a current flowing through each photodiode in response to receiving a light pulse, for example, such that the amount of current passing through each photodiode is proportional to the intensity of the light pulse received by that photodiode. It is a reverse biased photodiode that generates a current.

[0026] Although not shown for simplicity, receiver 120 may include optics for filtering the wavelength of the received light such that photodetector 121 receives primarily light corresponding to the wavelength of the optical pulses 115 emitted by transmitter 110 (and receives weaker light corresponding to other wavelengths). For example, receiver 120 may include a bandpass filter for filtering optical signals outside of a wavelength range centered around the fundamental wavelength of the optical pulses emitted by transmitter 110.

[0027] Detector circuit 122 may use any suitable technique to sample the photodetector signal provided by photodetector 121 to determine the intensity level of the received light pulse 125. In some implementations, detector circuit 122 may sample the photodetector signal at multiple intervals or sampling times. In other implementations, detector circuit 122 may continuously sample the photodetector signal. Detector circuit 122 may provide the determined intensity level to ADC 123, for example, as an analog signal having a magnitude (e.g., voltage magnitude or current magnitude) indicative of the light information contained in the photodetector signal. In some embodiments, detector circuit 122 may amplify and / or filter the photodetector signal.

[0028] The ADC 123 may receive an analog signal indicative of the intensity level of the received light pulse 125 from the detector circuit 122 and convert the analog signal into digital data that can be processed by the LIDAR controller 130. The ADC 123 may be any suitable ADC, such as, but not limited to, a flash ADC, a successive approximation register (SAR) ADC, or a delta-sigma ADC. In some implementations, each photodetector 121 may correspond to a respective ADC. In other implementations, multiple photodetectors 121 may correspond to a single ADC (e.g., to reduce the size, cost, and / or power consumption of the LIDAR device 100). In some other implementations, the ADC 123 may be omitted. Each of the photodetectors 121 (and corresponding ADC 123) may be associated with a particular light-emitting element 112. In this manner, multiple ToFs may be measured, and therefore multiple distances may be determined, during a single scan of the LIDAR device 100.

[0029] The LIDAR controller 130 may include a processor 131, a memory 132, and a digital signal processor (DSP) 133. The DSP 133 may process digital data provided by one or more ADCs 123 to determine information about the light pulses received by the number of photodetectors 121. In some implementations, the intensity and / or arrival time of the light pulses may be used to determine the size, shape, and location of multiple objects detected in the surrounding environment. For example, if the time of emission of the light pulse is known, the DSP 133 may use the time of arrival to determine the ToF. In another example, if the intensity of the emitted light pulse is known, the DSP 133 may use the measured intensity to determine the energy loss of the reflected light pulse. Objects relatively closer to the LIDAR device 100 may reflect the emitted light pulse 115 before objects relatively farther from the LIDAR device 100. Furthermore, light reflected from objects relatively closer to the LIDAR device 100 may experience less pulse spread than light reflected from objects relatively farther from the LIDAR device 100 (assuming similar surface reflectivities between objects at various distances). Thus, in some implementations, the distance between the LIDAR device 100 and an object may be estimated based on the rising and falling edges of the received light pulses 125.

[0030] Processor 131 may be one or more any suitable processors capable of executing scripts or instructions of one or more software programs stored in LIDAR device 100 (e.g., in memory 132). In some implementations, processor 131 may provide at least a portion of a machine-readable medium on which program instructions or scripts may be stored. In other implementations, processor 131 may be or include one or more field programmable gate arrays (FPGAs) or programmable logic devices (PLDs).

[0031] The memory 132 may store information about the transmitter 110, the receiver 120, the surrounding environment, or any combination thereof. The memory 132 may also include a non-transitory computer-readable medium (e.g., one or more non-volatile memory elements such as an EPROM, an EEPROM, a flash memory, a hard drive, etc.) that may store multiple software (SW) modules, each including instructions that, when executed by the processor 131, cause the LIDAR device 100 to perform all or a portion of the operations described herein. In some other implementations, the LIDAR controller 130 may be instructed to perform one or more operations related to emitting or receiving light pulses by one or more processors external to the LIDAR device 100 (e.g., included in a processing system of a vehicle coupled to the LIDAR device 100). For example, the LIDAR device 100 may be coupled to a vehicle processing hub or another processing system of the vehicle (such as via a Controller Area Network (CAN) bus), and the processing system may instruct the LIDAR device 100 to perform one or more operations and may receive information from the LIDAR device 100 in response (such as obtaining ToF or intensity information measured by the LIDAR device 100 and used by the processing system to generate a point cloud or other type of depth map or rendering of the environment).

[0032] FIG. 2A illustrates an exemplary LIDAR device within an environment. In the example of FIG. 2A, LIDAR device 100 is positioned within environment 200, which includes automobile 201 and canopy 202. In the simplified example, LIDAR device 100 is shown to include three light emitters 112A-112C, each emitting a light pulse 115A-115C into environment 200. LIDAR device 100 is also shown to include three photodetectors 121A-121C, each receiving light pulses 125A-125C reflected from surfaces of objects within environment 200. A first light pulse 115A illuminates surface 205A of canopy 202, and first photodetector 121A receives the corresponding reflected light pulse 125A. A second light pulse 115B illuminates surface 205B of automobile 201, and second photodetector 121B receives the corresponding reflected light pulse 125B. A third light pulse 115C illuminates another surface 205C of the vehicle 201, and a third photodetector 121C receives a corresponding reflected light pulse 125C. The LIDAR device 100 can use one or more characteristics (e.g., timing, amplitude, pulse width, etc.) of the received light pulses 125A-125C to determine the distance between the LIDAR device 100 and each surface 205A-205C in the environment 200.

[0033] 2B is an example timing diagram 210 of waveforms corresponding to emitted and received light pulses of an example LIDAR device. Transmit waveforms 215A-215C may represent the intensity levels of each of the light pulses 115A-115C emitted from the LIDAR device 100 of FIG. 2A, and receive waveforms 225A-225C may represent the intensity levels of each of the light pulses 125A-125C received by the LIDAR device 100 of FIG. 2A. The light pulses 115A-115C are emitted from the LIDAR device 100 at the same time t0 (or at least substantially simultaneously), and the reflected light pulses 125A-125C are 、 At different times t A ~t CThe transmitted waveforms 215A-215C include pulses 216A-216C, respectively, that represent the time t when the corresponding light pulses 115A-115C are emitted from the LIDAR device 100. The received waveforms 225A-225C represent the time t when the corresponding reflected light pulses 125A-125C are received by the LIDAR device 100. A ~t C 2. The pulses 216A-216C each represent a pulse A clock detection technique (e.g., determining peak amplitude, determining center, determining average time between threshold crossings, etc.) is used to determine the reception time t A ~t C can be obtained from the pulses 226A to 226C. A ~~t C can be used to determine the distances between the LIDAR device 100 and the surfaces 205A-205C of the environment 200, respectively.

[0034] Many LIDAR devices are capable of adjusting one or more characteristics of the emitted light pulses. For example, the LIDAR device 100 (FIG. 1) may configure the transmitter 110 to adjust one or more of the timing, frequency, or intensity of the light pulses of the emitted light 115. In some examples, the LIDAR device 100 may dither the light pulses of the emitted light 115 to control the timing. Some LIDAR devices may be able to adjust the power provided to the light emitting elements 112 (e.g., LEDs) to control the intensity. Some LIDAR devices may be able to adjust the frequency of the light pulses of the emitted light 115 by adjusting a reference signal frequency (e.g., a light source). The receiver 120 may also be configured to detect differences in the timing, frequency, or intensity of the light pulses of the received light 125. For example, the sensitivity of the photodiode array of the photodetector 121 may be sufficient to determine differences in intensity, and the sampling rate of the photodiode array and / or ADC may be sufficient to determine differences in the timing or frequency of the light pulses of the received light 125.

[0035] In some aspects, the LIDAR device 100 (such as the LIDAR controller 130) may be configured to encode information into the emitted light 115 via frequency adjustment, intensity adjustment, and / or temporal dithering of the emitted pulses, and the transmitter 110 may be configured to communicate such encoded information via the emitted light 115. For example, the LIDAR device 100 may receive a transmitted data signal, and the LIDAR device 100 may use the emitted light (sometimes referred to herein as an optical signal) as a carrier signal. In this manner, the LIDAR device 100 may modulate the optical signal to include the data signal, and the LIDAR device 100 may transmit the modulated optical signal. In the case of a first LIDAR device, the receiver 120 may be configured to receive the modulated optical signal transmitted by the transmitter 110 of the second LIDAR device. The modulated optical signal may be encoded with a data signal from the second LIDAR device, and the LIDAR controller 130 of the first LIDAR device may be configured to extract the data signal from the received modulated optical signal. For example, the LIDAR controller 130 may demodulate the received optical signal to generate the data signal. Because such communication between LIDAR devices is point-to-point (and therefore does not require a central network) and the signal frequency is higher than that of cellular communication, throughput may be higher and latency may be lower than traditional cellular communication. As used herein, the term "modulated optical signal" refers to an optical signal (such as light emitted by the LIDAR device 100) modulated to include a data signal.

[0036] In some implementations, the LIDAR device 100 may be configured to switch between a distance measurement mode (which may be referred to herein as a "detection mode") and a communication mode for transmitting modulated optical signals including data signals. For example, the LIDAR controller 130 may determine when the LIDAR device 100 transmits or receives information via modulated optical signals and when the LIDAR device 100 operates to detect the surface of an object in the environment. For example, the LIDAR controller 130 may determine to place the LIDAR device 100 in the communication mode during a first time portion and the LIDAR device 100 in the detection mode during a second time portion. Thus, the LIDAR device 100 may switch between the communication mode and the detection mode. In other implementations, the emitted light 115 may be used to communicate information to another receiver (e.g., by adjusting the frequency) and to detect the surface of an object (e.g., by detecting a difference in the intensity of the received light 125), and the LIDAR device 100 may be configured to perform both modes simultaneously. For example, the LIDAR device 100 may emit a modulated optical signal (including a data signal). The LIDAR device 100 may receive a reflection of the modulated optical signal and use the reflection to determine the depth of the object from the LIDAR device 100. Additionally, a second LIDAR device may receive the modulated optical signal and demodulate the optical signal to generate the included data signal from the LIDAR device 100.

[0037] A device, such as a vehicle or infrastructure, may include or be coupled to one or more LIDAR devices (such as LIDAR device 100) configured to communicate data signals (e.g., provided by the vehicle or infrastructure) via emitted light 115. In this manner, the device may communicate with the vehicle or infrastructure using the configured LIDAR device. In some embodiments, LIDAR device 100 may be configured to use a communication protocol adopted by other vehicles and infrastructure that include LIDAR devices. The communication protocol may be ad hoc or managed, and any suitable packetization of information may be used for the communication protocol. For example, LIDAR communication within a fleet of vehicles may be based on a specific protocol. In some implementations, a standardized protocol (or a protocol adopted by multiple parties) may more easily enable integration of LIDAR communication between vehicles and infrastructure. Such a protocol may include a defined packet format for transmitting and receiving information. Protocols from other communication media, such as cellular communication, Wi-Fi communication, digital subscriber line (DSL) communication, or fiber optic communication, may be utilized to create a protocol for communication between LIDAR devices.

[0038] FIG. 3 illustrates an example packet format for multiple received packets 300 based on an example communication protocol. In some implementations, the LIDAR device 100 may include a buffer or other suitable memory (e.g., memory 132) for sequencing one or more received packets 300 (e.g., packets 1-6). The LIDAR device 100 may process the buffered packets for vehicle or infrastructure use to perform one or more operations. In another example, the buffered packets may be provided to a processing system of the device for processing. As shown, the format of packets 1-6 includes fields 302-308, which include a transmitter ID 302, a location 304, a payload type 306, and a payload 308. Any suitable packet format may be used, and therefore additional, fewer, different, or different organization of fields may be included in the packets. For example, packets 1-6 may include a cyclic redundancy check (CRC) field after the payload 308 to correct errors in the received packets.

[0039] Transmitter ID 302 may indicate the device transmitting the packet. In some implementations, each vehicle may include a unique identifier to identify that particular vehicle. For example, if a fleet of 200 autonomous vehicles communicate with each other, each vehicle may include an identifier (“vehicle_1” through “vehicle_200,” etc.) that is different from the other vehicles in the fleet. If a LIDAR device equipped on a vehicle associated with “Vehicle_100” transmits packets 1 and 2 to LIDAR device 100, transmitter ID 302 of the packet may include the unique identifier “Vehicle_100.” Similarly, if a LIDAR device equipped on a vehicle associated with “Vehicle_102” transmits packet 3 to LIDAR device 100, transmitter ID 302 of the packet may include the unique identifier “Vehicle_102.” Some vehicles may not include a unique identifier or may not otherwise be identified, but the vehicles may still communicate packets to LIDAR device 100. In one example, Sender ID 302 may include a Sender ID value unique to a previously unidentified vehicle or device (shown as "Unidentified_Vehicle" for packet 4). The Sender ID value may be filled with zeros, a null value field, or otherwise appropriately populated to indicate that the vehicle or infrastructure associated with the packet does not have a unique identifier.

[0040] Infrastructure (such as a toll booth, a tunnel, or an entrance gate for a high-occupancy vehicle (HOV) lane) may include or be coupled to one or more LIDAR devices to transmit and receive packets to and from LIDAR device 100. For example, packet 5 may be transmitted by a transmitter located at the entrance to a tunnel (which may be uniquely identified as "Tunnel_40"), and packet 6 may be transmitted by a transmitter located at a toll booth (which may be uniquely identified as "Toll Booth_30"). Any suitable vehicle or infrastructure may include a LIDAR device for LIDAR communication and may include a unique transmitter ID.

[0041] In some implementations, emergency vehicles, construction zones, and other devices associated with priority transmissions may include a transmitter ID to indicate that the packet is for priority transmission. When the LIDAR device 100 sequentially decodes packets, the LIDAR device 100 may first process the transmitter ID 302 and determine that the packet is a priority transmission. In this manner, the LIDAR device 100 may determine which packets are prioritized before completing processing of the remainder of the packet. Other suitable implementations of the transmitter ID 302 may be used, and the present disclosure is not limited to the provided examples. For example, the transmitter ID may be configured to distinguish between fleets, types of vehicles (such as private cars, commercial trucks, school buses, etc.), etc.

[0042] Location 304 may indicate the location of a vehicle or infrastructure relative to the transmitting device. For example, the vehicle or infrastructure may include a Global Positioning System (GPS) receiver to determine latitude and longitude. The latitude and longitude may then be provided in location 304, as shown in packets 1-6. Alternatively, location 304 may include location information related to LIDAR device 100, as determined by another vehicle or infrastructure via its own LIDAR device. For example, a transmitting LIDAR device may determine the relative distance and position between the transmitting vehicle and the receiving vehicle or infrastructure, and the distance and position of the device may be transmitted in location 304. In some implementations, if location information cannot be transmitted (e.g., the vehicle does not include a GPS receiver), location 304 may be filled with zeros or otherwise solid to indicate that no location is provided.

[0043] Payload type 306 may indicate the type of information provided in the packet's payload 308. In the example of packets 1-6, payload type 306 of packet 1 indicates "HEARTBEAT" or "BEAT," which may be a signal periodically transmitted by "Vehicle_100." In some implementations, "HEARTBEAT" or "BEAT" may indicate that the packet is similar to a beacon, and basic information may be provided in payload 308. In some examples providing basic vehicle or infrastructure information, Payload 1 may include the transmitter's dither rate or other operating parameters, and / or the vehicle's trajectory or other status information. Alternatively, no information may be provided in the payload 308 of such a packet (e.g., the payload is filled with zeros, and the packet merely provides location information via location 304 and notifies other devices of the transmitter's presence).

[0044] The payload type 306 of Packet 2 is "RENDER_DESCRIPTION." As explained in the use case below, the LIDAR device 100 can measure surfaces within the line of sight (LoS) of the LIDAR device 100, but surfaces that are not within the LoS of the LIDAR device 100 may not be sensed or detected. In this way, a rendering of the environment based on measurements from the LIDAR device 100 may not include portions of the vehicle that are not within the field of view of the LIDAR device 100. "RENDER_DESCRIPTION" may indicate that basic information about the rendering of a vehicle or infrastructure may be provided in the payload 308. For example, Payload 2 may include information about the "Vehicle_100" being rendered. The dimensions may include dimensions of one or more associated shapes. In one implementation, the dimensions may include the orientation and dimensions of a geometric shape (such as a rectangle for a two-dimensional rendering or a rectangular prism for a three-dimensional rendering). Other suitable rendering information may include the texture, contours, or other features of the rendering of the object shown by payload 308. For example, an emergency vehicle or road construction zone may be highlighted in the rendering displayed by the vehicle to the driver and / or passengers. Thus, payload 308 may present a texture to highlight the portion of the rendering related to the emergency vehicle or construction zone. In another example implementation, similar textures may be used for vehicles within the same fleet. In this way, the driver and / or passengers may easily identify fleet vehicles in the displayed rendering. However, any suitable rendering description may be used, and the disclosure is not limited to the presented example.

[0045] In addition to or as an alternative to "RENDER_DESCRIPTION," payload type 306 may indicate "RENDER_IDENTIFICATION," such as in Packet 4. In some implementations, a list of standard vehicles (or infrastructure) and their associated rendering information may be stored in a database (such as a lookup table or another suitable data set organized for access). For example, if the vehicle sending the packet is a 1994 GeoMetro, payload 308 (such as Payload 4) may indicate "1994 GeoMetro." The database may be used in determining rendering details for a particular vehicle (such as the rendering's dimensions and texture), and existing renderings may be extended with additional rendering details. The database may also include information about a particular vehicle, such as its length, acceleration capabilities, etc. The term "rendering" as used herein may refer to a line drawing, point cloud, depth map, image, texture, shading, or other information that can be visualized or displayed.

[0046] The payload type of Packet 5 is "GATED_ENTRY," which may indicate that Payload 5 indicates the type of vehicle that is authorized to enter "Tunnel_40." For example, "Tunnel_40" may be restricted to multi-occupancy vehicles or autonomous vehicles, which may be indicated in Packet 5. Thus, a vehicle including LIDAR device 100 may determine whether it is authorized to enter "Tunnel_40" based on the information in Payload 5.

[0047] The payload type of Packet 6 is "FEE_CHARGE," which may indicate that Payload 6 indicates the fee charged by "Toll Booth_30." For example, if the bridge toll is $5, Payload 6 may indicate that $5 will be automatically collected from the driver's or fleet's account when a vehicle passes through "Toll Booth_30." In some implementations, Payload 308 may include a combination of the fee to be charged and restrictions on vehicle entry. For example, a transmitter for a parking garage space may indicate the parking fee and the vehicles allowed in the space (e.g., designated handicapped parking, reserved for electric vehicle spaces, etc.). While a vehicle is parking in the space, the transmitter may update the parking fee based on the rate so that the driver is aware of the current parking fee.

[0048] The payload type of Packet 3 is "VEHICLE_LOCATOR," which may indicate that Payload 3 indicates the vehicle being found. For example, if "Vehicle_005" is offline, a vehicle in the fleet may send a packet attempting to locate "Vehicle_005." Thus, Payload 3 may include a "Vehicle_005" identifier to indicate which vehicle should be located. In some implementations, LIDAR device 100 may use multiple packets received to locate a vehicle. The LIDAR device 100 may persist the transmitter ID. Thus, the LIDAR device 100 may be configured to search for the persisted transmitter ID in received packets to determine whether "Vehicle_005" has communicated with the LIDAR device 100. The LIDAR device 100 may then indicate this to "Vehicle_102." For example, the LIDAR device 100 may transmit the location received in the packet from "Vehicle_005" to "Vehicle_102." In some other implementations, the LIDAR device 100 may be configured to propagate a message to other vehicles or infrastructure indicating that "Vehicle_005" should be located. In this manner, if "Vehicle_005" receives a propagated message from another vehicle or infrastructure, "Vehicle_005" may decide to communicate its location to the fleet.

[0049] Other suitable payload types and payloads may exist (such as in different use cases for LIDAR communication as described herein), and the disclosure is not limited to the examples provided. In some example implementations, a vehicle may include multiple sensors to collect information for packet generation. For example, the payload may include information regarding the number of occupants, the vehicle's operating mode, etc., and sensors such as pressure sensors, occupancy sensors, engine sensors to detect the operating mode, etc. may be used to collect such information.

[0050] A wide range of information may be provided and received via LIDAR communications, which may be applied to a variety of use cases as described herein.

[0051] LIDAR communication implementation Vehicle Locator A vehicle may operate in an area where there is no cellular network for communication. For example, if the vehicle traverses a rural or sparsely populated area, the vehicle may not have a stable cellular connection or may otherwise be unable to communicate with a base station. In another example, the vehicle's cellular modem may be inoperable. If the vehicle is part of a fleet of vehicles (such as a taxi or rental car fleet), the dispatcher may be unable to determine the vehicle's location, and the dispatcher may desire to find the vehicle.

[0052] In some implementations, searching for missing, stranded, or offline vehicles may be a proactive measure, where a vehicle or infrastructure that finds an offline vehicle reports the offline vehicle to a dispatcher. In other implementations, searching for an offline vehicle may be a reactive measure, where a vehicle records the last known location of an offline vehicle. In this way, when the vehicle synchronizes with a dispatcher or the rest of the fleet (such as when the vehicle is charging or otherwise not operating), the vehicle reports the results.

[0053] Other vehicles in the fleet may be instructed to transmit vehicle locator messages via LIDAR communications. For example, each fleet vehicle may be instructed to periodically transmit a packet with a payload type of “Vehicle_Locator,” thereby identifying offline vehicles in the packet payload. The dispatcher may instruct infrastructure, such as fleet-owned electric vehicle charging stations or parking spaces that include or are coupled to LIDAR devices, to transmit vehicle locator messages. As described above, other vehicles may also propagate such received messages to increase the number of transmitters transmitting vehicle locator messages. In this manner, a vehicle that may be offline to its primary communication method (e.g., cellular) can receive a message from a passing vehicle or infrastructure, and the vehicle may decide to communicate its location to the dispatcher via LIDAR communications. In another example, a vehicle or infrastructure that finds an offline vehicle may communicate the offline vehicle's location to the dispatcher.

[0054] For example, if another fleet vehicle is in LIDAR communication with the offline vehicle, the offline vehicle may send a message via the LIDAR communication to the other fleet vehicle that its cellular modem is inoperable and to contact a dispatcher, for example, via the other vehicle's cellular modem. In this manner, the dispatcher may communicate with the offline vehicle via the other fleet vehicle.

[0055] In addition to or instead of dispatchers finding offline fleet vehicles, vehicle locator messages may be used in emergency situations such as stolen vehicles. The payload of a vehicle locator packet may include a description of the vehicle to be located, and such a packet may be propagated to other vehicles and infrastructure to locate the stolen vehicle. In some implementations, the vehicle may display a notification to the driver in response to receiving the packet. In other implementations, a stolen vehicle may be automatically identified based on the stolen vehicle's dimensions or other characteristics. For example, detected surfaces of other vehicles using a LIDAR device may be used to identify vehicles of the same make and model based on similar dimensions or other characteristics of the stolen vehicle. In another example, after detecting a missing vehicle, a visible light camera may be used to capture an image of the license plate, which may be analyzed to identify the missing vehicle based on the license plate number. The vehicle may report the location of the missing vehicle to a central office (e.g., via a cellular modem).

[0056] In some other implementations, vehicle locator messages may be initiated by an offline or disabled vehicle. For example, a fleet vehicle's cellular modem may become inoperable, preventing the fleet vehicle from communicating with the fleet via the cellular network. If a dispatcher provides information about where the fleet vehicle should go via cellular communications, the fleet vehicle may be unable to determine where the dispatcher wants the vehicle to go. In the example of a taxi fleet of autonomous vehicles, the dispatcher may want to send a vehicle to pick up a customer at a specific address and drive the customer to a specific destination. However, the vehicle is unable to receive instructions from the dispatcher. LIDAR communications may be used to provide messages from a vehicle (such as a vehicle with a disabled cellular modem) to the dispatcher. For example, a vehicle may send a message intended for the dispatcher via LIDAR communications to other fleet vehicles. The other fleet vehicles may then send the message to the dispatcher. In this way, the dispatcher is notified that the vehicle may not be able to receive messages from the dispatcher, and the dispatcher may update the processing request without including the vehicle in the fleet. The dispatcher may also provide instructions to the vehicle via LIDAR communications (e.g., via other fleet vehicles) to wait for service at a designated location, remove itself from fleet operation, or other appropriate operation.

[0057] In some implementations, a vehicle may have a home location away from a taxi fleet dispatcher. For example, if a vehicle has no place to go (e.g., because of a passenger communicated by a dispatcher), the vehicle may return to a designated location within the service area, such as a parking space for a particular vehicle within the service area. The space may be associated with a LIDAR device that can communicate with the vehicle (e.g., allowing the dispatcher to communicate with the vehicle). When the vehicle is fully operational and back in standard communication with the dispatcher, the designated location may be a waiting area for the vehicle until the vehicle is activated for a passenger. If the vehicle cannot communicate with the dispatcher and is not being used for a passenger, the vehicle may also return to a designated location (e.g., a parking space). References herein to offline vehicles may refer to stranded, missing, or other vehicles that are not operating or cannot be expected to operate within a default, typical, or standard mode of operation.

[0058] FIG. 4 illustrates an example parking space for an autonomous fleet vehicle 402 to return to when not in use. Illustrated is an environment 400 including a parking space 410. An autonomous vehicle 402 includes one or more LIDAR devices, such as LIDAR device 404. A stand 408 may be located near the parking space 410. Examples of the stand 408 include a parking meter, a charging station, or a taxi stand for pedestrian clients. In some implementations, the stand 408 includes a LIDAR device for communicating with the vehicle 402 via LIDAR communication 406. The stand 408 also includes a wired or wireless backhaul (such as a cellular modem or fiber optic connection). A vehicle 402 that cannot communicate directly with a dispatcher may communicate with the stand 408 via LIDAR communication, and the stand 408 may communicate with the dispatcher via the backhaul. While the vehicle 402 is described as communicating with the stand 408 via LIDAR communication, in some other embodiments, additional or alternative communication systems may be used. Other systems may include wireless local area network systems (such as IEEE 802.11-based systems), Bluetooth® systems, cellular systems (such as 3G, 4G, 5G), visible light communication systems, near field communication (NFC) systems, and the like.

[0059] When communicating with the stand 408, the vehicle 402 may provide a vehicle locator packet to the stand 408. In some implementations, sending the vehicle locator packet may indicate to the stand 408 and the dispatcher that the vehicle 402 is unable to communicate directly with the dispatcher (e.g., via a cellular modem). In some examples, the dispatcher may remove the vehicle 402 from service, send maintenance personnel to the vehicle 402, tell the vehicle 402 where to go for maintenance, or communicate a new passenger to be handled by the vehicle 402. In some implementations, the stand 408 may also communicate with vehicles 402 passing by on the road. If the vehicle 402 is unable to communicate directly with the dispatcher, the stand 408 may update the dispatcher with the location of the vehicle 402.

[0060] Vehicle call In addition to locating the vehicle, LIDAR communications can be used to summon the vehicle. Returning to FIG. 4 , stand 408 may be a pedestrian taxi stand. A person can walk up to stand 408 and input a desired destination (e.g., via a smartphone app, a graphical user interface on stand 408, or another suitable interface with stand 408). Multiple automated vehicles may line up at a location away from stand 408 (e.g., in a parking lot around the corner) and communicate with another LIDAR transmitter (e.g., one or more stands within the parking lot). Parking space 410 may be the location from which the next vehicle in the queue is summoned, or parking space 410 may be the location where a user enters a vehicle and boards the requested ride.

[0061] The stand 408 can handle multiple ride requests simultaneously. As a result, the stand 408 can summon multiple vehicles (e.g., using one or more LIDAR transmitters in a staging area for lined-up vehicles). Each summoned vehicle may be assigned a particular client or ride, and as the vehicle approaches the parking space 410, the vehicle may indicate its assigned ride to the stand 408 (via LIDAR communication 406). In this manner, the stand 408 may notify customers of the ride assigned to their vehicle in the parking space 410 to expedite boarding and coordinate rides for multiple customers from the same parking space 410.

[0062] notification LIDAR communications can be used to provide notifications to vehicles or drivers. For example, vehicles may be notified via LIDAR communications of approaching emergency vehicles, road hazards, school routes during school hours, speed limit changes, construction zones, etc. In the case of emergency vehicles, emergency broadcasts are communicated by infrastructure and vehicles that have previously received the broadcast, and the emergency vehicle is notified via LIDAR communications. Their presence can be communicated to other vehicles.

[0063] 5 illustrates an example environment 500 for communicating the presence of an ambulance 502 to a vehicle 506. The vehicle 508 may detect the ambulance 502. For example, the ambulance 502 may broadcast an emergency signal indicating the presence of the ambulance 502. In another example, a LIDAR device or other sensor on the vehicle 508 may detect the ambulance 502, and the vehicle 508 may identify the ambulance 502 as an emergency vehicle.

[0064] In some examples other than ambulance 502, vehicle 508 may detect or be notified of a person in the environment that may cause other vehicles to change course. For example, vehicle 508 may detect one or more bicyclists in a lane, children playing nearby in the street, children or other pedestrians crossing the street (e.g., before or after work), etc. In other examples, vehicle 508 may detect a stalled car, a minor traffic accident (a "minor crash"), a road closure, or other traffic situation that may cause other vehicles to change course or receive notification of such an accident. In such a scenario, the vehicle may slow down in the area and adjust its scan of the environment using one or more LIDAR devices to focus on a particular area, ensuring, for example, that it changes lanes to provide more space for the detected person, situation, etc.

[0065] Vehicle 508 may not be within LIDAR communication range with vehicle 506. For example, LIDAR communication may be based on LoS, and vehicle 506 may be blocked or otherwise obstructed by a building, preventing vehicle 508 from communicating directly with vehicle 506. In some implementations, vehicle 508 may use one or more vehicles and / or infrastructure to conduct LIDAR communication with vehicle 506 (e.g., communicate around a "corner"). For example, vehicle 508 may use LIDAR communication 510 with vehicle 504 to indicate the presence of ambulance 502. Additionally or alternatively, vehicle 508 may communicate with stand 514 via LIDAR communication 516 to indicate the presence of ambulance 502.

[0066] While FIG. 5 illustrates direct LoS communication between the LIDAR devices (510, 512, 516, and 518), LoS communication between the LIDAR devices may include one or more signal reflections from surfaces. For example, a focused beam of optical signals from a transmitting LIDAR device may be intentionally reflected off one or more building walls, signs, or other objects in the environment 500, and the reflections may be received by the receiving LIDAR device. If the beam is sufficiently focused, the transmission contains sufficient transmit power, and one or more surfaces have sufficient reflectivity, the reflections received by the receiving LIDAR device may be processed and determined to be communications transmitted by the transmitting LIDAR device (such as an announcement of the presence of ambulance 502). In this manner, the LIDAR devices may communicate with each other without a direct LoS. In this specification, LoS may include a direct LoS or an indirect LoS (which may include one or more reflections).

[0067] Vehicle 504 or stand 514 may be configured to indicate the presence of ambulance 502 or other traffic conditions to other vehicles and infrastructure within LIDAR communication range. For example, vehicle 504 may indicate the presence of ambulance 502 using LIDAR communication 512. In another example, stand 514 may indicate the presence of ambulance 502 using LIDAR communication 518. In this example, indicating the presence of ambulance 502 may involve two hops to get from vehicle 508 to vehicle 506 (e.g., via LIDAR communications 510 and 512 or via LIDAR communications 516 and 518).

[0068] In some implementations, the indication of an emergency vehicle (or other notification) may be propagated a defined number of hops or a defined distance. In this way, notification of the presence of the emergency vehicle to other vehicles is limited to the area surrounding the emergency vehicle or traffic situation. In some examples, vehicles Vehicle 508 may also determine the trajectory, route, or other information of the ambulance 502 and display such information to vehicle 504 or stand 514. The distance or number of hops for propagating instructions may, for example, increase in the direction of the route (or conversely, decrease in the opposite direction of the route). If vehicle 506 is an autonomous vehicle, vehicle 506 may proactively pull over, stop, change lanes, or change route in response to receiving an indication of the presence of ambulance 502 or other traffic conditions. In some other implementations, vehicle 506 may provide visual or audio notifications to the driver (and / or passengers) indicating the presence of ambulance 502 or traffic conditions. For example, a speaker or display on the vehicle may notify the driver and / or passengers of the presence of ambulance 502 or traffic conditions.

[0069] Another example notification relates to available parking spaces in a parking lot or garage. In some implementations, the entrance gate of the parking garage may communicate available spaces to an entering vehicle via LIDAR communication. If the vehicle is autonomous, the vehicle may be assigned a parking space and automatically proceed to and park in the space. In this way, the parking garage or parking lot may efficiently manage parking for entering and exiting vehicles. In other implementations, the entrance gate may indicate available parking spaces to the vehicle, and the vehicle may notify the driver and / or passengers of the locations of the available spaces. For example, the vehicle may display a map of the parking garage and indicate the locations of the available spaces on the displayed map. The driver and / or passengers may be notified before the vehicle enters the parking garage (e.g., as the vehicle approaches the garage), and the driver and / or passengers may select a preferred space.

[0070] As noted above, a further example notification may be regarding a traffic accident or other obstruction to traffic. The notification may be propagated via LIDAR communications to a vehicle preceding the obstruction, which may determine an alternate route and / or notify a driver and / or passengers of a possible delay based on the received notification. Other suitable notification examples may also be transmitted via LIDAR communications, and the disclosure is not limited to the above examples.

[0071] Access area restrictions Another use case for LIDAR communication is restricting access to defined areas. Some areas may be restricted to certain vehicles or vehicle types. For example, an ambulance entrance at a hospital may be restricted to ambulances. In another example, disabled parking spaces may be restricted to vehicles with a disabled tag. In a further example, HOV lanes may be restricted to vehicles with, for example, three or more occupants. In another example, a portion of a city center may be restricted to autonomous vehicles, low- or zero-emission vehicles, taxis, or other specific types of vehicles. In a further example, a company parking garage may be restricted to employee vehicles or vehicles with security clearance.

[0072] LIDAR communications may be used to indicate restrictions for particular areas and to grant access to particular vehicles to such areas. In some implementations, a LIDAR device in an area (such as an entrance gate, stand, or other infrastructure) may transmit the restrictions in that area to an approaching vehicle. The vehicle then communicates the necessary authorization (via its own LIDAR device) to the LIDAR device, indicating that the vehicle is authorized for the area. The vehicle may then enter the area (such as a gate opening, or a vehicle otherwise authorized to access the area) and be permitted to operate. In other implementations, the vehicle may store the necessary credentials for the area. In this way, the vehicle may preferentially communicate its credentials to the LIDAR device in order to gain access. For example, when an ambulance approaches the ambulance entrance of a hospital, the ambulance may use the LIDAR device to identify itself as a rescuer authorized to enter. It may be identified as an express vehicle.

[0073] Access restrictions to an area may change over time. For example, HOV lanes may be restricted only during designated congestion hours. In another example, if smog levels exceed a threshold, a city center may be restricted to zero-emission vehicles. In a further example, areas surrounding an accident or where police or emergency responders are needed may be restricted to such police or emergency response vehicles. Because LIDAR communications do not require a centralized network, changes to access restrictions may be implemented and provided to vehicles more quickly than, for example, remotely adjusting and establishing access parameters for an area via other communications. In some examples, a vehicle may notify the driver and / or passengers that it is entering a restricted area. In this manner, the driver, passengers, or the vehicle itself may adjust their route to avoid traveling around the restricted area.

[0074] In another implementation, an area may include different tolls or fees based on the time of day, day of the week, area congestion, special events, etc. Such tolls or fees may be communicated to the vehicle using LIDAR communications before the vehicle enters the area. Then, based on the information, the driver and / or passengers may decide to enter the area (or the vehicle may automatically decide). For example, some roads have hot lanes, and the price to enter such lanes is based on the time of day and road congestion. The driver and / or passengers may be notified of the price to enter the hot lane and may then decide whether to enter. Alternatively, the vehicle may be configured to enter the hot lane based, for example, on whether the price is lower than a threshold amount or a ride priority. For example, if a pregnant couple is traveling to the hospital, a ride priority may be set to override the price constraint for the hot lane. Other suitable use cases for restrictions may exist, and the disclosure is not limited to the examples provided.

[0075] Visualization or rendering of objects Another use case for LIDAR communication is to provide rendering or visualization information. As described with respect to FIG. 3, packets may include “RENDER_DESCRIPTION” or “RENDER_IDENTIFICATION” information for rendering or visualizing a vehicle (or infrastructure, such as stand 408 in FIG. 4). Rendering or visualization of objects in the vehicle's environment may be limited to the LoS of the LIDAR device connected to the vehicle. As a result, surfaces that are not within the LoS of the LIDAR device may not be detected and rendered.

[0076] 6A shows an example environment 600 of a vehicle 602. The environment 600 includes buildings 604 and 606 and vehicles 608 and 610. As shown, the view of the vehicle 608 from the LIDAR device 612 of the vehicle 602 includes the front and driver-side surfaces of the vehicle 608. The view of the vehicle 610 from the LIDAR device 612 is obstructed by the building 606. As a result, the view of the vehicle 610 includes only a portion of the front and driver-side surfaces. Furthermore, the view of the buildings 604 and 606 from the LIDAR device 612 is of surfaces facing the vehicle 602. The LIDAR device 612 may collect information about the depth of surfaces within the field of view of the LIDAR device 612, and the vehicle 602 may visualize or render the environment 600 including the detected surfaces. As used herein, a vehicle that visualizes or renders an environment may include one or more of a vehicle or infrastructure that instructs a display coupled to the vehicle or infrastructure (such as a tablet communicatively coupled to the vehicle's processing system, an in-dash entertainment system coupled to the vehicle's processing system, a display remote from the vehicle) to render at least a portion of the environment on the display, or a vehicle that renders at least a portion of the environment on an integrated display (such as via an integrated entertainment system in the center console of the vehicle). A vehicle or infrastructure may refer to a device or system coupled to the vehicle or infrastructure that performs an operation (such as generating a rendering to be displayed and providing the rendering to a display coupled to the vehicle or infrastructure).

[0077] 6B shows an example rendering 650 of environment 600 as sensed by LIDAR device 612. As described above, example rendering 650 may be displayed on a display integrated into the vehicle or coupled to a processing system of the vehicle. Rendering 650 includes renderings of a surface 654 corresponding to building 604, a surface 656 corresponding to building 606, a surface 658 corresponding to vehicle 608, and a surface 660 corresponding to vehicle 610. Example rendering 650 also includes a representation 652 of vehicle 602 to provide perspective. For example, the representation may be an icon, stock image, block, or other stored representation of the vehicle used in the rendering.

[0078] The example rendering 650 is simplified to illustrate aspects of the present disclosure, and the rendering 650 may include additional information or detail. For example, road lines and markers may be reflective and therefore detected via LIDAR. In another example, additional features of buildings 604 and 606 and vehicles 608 and 610 within the LoS of the LIDAR device 612 may be detected via LIDAR. While the example rendering 650 is shown as two-dimensional from a bird's-eye view, the rendering may be from any suitable orientation or any suitable perspective. For example, the rendering may be a three-dimensional point cloud as seen from the LIDAR device 612, a two-dimensional view of the environment as seen from the LIDAR device 612, or any other suitable rendering.

[0079] Example rendering 650 may be provided to the driver and / or passengers by vehicle 602 via a vehicle display. For example, example rendering 650 may be displayed on a display integrated into the vehicle, or the vehicle may provide rendering instructions to a display coupled to the vehicle's processing system. As shown in example rendering 650, depth information may not be determined for occluded surfaces of objects and buildings. For example, rendering 650 does not include information for the passenger-side surface and rear of vehicle 608, does not include information for the passenger-side surface, rear, and portions of the driver-side surface of vehicle 610, and does not include information for the sides of buildings 604 and 606 that do not face the vehicle 602.

[0080] In some implementations, another LIDAR device may transmit rendering information of one or more objects in environment 600 to vehicle 602. For example, LIDAR device 614 of vehicle 608 may transmit information about vehicle 608 for rendering, and LIDAR device 616 of vehicle 610 may transmit information about vehicle 610 for rendering. In some implementations, dimensions of shapes for rendering the vehicles may be transmitted. For example, dimensions of one or more rectangles for rendering vehicle 608 in rendering 650 may be transmitted from LIDAR device 614 to LIDAR device 612. Dimensions of one or more rectangles for rendering vehicle 610 in rendering 650 may be transmitted from LIDAR device 616 to LIDAR device 612. Other features related to the rendering of vehicle 608 and vehicle 610 may also be transmitted. Example features include textures, such as color, shading, highlights, etc., for the vehicle rendering.

[0081] In some other implementations, LIDAR devices 614 and 616 may transmit identifying information for their respective vehicles. For example, LIDAR device 614 may transmit information about the make and model of vehicle 608, the vehicle identification number of vehicle 608, the license plate number of vehicle 608, whether the vehicle is part of a particular vehicle fleet (such as the same vehicle fleet as vehicle 602), a unique identifier within the vehicle's fleet, or other identifying information.

[0082] Vehicle 602 (or a memory coupled to a display for displaying the rendering) may store rendering information for a particular vehicle, and vehicle 602 may use vehicle identification information to retrieve the stored rendering information for the particular vehicle. For example, vehicle memory (or other memory) may store rendering information for multiple vehicle makes and models (including vehicle 608 and vehicle 610), and such rendering information may be retrieved and used to enhance rendering 650 for environment 600. When rendering 650 is displayed on a display coupled to vehicle 602, vehicle 602 may provide the rendering information retrieved from vehicle memory, or vehicle 602 may provide information identifying the rendering information in external memory (such as providing make and model information to an entertainment system connected to the vehicle). Example rendering information may include a stock image or drawing of a particular vehicle. In some examples, the rendering information may be used to replace or combine corresponding portions of rendering 650. In other examples, adjusting the rendering may include highlighting object renderings. For example, the rendering may be updated to inform the driver or passengers of the condition of the emergency vehicle or the vehicle's environment.

[0083] FIG. 6C shows an example environment 670 for a vehicle 672. The example environment 670 includes buildings 684 and 686 and vehicles 674 and 676. The environment 670 also includes an emergency vehicle 678. FIG. 6D shows an example top-down rendering 690 of the environment 670 for the vehicle 672 of FIG. 6C. The rendering 690 includes renderings of a surface 694 corresponding to building 684, a surface 696 corresponding to building 686, and a surface 698 corresponding to vehicle 674. The example rendering 690 also includes a representation 692 of the vehicle 672 to provide perspective. The vehicle 676 and the emergency vehicle 678 may not be within the LoS of the LIDAR device coupled to the vehicle 672. As a result, the rendering 690 may not include a rendering of the vehicle 676 or a rendering of the emergency vehicle 678.

[0084] FIG. 7 shows a flowchart depicting example operations 700 for coordinating the rendering of an environment, such as a vehicle, a LIDAR device, infrastructure, etc. The example operations 700 are described below with respect to vehicles 602 and 672 of FIGS. 6A and 6C for illustrative purposes only. Those skilled in the art will recognize that the example operations 700 may be performed by any suitable device (e.g., a controller coupled to a vehicle), infrastructure, and / or vehicle according to various implementations, and that the example operations 700 described herein may be performed with additional steps, fewer steps, steps in a different order, parallel steps, or any combination thereof. As used herein, a vehicle 602 performing one or more steps may refer to a processing system coupled to the vehicle that performs one or more steps, a processing system integrated into the vehicle 602 that performs one or more steps, or other suitable embodiment for implementing the described method. For example, a controller may be located separately from the vehicle, or the controller may be otherwise coupled to the vehicle. Such a controller may also be coupled to one or more LIDAR devices, coupled to a display (to render or display other information), and / or coupled to one or more other input / output components (such as a speaker, a keypad, etc.) Vehicle 602 performing one or more steps may, in some implementations, mean that a controller performs the one or more steps.

[0085] Starting at 702, the vehicle 602 may receive data associated with a modulated optical signal emitted by a transmitter of a first LIDAR device. For example, a device may be coupled to the LIDAR device 612, and a modulated optical signal may be transmitted from the LIDAR device 614 or the LIDAR device 616 to the LIDAR device 612. The modulated optical signal includes an optical carrier signal modulated to include a data signal. The LIDAR device 612 or a device coupled to the LIDAR device 612 may extract the data signal and determine the data included in the data signal.

[0086] Vehicle 602 may also generate a rendering of the vehicle's environment based on information from one or more LIDAR devices coupled to the vehicle (704). For example, vehicle 602 may include a processing system for receiving measurements from LIDAR device 612 when LIDAR device 612 is in detection mode. The measurements may be used to generate a rendering, such as a point cloud, a depth map, or other suitable representation of environment 600. For example, the processing system may generate example rendering 650 of environment 600 (or example rendering 690 of environment 670) based on measurements received from LIDAR device 612 or other LIDAR devices coupled to the vehicle, where the rendering may include a representation of surfaces detected in environment 600 (or environment 670).

[0087] Based on the received data, the vehicle 602 may update its rendering (706). For example, the vehicle 602 may update its rendering of objects in the environment (708). In some example implementations, the vehicle 602 may receive an object identifier in the received data. For example, the LIDAR device 612 in communication mode may receive a packet containing an identifier for the vehicle 608 from the LIDAR device 614 via a modulated optical signal. The identifier may be a specific vehicle ID, a vehicle identification number, the make and model of the vehicle, or another suitable identifier in the packet.

[0088] The vehicle 602 may determine rendering information for the object based on the identifier. Example rendering information may include dimensions of one or more shapes in which the object will be rendered (e.g., one or more rectangles or other shapes in which a vehicle will be rendered), a representative image of the object (e.g., a representative drawing of the vehicle based on the vehicle type, vehicle make and model, etc.), a stock image of the object (e.g., a stock photo of the vehicle), or a texture to be applied to the rendering of the object (e.g., a texture to be applied to the rendering of the vehicle). The rendering information may also indicate whether to highlight or adjust (e.g., change size, dimensions, stretch, etc.) the rendering of the object. In some implementations, when the vehicle 602 receives an identifier (e.g., make and model) of the vehicle 608, the vehicle 602 may determine rendering information for the vehicle 608 (e.g., a stored image of the make and model of the vehicle 608) by scanning a database indexed by identifier (e.g., make and model) associated with the rendering information (e.g., a reference image of the vehicle, stored dimensions of the vehicle, highlighting information if it is an emergency vehicle, etc.). Vehicle 602 may then update the object rendering based on the determined rendering information. For example, rendered surface 658 in rendering 650 may be combined with images or other rendering information stored for vehicle 608 to update rendering 650. In another example, the rendered surface may be highlighted. In another example of updating an object rendering, the object rendering may be added to a rendering of the environment. For example, a rendering of vehicle 676 and / or a rendering of emergency vehicle 678 may be added to rendering 690.

[0089] In some other implementations, the LIDAR device may be configured to transmit a data signal that includes a specific description of how an object should be rendered. For example, the LIDAR device 614 of the vehicle 608 may transmit a rendering model, texturing, and any other characteristics for rendering the vehicle. In this manner, memory containing lookup tables, databases, etc. may not be used to determine object rendering information when updating the rendering of the environment.

[0090] 7, after updating the rendering, the vehicle 602 may provide the updated rendering for display (710). For example, a controller coupled to the vehicle 602 may provide the updated rendering to a display coupled to the vehicle 602, and the display may display the updated rendering to the driver and / or passengers. In another example, the rendering may be used to inform others about the environment of the vehicle 602. It may be provided on a remote display.

[0091] FIG. 8 shows an example rendering 800 of the environment 600 of FIG. 6. Rendering 800 may be an example of a rendering that adjusts rendering 650 of FIG. 6B based on rendering information for vehicles 608 and 610. For example, vehicle 602 may store representative images (stock images, line drawings, etc.) or other suitable rendering information for received identifiers of vehicles 608 and 610. LIDAR devices 614 and 616 may transmit vehicle identifiers to LIDAR device 612, and vehicle 602 may use the received vehicle identifiers to determine rendering information for vehicles 608 and 610 and adjust rendering 650.

[0092] The rendered surface 658 associated with the vehicle 608 may be combined with determined rendering information for the vehicle 608 (such as a stock image or stored drawing of the vehicle 608). For example, the image of the vehicle 608 may be aligned with the rendered surface 658 (such as by adjusting the size and / or orientation to align the image of the vehicle 608 with the rendered surface 658). In the example rendering 800, the object 808 corresponds to the adjusted rendering of the vehicle 608. Similarly, the object 810 in the rendering 800 corresponds to the adjusted rendering of the vehicle 610.

[0093] Although not shown, other objects that may be represented in rendering 650 may include infrastructure, buildings, road markings, pedestrians, or other non-vehicles for which the rendering may be adjusted. In some examples, LIDAR devices coupled to the vehicles may transmit information about objects in the environment other than the vehicles. For example, one or more LIDAR devices (such as LIDAR device 614) of vehicle 608 may be used to determine the depth of objects in the vehicle's environment, and one or more LIDAR devices (such as LIDAR device 616) of vehicle 610 may be used to determine the depth of objects in the vehicle's environment. In one example, each vehicle 602, 608, and 610 may generate a rendering based on measurements provided by its respective vehicle's LIDAR device. In some implementations, vehicles 602, 608, and 610 may share LIDAR measurements with each other (via LIDAR communication), and the combined LIDAR measurements may be used by vehicle 602 to generate a more comprehensive rendering of environment 600 than rendering 650.

[0094] In some implementations, any appropriate updated rendering may be displayed on the display of vehicle 602. For example, updated renderings may be generated for areas of the environment that are not yet within range or LoS of the LIDAR device of vehicle 602. Vehicle 610 may detect objects near the street behind vehicle 610 (e.g., using LIDAR device 616), which may be used to render environment 600 that is not within the field of view of vehicle 612. Vehicle 610 may transmit rendering information of at least a portion of its environment (e.g., rendering information of pedestrians, bicyclists, parked cars, traffic cones, obstructions, obstacles, etc.) to vehicle 602. In this manner, vehicle 602 may use the rendering information to include additional information about environment 600 (which is blocked by building 606 from the field of view of vehicle 602's LIDAR device 612) on the right side of rendering 650. In some aspects, rendering information may be shared among multiple vehicles and infrastructure, and a rendering of a particular vehicle (such as vehicle 602) may include an area well beyond the vehicle. For example, rendering information about the vehicle's environment from vehicle 610 over multiple hops may be transmitted over hops to vehicle 610 and then to vehicle 602. The rendering information may then be used by vehicle 602 to expand the area covered by the rendering generated by vehicle 602.

[0095] The received data is used to update renderings as well as for navigation or may be used for other automated operations. For example, the updated rendering may show additional obstructions or obstacles, so the vehicle may navigate (e.g., update its navigation) to avoid the additional obstacles during operation. Referring back to renderings 650 and 800 in FIGS. 6B and 8, respectively, rendering 650 does not show the rear of vehicle 610, while updated rendering 800 shows all of vehicle 610. If the route of vehicle 602 is determined to result in a collision with the rear of vehicle 610 as determined based on the received data, the navigation of vehicle 602 may be updated to avoid vehicle 610. In another example, the driver and / or passengers may be notified of a possible collision or obstruction based on the updated rendering. For example, an updated rendering including the obstruction or obstacle may be displayed to the driver and / or passengers, or the driver and / or passengers may be otherwise notified of the obstruction or obstacle. In this way, the vehicle occupant may be informed of a change in route, a slowdown of the vehicle, or other change in the autonomous vehicle's navigation before the obstruction or obstacle comes into the occupant's field of view. Additionally, or alternatively, a dispatcher or central office of the fleet that includes the vehicle may be notified of the obstruction or obstacle to explain why the vehicle's navigation should be adjusted.

[0096] In addition to updating rendering 650 to include a make or other representative image of a vehicle (or another object in the environment), rendering 650 may be updated to include a texture for that make. For example, renderings of vehicles in the same fleet may be coordinated to have similar textures to indicate that they belong to the same fleet.

[0097] FIG. 9A shows an example rendering 900 of the environment 600 of FIG. 6A , in which vehicle 608 and vehicle 602 belong to the same fleet. As shown, objects 902 and 908 in rendering 900 may be textured (e.g., shading) similarly. Object 910, which corresponds to vehicle 610 that is not part of the same fleet, may have a different texture than objects 902 and 908. For example, object 910 may include different shading than objects 902 and 908. Objects such as infrastructure, buildings, roads, and vehicles may each have different textures depending on the object type. For example, stands may be rendered using the same texture, emergency vehicles may be rendered using the same texture, and so on. In this manner, a driver and / or passengers may easily identify similar objects in the environment based on the texture of the displayed rendering. In some implementations, updating the rendering may include highlighting the rendering of the object. For example, the rendering of an emergency vehicle may be highlighted to notify the vehicle's occupants of the emergency vehicle's presence. Similarly, construction zones, safety zones, or other areas of the environment may be highlighted to notify vehicle occupants of their presence within the environment.

[0098] 9B shows an updated example rendering 950 of environment 670 (FIG. 6C). Updated rendering 950 includes a rendering 952 of emergency vehicle 678. Rendering 952 may be highlighted to indicate that vehicle 678 is an emergency vehicle. Rendering 952 may also include a representative image 954 included in vehicle 674 and a representative image 956 included in vehicle 676, which was not originally included in rendering 670.

[0099] 6C , vehicle 676 and emergency vehicle 678 may be blocked from the LoS of the LIDAR device coupled to vehicle 672. As a result, rendering 690 of FIG. 6D (based on measurements from the LIDAR device) does not include renderings of vehicle 676 and emergency vehicle 678. In some implementations, the LIDAR device coupled to vehicle 672 may receive a reflection of a modulated optical signal from a LIDAR device coupled to vehicle 676 or emergency vehicle 678. The modulated optical signal may include data indicative of the presence of emergency vehicle 678. The data may also include information such as the type of emergency vehicle, whether emergency vehicle 678 is responding to an emergency, and / or other information about the emergency vehicle, such as the route the emergency vehicle is traveling, its speed, etc. The data may also include information about other vehicles or objects. For example, the data may indicate the presence of vehicle 676.

[0100] In some other implementations, vehicle 672 may receive data regarding emergency vehicle 678 from an intermediate device. Vehicle 676 may detect the presence of emergency vehicle 678 (e.g., via a LIDAR device coupled to vehicle 676). Vehicle 676 may transmit data regarding emergency vehicle 678 to vehicle 674 within its LoS via modulated optical signal 680. Vehicle 674 may then transmit data regarding emergency vehicle 678 to vehicle 672 within its LoS via modulated optical signal 682. While one intermediate node is shown for transmitting the data, any number of hops may be performed by the data. Once vehicle 672 receives the data regarding emergency vehicle 678 (and data regarding vehicle 676), vehicle 672 may update rendering 690 to highlight or otherwise indicate the presence of emergency vehicle 678 as well as fill in the blockage caused by building 684. In addition to or instead of highlighting the emergency vehicle, the vehicle 672 may otherwise notify the driver and / or crew by, for example, flashing the display, providing an audio notification, applying a special texture to the rendering, and / or providing a text notification on the display. Although the examples in Figures 6C, 6D, and 9B are described with respect to an emergency vehicle (such as an ambulance), the actions may be applied to any vehicle, object, or area of ​​interest (such as a construction zone, school zone, accident area, etc.).

[0101] In some implementations, the rendering information for objects in the environment is stored in software The rendering information may be provided via a software-as-a-service (SaaS) service. For example, a unique identifier for the object may be received by vehicle 602. Vehicle 602 may then communicate with the service (e.g., via a cellular modem or other communication means) to obtain rendering information for the object. For example, vehicle 602 may receive a vehicle identification number for vehicle 608, and vehicle 602 may communicate with a remote server that stores rendering information for multiple objects, including vehicle 608. The remote server may provide the rendering information for vehicle 608 in response to a request, and the rendering information may be used by vehicle 602 to update the rendering.

[0102] In some other implementations, the vehicle 602 may store rendering information for objects, and a remote server may provide rendering information for objects not stored locally for the vehicle 602. Additionally, or instead, the remote server may be used to update the locally stored rendering information. For example, if a new make and model of a vehicle is released, rendering information for the vehicle may be created but not yet stored in the memory of the vehicle 602. LIDAR communications may be used to update the locally stored rendering information of the vehicle 602 to include the new rendering information. In some implementations, the vehicle 602 may update its stored rendering information when coupled to a stand with a dedicated backhaul (such as stand 408 in FIG. 4 ).

[0103] Other use cases for LIDAR communication exist, and the present disclosure is not limited to the above examples. For example, a device (such as a vehicle) may include software or firmware that periodically requires updating. A LIDAR device may be used to receive software updates from other devices. For example, a software update containing rendering information for a new make and model of vehicle may be provided to a first subset of vehicles. These vehicles may then transmit the update data to other vehicles in the environment via LIDAR communication. In this manner, the update data may be distributed quickly without all vehicles needing to connect to a central location (such as a server that stores the updates). For example, communication between devices (such as vehicles and infrastructure) may resemble a peer-to-peer network, allowing update data to be propagated between devices without the need for a central repository for downloading the update data. Another Exemplary Use Case This may include audio or video transmission between devices (such as voice calls, messaging, video conferencing, etc.).

[0104] In general, one or more actions may be based on data received via modulated optical signals for LIDAR communications. As described herein, LIDAR communications may be used to provide data to, for example, affect navigation of an autonomous vehicle, provide notifications to a driver or passengers, coordinate access to locations, etc.

[0105] 10 illustrates a flowchart depicting example operations 1000 for performing one or more operations based on data received in a transmission from a LIDAR transmitter. For purposes of describing aspects of the present disclosure, FIG. 10 is discussed in terms of an apparatus performing one or more operations. The operations 1000 may be performed by a vehicle, infrastructure, a dispatcher or central office coordinating a fleet of vehicles, a processing system (such as an entertainment system) coupled to a vehicle, or an appropriate control system (such as a control system coupled to a vehicle for Level 4 autonomy). The example operations 1000 are not limited to being performed by or performing the operations of a particular apparatus.

[0106] Starting at 1002, a device may receive data associated with a modulated optical signal emitted by a transmitting LIDAR device. For example, a device (such as a vehicle, a processing system, etc.) may be coupled to a LIDAR device configured to receive a modulated optical signal from a separate transmitting LIDAR device. A receiver of the receiving LIDAR device may receive the modulated optical signal, which may include an optical carrier signal modulated to carry a data signal including data. The carrier signal may be a signal typically used for detection and ranging. The receiving LIDAR device (or a device coupled to the receiving LIDAR device) may demodulate the modulated optical signal to generate and provide the data provided in the modulated optical signal. In this manner, the device (such as a processing system or a vehicle) may receive data associated with the modulated optical signal. In some implementations, the data may include one or more packets. Referring back to FIG. 3, an example packet may include information specific to, for example, locating the vehicle, using the vehicle to access a particular area or location, object rendering, etc.

[0107] Referring back to FIG. 10 , the device may determine one or more actions to perform based on the received data (1004). For example, the device may determine one or more actions based on the received one or more packets. In some implementations, the device may determine one or more vehicle locator actions (1006). As an example, if a vehicle receives information in the data that it should locate its vehicle, the vehicle may determine to return to a specified location (such as a parking space including a stand as shown in FIG. 4 ) and communicate the vehicle's location and current status to the stand (via LIDAR communications). In another example, if a device receives information about a missing vehicle and the device recently communicated with the missing vehicle, the device may communicate the last received location of the missing vehicle and other information about the missing vehicle, such as vehicle, infrastructure, dispatch, etc. Additionally or alternatively, the device may propagate a missing vehicle message to other vehicles and devices via LIDAR communications.

[0108] In some other implementations, the device may determine 1008 a hail action for one or more vehicles. Referring back to FIG. 4 , if stand 408 is a pedestrian stand for a fleet of automated taxis, stand 408 may communicate with the vehicle via LIDAR communications that the vehicle has been assigned to a particular client and ride. The vehicle may receive data from stand 408 via LIDAR communications between a LIDAR device coupled to the vehicle and a LIDAR device coupled to the stand and located in another section where the vehicle is lined up. Based on the received data, the vehicle may determine 1008 a hail action for one or more vehicles near stand 408. The vehicle or device may approach the space and communicate information regarding the ride or client assigned to the vehicle with the stand 408. The stand 408 may notify the client that the vehicle has arrived so that the client can board. In another example, the vehicle or device may determine that a driver (such as a taxi driver) should be notified about the client and / or ride assigned to the vehicle. The taxi driver may approach the client based on the notification to begin the ride.

[0109] Referring back to FIG. 10 , in some further implementations, a device may determine one or more notification actions (1010). In some examples, referring back to FIG. 5 , a vehicle 506 may receive data that an ambulance 502 or other emergency vehicle is nearby. The vehicle 506 may pull up to a curb, stop, or otherwise adjust the vehicle's operation or travel to allow the ambulance 502 to pass safely. Additionally or alternatively, the vehicle 506 or another device may decide to notify the driver and / or passengers of the presence of the ambulance 502. In some other examples, the vehicle 506 or device may decide to indicate the presence of the ambulance 502 to other vehicles, infrastructure, or other devices via LIDAR communications (thereby propagating instructions to other vehicles and infrastructure in the area of ​​the ambulance 502).

[0110] In another example of a notification operation, a vehicle may receive data regarding available parking spaces in a parking garage. Thus, the vehicle may navigate to and park in one of the available spaces. The vehicle or device may decide to notify the driver and / or passengers of available spaces, etc. (e.g., by showing the available spaces on a displayed map of the parking garage). Other example notification operations may include updating the route based on notifications of accidents or traffic along the current route, notifying other vehicles of the accident or traffic via LIDAR communications, notifying the driver and / or passengers of changes in route or traffic conditions, etc.

[0111] Referring back to FIG. 10 , in some other implementations, the device may determine operation in one or more restricted access areas (1012). For example, if the city center is restricted to autonomous vehicles, zero-emission vehicles, or other specific types of vehicles, the vehicle may receive data related to the access restrictions from a LIDAR device coupled to the vehicle. The LIDAR device may receive data in a modulated optical signal transmitted by a LIDAR device coupled to the infrastructure or another suitable device. For example, a gate, toll booth, or toll booth may include a LIDAR device for transmitting access restriction instructions to the vehicle. The vehicle may determine whether the vehicle is authorized to enter the area based on the instructions. If the vehicle is authorized, the vehicle may also determine one or more navigation operations to access the area. For example, if access is restricted to autonomous vehicles, the autonomous vehicle may enter the city center. Additionally or alternatively, the vehicle may decide to notify the driver and / or passengers about approaching the restricted access area and the access restrictions. In some examples, the driver may decide whether to access the area based on the notification. In some further examples, a vehicle may be notified of a toll or fee for accessing a particular area. For example, a bridge, lane, or area in a city may be associated with an access fee. Before accessing infrastructure in a limited area (such as providing payment at a toll booth before crossing a bridge), the vehicle may provide account or other payment information via LIDAR communication. In some examples, the fee may be communicated to the driver and / or passengers.

[0112] As restrictions to an area may change based on current conditions (time of day, day of week, current traffic congestion, smog levels, situations requiring area restrictions to emergency vehicles, etc.), changes to such restrictions may be communicated. In response, the vehicle may, for example, drive to change access restrictions to the area. The vehicle may decide to notify the driver and / or passengers, adjust its route to travel around the newly restricted area, or notify other vehicles of the change via LIDAR communications.

[0113] 10 , in some further implementations, the device may determine one or more visualization or rendering operations (1014). For example, the vehicle's environment, the LIDAR device, the infrastructure, or other suitable device may be rendered (e.g., generating a point cloud, mapping the environment, a depth map, etc.), and the rendering may be displayed to the driver and / or passengers. Referring back to FIGS. 6A and 6B , the vehicle 602 (or another suitable device coupled to the vehicle 602, such as an entertainment system, a processing system, etc.) may generate a rendering 650 from data provided by the LIDAR device 612. The vehicle 602 (or other suitable device) may also receive one or more identifiers of the vehicles 608 and 610 in transmissions from the LIDAR devices 614 and 616, respectively. Accordingly, vehicle 602 (or other suitable device) may determine rendering information for vehicles 608 and 610 based on the received identifiers (such as by looking up information in a vehicle or device memory, communicating with a remote service to provide rendering information for one or more objects, determining rendering information encoded in transmissions from the respective LIDAR devices, etc.). Accordingly, the rendering operation may include updating rendering 650 using the determined rendering information. For example, rendering 650 of FIG. 6B may be updated to rendering 800 of FIG. 8 based on representative images (e.g., models) of vehicles 608 and 610 used to update the rendering. Other example rendering operations may include applying a defined texture to emergency vehicles, applying similar textures to similar types of objects, applying a unique texture that is specific to each vehicle or object in the environment and associated with a unique identifier (such as unique transmitter ID 302 of FIG. 3), etc. In some examples, vehicle 602 or other suitable device may determine that an updated rendering of the environment should be displayed to the driver and / or passengers.The rendering operations may also include contacting a remote service to update a stored database of rendering information, updating the database based on receiving new rendering information (such as a new make or model), or propagating rendering information updates via LIDAR communications to other vehicles or devices in the environment.

[0114] Next, referring back to FIG. 10 , the device may instruct one or more actions to be performed (1016). For example, if one or more rendering actions are determined, the device may instruct a display or another device coupled to the display to display the rendering and display the updated rendering. In another example, if the device is a vehicle and the one or more actions are operations in a restricted access area, the vehicle may instruct one or more vehicle components to navigate the vehicle into or out of the area. The device may also provide instructions to the driver and / or passengers to display information about the restricted access area.

[0115] While portions of the example processing of the example operations 1000 of FIG. 10 are described from the perspective of a vehicle, any suitable device may perform one or more associated operations. For example, a station (or a processing system coupled to the station) may perform a vehicle locator operation (e.g., contacting a dispatcher) in response to receiving a LIDAR transmission from a missing vehicle. In another example, a station may perform a vehicle summoning operation (e.g., notifying a client that a vehicle has arrived) in response to receiving a LIDAR transmission from a summoned vehicle. In another example, a station may perform a notification operation (e.g., sending a notification of an ambulance in an area) in response to receiving a notification in a LIDAR transmission. In a further example, a toll booth or gate may perform a limited access area operation (e.g., changing access fees or access restrictions in an area) in response to receiving a traffic congestion update from LIDAR devices for infrastructure or vehicles in the area. As discussed, the example operations 1000 are not limited to being performed by any particular device (e.g., a vehicle).

[0116] Additionally, while example actions (such as calling a ride, restricting access, and updating renderings) are described individually, the device may be configured to perform any number of actions and combinations of actions. For example, the device may be configured to update renderings related to an emergency vehicle, notify vehicle occupants of the presence of the emergency vehicle, update areas to which the vehicle is restricted based on the presence of the emergency vehicle, and update the vehicle's navigation to avoid the restricted areas.

[0117] Those skilled in the art will understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof. Furthermore, those skilled in the art will understand that examples are provided for illustrative purposes when describing aspects of the present disclosure. Aspects of the present disclosure may be implemented by any suitable device (such as an autonomous vehicle, a fleet of autonomous vehicles, infrastructure, a control system for a vehicle or infrastructure, an entertainment system coupled to a vehicle, or other device) and are not limited to the specific examples herein.

[0118] Furthermore, those skilled in the art will understand that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the aspects disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, the various components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled engineers may implement the described functionality in various ways for each particular application, and such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure. For example, a processing system of a vehicle or other suitable device may include one or more processors and memory coupled to the one or more processors. The memory may include instructions executed by the one or more processors to cause the device to perform operations as described herein. The processing system may also be coupled to one or more LIDAR devices for LIDAR communication and surface detection and ranging. In some aspects, the processing system may include dedicated hardware, such as one or more integrated circuits, configured to perform one or more operations.

[0119] Thus, the methods, sequences, or algorithms described in connection with the aspects disclosed herein may be embodied directly in hardware, in software modules executed by a processor, or in a combination of the two. The software modules may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled (e.g., directly or remotely) to one or more processors such that the one or more processors can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to, the one or more processors.

[0120] In the foregoing specification, the example embodiments have been described with reference to specific example embodiments thereof. However, it will be apparent that various modifications and changes may be made thereto without departing from the broader scope of the implementation of the present disclosure, as set forth in the appended claims. For example, while the vehicles are generally shown as automobiles, any suitable vehicle, such as a motorcycle, drone, aircraft, watercraft, helicopter, or the like, may be used. Furthermore, a vehicle may include one or more LIDAR devices, and the vehicle may be coupled to LIDAR devices manufactured separately from the vehicle. The vehicle may also include one or more processing systems, entertainment systems, control systems, etc., and the vehicle may be coupled to such processing systems, entertainment systems, and control systems manufactured separately from the vehicle. For example, one or more LIDAR devices may be mounted on various parts of a vehicle to ensure coverage of the environment surrounding the vehicle. A LIDAR device may be coupled to a processing system in the vehicle (or a control system coupled to the vehicle) to provide data transmitted by other LIDAR devices for processing or to transmit data provided by the processing system to other LIDAR devices via modulated optical signals.

[0121] Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.

Claims

1. 1. An apparatus comprising: one or more processors; a memory coupled to the one or more processors and containing instructions, the instructions, when executed by the one or more processors, causing the one or more processors to: causing a transmitter to transmit a first modulated optical signal to a first light detection and ranging (LIDAR) device coupled to a vehicle, the transmitter being part of a second LIDAR device coupled to infrastructure, the first modulated optical signal including an optical carrier signal modulated to include a data signal; A device that performs an operation including the above.

2. The apparatus of claim 1 , wherein the infrastructure comprises at least one of a toll booth, a road construction area, a tunnel entrance, a traffic light, a charging station, an entrance gate, or a stand.

3. The apparatus of claim 1 , wherein the data signal includes transmitter identification information.

4. The apparatus of claim 1 , wherein the data signal includes an indication of a location of the infrastructure.

5. The apparatus of claim 1 , wherein the data signal includes operating parameters of the transmitter.

6. 10. The apparatus of claim 1, wherein the data signal indicates a restriction on entry to a restricted access region.

7. 7. The apparatus of claim 6, wherein the restricted access area comprises at least one of a tunnel, an ambulance entrance to a hospital, a handicapped parking space, a high-occupancy vehicle (HOV) lane, a parking lot, or a portion of a city.

8. The apparatus of claim 6 , wherein the infrastructure controls access to the restricted access area.

9. 10. The apparatus of claim 1, wherein the data signal indicates a fee for entering an area.

10. The apparatus of claim 9 , wherein the region includes at least one of a bridge, a lane, or a portion of a city.

11. The apparatus of claim 9 , wherein the infrastructure is a toll booth.

12. The apparatus of claim 1 , wherein the data signal includes information regarding at least one of a missing vehicle, a stolen vehicle, or an offline vehicle.

13. The apparatus of claim 1 , wherein the data signal summons the vehicle for a requested ride.

14. The apparatus of claim 1 , wherein the data signal includes a notification to the vehicle.

15. The device of claim 14 , wherein the notification is for at least one of an emergency vehicle, a road hazard, a school zone, a speed limit change, a construction zone, a traffic accident, or a traffic obstruction.

16. The apparatus of claim 1 , wherein the infrastructure is associated with a parking lot and the data signal indicates available parking spaces within the parking lot.

17. The apparatus of claim 1 , wherein the infrastructure is associated with a parking lot and the data signal indicates a fee charged for parking in the parking lot.

18. The operation is receiving data related to a second modulated optical signal emitted by the first LIDAR device and received by a detector, the detector coupled to the infrastructure, the second modulated optical signal responsive to the first modulated optical signal; The apparatus of claim 1 further comprising:

19. 1. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to: causing a transmitter to transmit a first modulated optical signal to a first light detection and ranging (LIDAR) device coupled to a vehicle, the transmitter being part of a second LIDAR device coupled to infrastructure, the first modulated optical signal including an optical carrier signal modulated to include a data signal; A non-transitory computer-readable medium for causing operations to be performed, including:

Images