Device based on light detection and distance measurement

Abstract

An apparatus based on light detection and ranging (LIDAR) is disclosed. The device may include one or more processors; and a memory coupled to the one or more processors. The memory includes instructions that, when executed by one or more processors, cause the one or more processors to perform operations. The operations include causing a transmitter to transmit a first modulated optical signal to a first light detection and ranging (LIDAR) device coupled to a vehicle, where the transmitter is coupled to an infrastructure, and where the first modulated optical signal includes an optical carrier signal modulated to include a data signal.

Description

[0001] This application is a divisional application of the application patent application with the application date of 2020-07-01, the application number of 202080061467.0, and the invention name of "Communication based on light detection and ranging". TECHNICAL FIELD

[0002] The present disclosure relates generally to LIDAR devices, and in particular to using LIDAR as a communication channel in self-driving applications. BACKGROUND

[0003] Light detection and ranging (LIDAR) devices can be used to detect objects in an environment. For example, a vehicle can include or be coupled to one or more LIDAR devices to detect, for example, other vehicles, pedestrians, traffic signals, obstacles, etc. A LIDAR device emits light at a particular frequency (e.g., at 800-1000 nm or at 1550 nm), and the LIDAR device receives reflections of the emitted light. The LIDAR device then determines a time of flight (ToF) of the light to estimate distances of multiple reflecting surfaces as the environment is scanned. The estimated distances can be used to generate a point cloud representation of the environment or otherwise used to render the environment or assist in operation of the vehicle.

[0004] For example, a vehicle can be configured to operate in an autonomous mode in which the vehicle navigates through an environment with little or no input from a driver. An autonomous vehicle can include one or more LIDAR devices to determine distances of objects in the environment, and the distances can be used to control navigation of the vehicle. In another example, a vehicle can include one or more LIDAR devices to assist a driver in, for example, performing adaptive cruise control, providing cross-traffic warnings, providing lane departure warnings, etc., all the way up to performing all safety-critical driving functions and monitoring road conditions in a fully autonomous system. SUMMARY

[0005] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0006] Innovative aspects of the subject matter described in this disclosure can be implemented with respect to a device coupled to one or more LIDAR devices. In some implementations, an example device includes one or more processors and a memory coupled to the one or more processors. The memory includes instructions that, when executed by the one or more processors, cause the device to receive data associated with modulated light signals emitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to a vehicle and the device, generate a rendering of an environment of the vehicle based on information from one or more LIDAR devices coupled to the vehicle, and update the rendering based on the received data. Updating the rendering includes updating an object rendering of an object in the environment of the vehicle. The instructions cause the device to provide the updated rendering for display on a display coupled to the vehicle.

[0007] Innovative aspects of the subject matter described in this disclosure can be implemented in a computer-readable medium. The computer-readable medium stores instructions that, when executed by one or more processors of a device, cause the device to receive data associated with modulated light signals emitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to a vehicle and the device, generate a rendering of an environment of the vehicle based on information from one or more LIDAR devices coupled to the vehicle, and update the rendering based on the received data. Updating the rendering includes updating an object rendering of an object in the environment of the vehicle. The instructions cause the device to provide the updated rendering for display on a display coupled to the vehicle.

[0008] Innovative aspects of the subject matter described in this disclosure can be implemented as a method. An example method includes receiving, by a device, data associated with modulated light signals emitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to a vehicle and the device, generating a rendering of an environment of the vehicle based on information from one or more LIDAR devices coupled to the vehicle, and updating the rendering based on the received data. Updating the rendering includes updating an object rendering of an object in the environment of the vehicle. The method also includes providing the updated rendering for display on a display coupled to the vehicle.

[0009] Another example device can include one or more processors and a memory coupled to the one or more processors. The memory comprises instructions that, when executed by the one or more processors, cause the device to receive data associated with a modulated light signal emitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to infrastructure (such as a toll booth, a road work zone, a tunnel entrance, etc.) and the device. The device can generate a rendering of an environment of the infrastructure based on information from the one or more LIDAR devices coupled to the infrastructure. The device can also update the rendering based on the received data. Updating the rendering can include updating an object rendering of an object in the environment of the infrastructure. The device can also provide the updated rendering for display. For example, the rendering can be displayed for a toll booth attendant, a central traffic office auditor, a construction worker bot at a construction site, etc.

[0010] The data can include an indication of an object in the environment, and updating the object rendering can include highlighting the object rendering during display, adjusting a texture of the object rendering, including a representative image of the object in the rendering of the environment, and / or adjusting a dimension of the object rendering. Displaying the updated rendering can include notifying a viewer that the object in the environment of the infrastructure is an emergency vehicle.

[0011] The device can also determine a navigation operation for one or more vehicles in the environment of the infrastructure based on the received data, and the device can provide the adjusted navigation operation to the one or more vehicles. The modulated light signal can be received by the second LIDAR device operating in a communication mode, and the object in the environment of the infrastructure and a line of sight of the second LIDAR device can be sensed by the second LIDAR device operating in a detection mode.

[0012] The device can also receive second data associated with a second modulated light signal received by the second LIDAR device, and update one or more entries in a local database based on the received second data. In some implementations, the device can transmit the updated data to other LIDAR devices to update databases local to the other LIDAR devices.

[0013] According to another aspect of the disclosure, a device is also provided. The device includes one or more processors and a memory coupled to the one or more processors. The memory comprises instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including causing a transmitter to transmit a first modulated light signal to a first light detection and ranging (LIDAR) device coupled to a vehicle, wherein the transmitter is coupled to infrastructure, and wherein the first modulated light signal includes an optical carrier signal modulated to include a data signal.

[0014] According to another aspect of the disclosure, an apparatus is also provided. The apparatus includes one or more processors and memory coupled to the one or more processors comprising instructions that, when executed by the one or more processors, cause the apparatus to receive data associated with a modulated light signal emitted by a first light detection and ranging (LIDAR) apparatus and received by a detector, wherein the first LIDAR apparatus is coupled to a vehicle, wherein the detector is coupled to an infrastructure, and wherein the modulated light signal comprises a light carrier signal modulated to include a data signal including data, determine one or more operations to perform based on the received data, and send a communication related to the one or more operations.

[0015] The details of one or more implementations of the subject matter described in this disclosure are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, the drawings, and the claims. Note that the relative dimensions of the following figures can not be drawn to scale. BRIEF DESCRIPTION OF DRAWINGS

[0016] FIG. 1 is a block diagram of an example LIDAR apparatus.

[0017] FIG. 2A An example LIDAR apparatus in an environment is shown.

[0018] FIG. 2B is an example timing diagram of waveforms corresponding to emitted light pulses and received light pulses of an example LIDAR apparatus.

[0019] FIG. 3 An example packet format from multiple received packets of a LIDAR transmission based on an example communication protocol is shown.

[0020] FIG. 4 An environment including example parking spaces returned by autonomous fleet vehicles when not in use is shown.

[0021] FIG. 5 An example environment for conveying the presence of an ambulance to another vehicle in an environment is shown.

[0022] FIG. 6A An example environment of vehicles is shown.

[0023] FIG. 6B An example rendering of an environment of vehicles sensed by a LIDAR apparatus is shown.

[0024] FIG. 6C An example environment of vehicles including an emergency vehicle is shown.

[0025] FIG. 6DShowing the data sensed by the LIDAR device FIG. 6C Example rendering of the vehicle's environment.

[0026] FIG. 7 A flowchart depicting an example operation of rendering for adjusting the environment of a LIDAR device is shown.

[0027] FIG. 8 Show FIG. 6A An example of an adjusted rendering of the environment.

[0028] FIG. 9A Show FIG. 6A An example rendering of an environment where two vehicles belong to the same convoy.

[0029] FIG. 9B Show FIG. 6C Example rendering of an environment, where the environment includes an emergency vehicle.

[0030] FIG. 10 A flowchart is shown depicting an example operation for performing one or more operations based on data received from a transmission from a LIDAR transmitter.

[0031] The same reference numerals and names in the various figures indicate the same elements. Detailed Implementation

[0032] Efforts have been made to couple vehicles and infrastructure via cellular communications. For example, some vehicles include cellular modulators to communicate with other vehicles or equipment via the 5 GHz radio spectrum. In another example, a vehicle may be equipped with dedicated short-range communication (DSRC) equipment to communicate with other vehicles or equipment via the 5.9 GHz radio spectrum. One problem with cellular or radio technologies used for communications is that vehicles may require cellular connectivity or require another vehicle or infrastructure equipped with consistent communication technology. For example, some rural or saturated urban environments may not have available cellular connectivity, and vehicles may not be able to communicate with other vehicles via their cellular modems. Another problem with cellular technologies used for communications is the inherent latency associated with communications. For example, cellular communication between vehicles may require communication through one or more base stations in a cellular network, which can delay communication. Furthermore, the wavelength of the radio signals used for communication limits the communication bandwidth. In addition, cellular and radio signals are typically omnidirectional, and transmitting multiple omnidirectional signals over the air can quickly saturate an environment.

[0033] Many vehicles and infrastructure (e.g., toll booths, traffic signals, charging stations, etc.) can include or be coupled to one or more LIDAR devices. Moreover, as autonomous vehicles become more prevalent, so too will equipped LIDAR devices. In some aspects, in addition to performing ToF measurements (for detecting surfaces in the environment), LIDAR devices can be configured to communicate with other LIDAR devices. Because the emitted light signals have higher frequencies than radio signals, communication between LIDAR devices can have higher communication bandwidth than cellular communication. Moreover, communication between LIDAR devices does not require a cellular network or other infrastructure. Furthermore, light signals (e.g., signals with wavelengths near 1000 nm) can be emitted in a focused dispersive mode to prevent saturation and interference of multiple light signals transmitted in the air at the same time.

[0034] Implementations of the subject matter described herein can allow LIDAR devices to communicate with another LIDAR device (referred to herein as “LIDAR communication”). LIDAR communication can be used for vehicle-to-vehicle (V2V) communication or vehicle-to-infrastructure (V2X) communication, and LIDAR communication can occur between any compatible vehicles and / or infrastructure (e.g., within a vehicle fleet or between unrelated vehicles that include configured LIDAR devices). LIDAR communication can be used for a variety of situations and use cases, as described herein.

[0035] For the purpose of describing the innovative aspects of the present disclosure, the following description is directed to particular implementations. However, a person of skill in the art will readily recognize that the teachings herein can be applied in a multitude of different ways. The described implementations can be implemented in any device, system, or vehicle that includes or is coupled to one or more LIDAR devices. In some implementations, a “device” for performing the operations described herein can refer to a control device or system coupled to a vehicle and one or more LIDAR devices; a vehicle that includes a control device or system and is coupled to one or more LIDAR devices; a control device or system coupled to infrastructure or another non-vehicle system; or other suitable implementations. Similarly, a “vehicle” can refer to a control device or system coupled to the vehicle; a vehicle separate from a control device or system coupled to the vehicle; a combination of a control device or system and a coupled vehicle; or other suitable implementations.

[0036] In the following description, numerous specific details are set forth such as examples of specific components, systems, and processes to provide a thorough understanding of the present disclosure. Also, in the following description and for purposes of explanation, specific nomenclature and / or details are set forth to provide a thorough understanding of the example embodiments. However, it will be apparent to one skilled in the art that the example embodiments can be practiced without these specific details. In other instances, well-known circuits, systems and devices are shown in block form to avoid obscuring the present disclosure. Any signals provided over various buses described herein can be time-multiplexed in a single physical line and / or transmitted on one or more superimposed carrier frequencies. Further, the interconnection between circuit elements or software blocks can be shown as buses or single signal lines. Each of the buses can alternatively be a single signal line, and each of the single signal lines can alternatively be buses, and lines or buses can represent any one or more of a number of physical or logical mechanisms for communication between components. Additionally, the term "coupled" as used herein means coupled directly to or through one or more intervening components or devices.

[0037] FIG. 1 is a block diagram of an example light detection and ranging (LIDAR) device 100. The LIDAR device 100 can be used to detect object surfaces in an environment by emitting light pulses that illuminate the object surfaces and by detecting light pulses that are reflected from the object surfaces. The LIDAR device 100 can determine distances to objects based on a time delay between the emission of a light pulse and the receipt of a corresponding light pulse that is reflected from a selected object. This time delay can be referred to as the ToF of the light pulse, which can be multiplied by the speed of light to determine the distance between the LIDAR device 100 and the object. Multiple pulses can be used to determine distance information for multiple points associated with objects in the environment. These points can be used to generate a point cloud or can be used to determine the location, size, shape, pose, and motion of the objects. In some implementations, information from the LIDAR device 100 can be used to control, for example, an autonomous vehicle so that the autonomous vehicle can navigate to a destination in the environment while avoiding obstacles. The LIDAR device 100 can also be used to measure distances for driver-assist operations.

[0038] The LIDAR device 100 is shown to include a transmitter 110, a receiver 120, and a LIDAR controller 130. The transmitter 110 can include a transmit controller 111, one or more light emitters 112, and a transmit aperture 113. The light emitters 112 can emit one or more light pulses 125 that can be used to detect objects in the surrounding environment. The light emitters 112 can include any number of suitable light sources such as, but not limited to, laser diodes, light emitting diodes (LEDs), vertical cavity surface emitting lasers (VCSELs), organic light emitting diodes (OLEDs), polymer light emitting diodes (PLEDs), light emitting polymers (LEPs), liquid crystal displays (LCDs), microelectromechanical systems (MEMS), or any other device configured to selectively transmit or emit light pulses 125 of defined wavelengths. Source wavelengths can include, for example, the ultraviolet, visible, and / or infrared portions of the electromagnetic spectrum. In some aspects, the light emitters 112 can be disposed on one or more substrates (e.g., printed circuit boards (PCBs), flexible PCBs, etc.). Although the light emitters 112 are described herein as emitting light pulses 115, those of ordinary skill in the art will readily appreciate that the light emitters 112 can transmit or emit light signals, beams, photons, etc. Accordingly, the terms light pulse, light signal, light beam, and photon can be used interchangeably herein.

[0039] The transmit aperture 113 is coupled to the light emitters 112 and can include any suitable components (e.g., mirrors, lenses, diffractive gratings, exit apertures, etc.) that can focus, direct, and / or condition the light pulses 115 for emission into the surrounding environment. In some implementations, the transmit aperture 113 can be configured to steer the light pulses 115 in one or more designated directions relative to the LIDAR device 100. The designated directions can span a range of directions such that distances between the LIDAR device 100 and multiple objects (e.g., vehicles, people, roads, traffic signals, traffic signs, obstacles, etc.) can be determined based on reflections of the light pulses 115 caused by the objects.

[0040] The transmit controller 111 can control operation of the light emitters 112 and the transmit aperture 113 and can adjust a number of parameters or settings of the light emitters 112 and the transmit aperture 113 or both. In some implementations, the transmit controller 111 can be responsive to one or more control signals provided by the LIDAR controller 130. For example, the transmit controller 111 can adjust a width, timing, frequency, and / or amplitude (intensity) of the light pulses 115 emitted by the light emitters 112 based on the one or more control signals. In other implementations, the transmit controller 111 can be omitted or can be included within the LIDAR controller 130.

[0041] Receiver 120 can include a plurality of photodetectors 121, detector circuitry 122, and an analog-to-digital converter (ADC) 123. Photodetectors 121 can receive light pulses 125 (e.g., photons) from the surrounding environment. In some implementations, received light pulses 125 can include components of emitted light pulses 115 reflected from one or more objects in the surrounding environment. Photodetectors 121 can be configured to convert received light pulses 125 into photodetector signals (e.g., analog current signals) indicative of intensity levels of received light pulses 125. Photodetectors 121 can be any suitable component or device capable of receiving or sensing light, including, for example, photodiodes (e.g., avalanche photodiodes), silicon photomultipliers (SiPMs), phototransistors, cameras (e.g., CMOS sensors), active pixel sensors (APSs), charge-coupled devices (CCDs), cryogenic detectors, etc. In some implementations, photodetectors 121 are reverse-biased photodiodes that, for example, generate a current in response to receiving a light pulse such that the amount of current through each photodiode is proportional to the intensity of the light pulse received by the photodiode.

[0042] Although not shown for simplicity, receiver 120 can include optics to filter the wavelengths of received light such that photodetectors 121 primarily receive light corresponding to the wavelengths of light pulses 115 emitted by transmitter 110 (and receive minimal light corresponding to other wavelengths). For example, receiver 120 can include a bandpass filter to filter out optical signals outside of a range of wavelengths centered on the fundamental wavelength of light pulses emitted by transmitter 110.

[0043] Detector circuitry 122 can sample the photodetector signals provided by photodetectors 121 using any suitable technique to determine intensity levels of received light pulses 125. In some implementations, detector circuitry 122 can sample the photodetector signals at a plurality of intervals or sampling times. In other implementations, detector circuitry 122 can sample the photodetector signals continuously. Detector circuitry 122 can provide the determined intensity levels to ADC 123, for example, as analog signals having amplitudes (e.g., voltage amplitudes or current amplitudes) indicative of the light information contained in the photodetector signals. In some aspects, detector circuitry 122 can amplify and / or filter the photodetector signals.

[0044] The ADC 123 can receive an analog signal from the detector circuit 122 indicative of the intensity level of the received light pulse 125 and can convert the analog signal to digital data that can be processed by the LIDAR controller 130. The ADC 123 can be any suitable ADC, such as (but not limited to) a flash ADC, a successive approximation register (SAR) ADC, or a delta-sigma ADC. In some implementations, each photodetector 121 can correspond to a respective ADC. In other implementations, multiple photodetectors 121 can correspond to a single ADC (e.g., to reduce the size, cost, and / or power consumption of the LIDAR device 100). In some other implementations, the ADC 123 can be omitted. Each photodetector 121 (and corresponding ADC 123) can be associated with a particular light emitter 112. In this way, multiple ToFs can be measured, and thus multiple distances can be determined during one pass of the LIDAR device 100.

[0045] The LIDAR controller 130 can include a processor 131, a memory 132, and a digital signal processor (DSP) 133. The DSP 133 can process the digital data provided by one or more ADCs 123 to determine information about the light pulses received by any number of photodetectors 121. In some implementations, the intensity and / or time of arrival of the light pulses can be used to determine the size, shape, and location of multiple detected objects in the surrounding environment. For example, the DSP 133 can use the time of arrival to determine the ToF, given the time of departure of the light pulse. In another example, the DSP 133 can use the measured intensity to determine the energy loss of the reflected light pulse, given the intensity of the emitted light pulse. Objects relatively close to the LIDAR device 100 can reflect the emitted light pulse 115 before objects relatively far from the LIDAR device 100. Additionally, light reflected from objects relatively close to the LIDAR device 100 can have less pulse broadening than light reflected from objects relatively far from the LIDAR device 100 (assuming similar surface reflectivity between objects at different distances). Thus, in some implementations, the distance between the LIDAR device 100 and an object can be estimated based on the rising and falling edges of the received light pulse 125.

[0046] The processor 131 can be any suitable processor or processors capable of executing scripts or instructions of one or more software programs stored in the LIDAR device 100 (e.g., within the memory 132). In some implementations, the processor 131 can include one or more microprocessors and a memory providing at least a portion of a machine-readable medium within which program instructions or scripts can be stored. In other implementations, the processor 131 can be an application-specific integrated circuit (ASIC). In some other implementations, the processor 131 can be or include one or more field-programmable gate arrays (FPGAs) or programmable logic devices (PLDs).

[0047] The memory 132 can store information related to the transmitter 110, the receiver 120, the surrounding environment, or any combination thereof. The memory 132 can also include a non-transitory computer-readable medium (e.g., one or more non-transitory memory elements, such as EPROM, EEPROM, Flash memory, a hard drive, etc.) that can store a number of software (SW) modules, each including instructions that, when executed by the processor 131, cause the LIDAR device 100 to perform all or a portion of the operations described herein. In some other implementations, the LIDAR controller 130 can be instructed by one or more processors external to the LIDAR device 100 (such as included in a processing system of a vehicle to which the LIDAR device 100 is coupled) to perform one or more operations related to transmitting or receiving light pulses. For example, the LIDAR device 100 can be coupled to a processing hub of a vehicle (such as via a controller area network (CAN) bus) or another processing system of the vehicle, and the processing system can instruct the LIDAR device 100 to perform one or more operations and receive information from the LIDAR device 100 in response (such as retrieving ToF or intensity information measured by the LIDAR device 100 that is used by the processing system to generate a point cloud or other type of depth map or rendering of the environment).

[0048] FIG. 2A An example LIDAR device in an environment is shown. In FIG. 2AIn the example of FIG. 1, LIDAR device 100 is located in an environment 200 that includes a car 201 and an overhang 202. In the simplified example, LIDAR device 100 is shown as including three light emitters 112A-112C that emit respective light pulses 115A-115C into environment 200. LIDAR device 100 is also shown as including three photodetectors 121A-121C that receive respective light pulses 125A-125C that are reflected from surfaces of objects in environment 200. First light pulse 115A illuminates a surface 205A of overhang 202, and first photodetector 121A receives a corresponding reflected light pulse 125A. Second light pulse 115B illuminates a surface 205B of car 201, and second photodetector 121 receives a corresponding reflected light pulse 125B. Third light pulse 115C illuminates another surface 205C of car 201, and third photodetector 121C receives a corresponding reflected light pulse 125C. LIDAR device 100 can use one or more properties of received light pulses 125A-125C (e.g., timing, amplitude, pulse width, etc.) to determine a distance between LIDAR device 100 and each of surfaces 205A-205C in environment 200.

[0049] FIG. 2B is an example timing diagram 210 of waveforms corresponding to emitted light pulses and received light pulses of an example LIDAR device. Transmit waveforms 215A-215C can indicate the intensity levels of respective light pulses 115A-115C emitted from LIDAR device 100, and receive waveforms 225A-225C can indicate the intensity levels of respective light pulses 125A-125C received by LIDAR device 100. FIG. 2A FIG. 2A is an example timing diagram 210 of waveforms corresponding to emitted light pulses and received light pulses of an example LIDAR device. Transmit waveforms 215A-215C can indicate the intensity levels of respective light pulses 115A-115C emitted from LIDAR device 100, and receive waveforms 225A-225C can indicate the intensity levels of respective light pulses 125A-125C received by LIDAR device 100. FIG. 2A FIG. 2A is an example timing diagram 210 of waveforms corresponding to emitted light pulses and received light pulses of an example LIDAR device. Transmit waveforms 215A-215C can indicate the intensity levels of respective light pulses 115A-115C emitted from LIDAR device 100, and receive waveforms 225A-225C can indicate the intensity levels of respective light pulses 125A-125C received by LIDAR device 100. A -t C A -t C A -t CThe corresponding reflected light pulses 125A-125C are received by the LIDAR device 100. The time of receipt t A -t C The determined time of receipt t A -t C may be used to determine the distance between the LIDAR device 100 and the respective surface 205A-205C of the environment 200.

[0050] Many LIDAR devices allow for adjustment of one or more characteristics of the emitted light pulses. For example, the LIDAR device 100 ( FIG. 1 ) can configure the transmitter 110 to adjust one or more of the timing of the light pulses of the emitted light 115, the frequency of the light pulses, or the intensity of the light pulses. In some examples, the LIDAR device 100 allows for dithering of the light pulses of the emitted light 115 to control the timing. Some LIDAR devices can allow for adjustment of the power provided to the light emitter 112 (e.g., LED) to control the intensity. Some LIDAR devices can allow for adjustment of the reference signal frequency (e.g., light source) to adjust the frequency of the light pulses of the emitted light 115. The receiver 120 can also be configured to sense differences in the timing, frequency, or intensity of the received light 125 light pulses. For example, the photodiode array of the photodetector 121 can be sensitive enough to determine intensity differences, and the sampling rate of the photodiode array and / or ADC can be fast enough to determine differences in the timing or frequency of the received light 125 light pulses.

[0051] In some aspects, the LIDAR device 100 (e.g., the LIDAR controller 130) can be configured to encode information into the emitted light 115 through frequency adjustment, intensity adjustment, and / or time dithering of the pulses to be emitted, and the transmitter 110 can be configured to communicate such encoded information through the emitted light 115. For example, the LIDAR device 100 can receive a data signal to send, and the LIDAR device 100 can use the light to be emitted (herein can be referred to as a light signal) as a carrier signal. In this way, the LIDAR device 100 can modulate the light signal to include the data signal, and the LIDAR device 100 can send the modulated light signal. For a first LIDAR device, the receiver 120 can be configured to receive the modulated light signal sent by the transmitter 110 of a second LIDAR device. The modulated light signal is encoded with the data signal from the second LIDAR device, and the LIDAR controller 130 of the first LIDAR device can be configured to extract the data signal from the received modulated light signal. For example, the LIDAR controller 130 can demodulate the received light signal to produce the data signal. Because this communication between LIDAR devices is point-to-point (thus no central network is needed), and the frequency of the signal is greater than cellular communication, the throughput can be higher and the latency can be lower than conventional cellular communication. A “modulated light signal” herein refers to a light signal that is modulated to include a data signal (e.g., the light emitted by the LIDAR device 100).

[0052] In some implementations, the LIDAR device 100 can be configured to switch between a distance measurement mode (which can be referred to herein as a "detection mode") and a communication mode for transmitting modulated light signals including data signals. For example, the LIDAR controller 130 can determine when the LIDAR device 100 is transmitting or receiving information through modulated light signals, and when the LIDAR device 100 is operating to detect surfaces of objects in the environment. For example, the LIDAR controller 130 can determine to place the LIDAR device 100 in the communication mode for a first portion of time and in the detection mode for a second portion of time. The LIDAR device 100 can thus switch between the communication mode and the detection mode. In some other implementations, the emitted light 115 can be used to communicate information to another receiver (e.g., by adjusting the frequency) and also to detect surfaces of objects (e.g., by sensing intensity differences in the received light 125), and the LIDAR device 100 can be configured to perform both modes simultaneously. For example, the LIDAR device 100 can emit a modulated light signal including a data signal. The LIDAR device 100 can receive a reflection of the modulated light signal and use the reflection to determine a depth of an object from the LIDAR device 100. Additionally, a second LIDAR device can receive the modulated light signal from the LIDAR device 100 and demodulate the light signal to generate the included data signal.

[0053] A device such as a vehicle or infrastructure can include or be coupled with one or more LIDAR devices (e.g., the LIDAR device 100) configured to communicate data signals (provided by, for example, the vehicle or infrastructure) via the emitted light 115. In this way, the device can communicate with the vehicle or infrastructure using the configured LIDAR device. In some aspects, the LIDAR device 100 can be configured to use a communication protocol employed by other vehicles and infrastructure including LIDAR devices. The communication protocol can be ad-hoc or managed, and any suitable packetization of information can be used for the communication protocol. For example, LIDAR communication within a fleet of vehicles can be based on a particular protocol. In some implementations, a standardized protocol (or a protocol employed by multiple parties) can more easily allow for integration of LIDAR communication between vehicles and infrastructure. Such a protocol can include a defined packet format for transmitting and receiving information. Protocols from other communication mediums such as cellular communication, Wi-Fi communication, digital subscriber line (DSL) communication, fiber optic communication, etc. can be utilized to create a protocol for communication between LIDAR devices.

[0054] FIG. 3An example packet format is shown for a plurality of received packets 300 based on an example communication protocol. In some implementations, the LIDAR device 100 can include a buffer or other suitable memory (e.g., memory 132) for queuing one or more received packets 300 (e.g., packets 1-6). The LIDAR device 100 can process the buffered packets for the vehicle or infrastructure to perform one or more operations. In another example, the buffered packets can be provided to a processing system of the device for processing. As shown, the format of the packets 1-6 includes fields 302-308 including a sender ID 302, a location 304, a payload type 306, and a payload 308. Additional fields can be included in the packets, fewer fields, different fields, or a different organization of fields as any suitable packet format can be used. For example, the packets 1-6 can include a cyclic redundancy check (CRC) field after the payload 308 to correct for any errors in the received packets.

[0055] The sender ID 302 can indicate the device that sent the packet. In some implementations, each vehicle can include a unique identifier to identify that particular vehicle. For example, if a fleet of 200 autonomous vehicles are communicating with each other, each vehicle can include an identifier that is unique relative to the other vehicles in the fleet (e.g., “vehicle_l” through “vehicle_200”). If a LIDAR device equipped on a vehicle associated with “vehicle_100” sends packets 1 and 2 to the LIDAR device 100, the sender ID 302 of the packets can include that unique identifier “vehicle_100”. Similarly, if a LIDAR device equipped on a vehicle associated with “vehicle_102” sends packet 3 to the LIDAR device 100, the sender ID 302 of the packet can include that unique identifier “vehicle_102”. Some vehicles can not include a unique identifier or be identified, but the vehicle can still communicate packets to the LIDAR device 100. In one example, the sender ID 302 can include a sender ID value specific to a previously unidentified vehicle or device (shown as “unidentified_vehicle” for packet 4). The sender ID value can be, for example, zero-padded, a field with a null value, or otherwise appropriately filled to indicate that there is no unique identifier for the vehicle or infrastructure associated with the packet.

[0056] Infrastructure (e.g., toll booths, tunnels, entrance gates to high occupancy vehicle (HOV) lanes, etc.) can include or be coupled to one or more LIDAR devices to send and receive packets to and from LIDAR device 100. For example, packet 5 can be sent by a transmitter located at a tunnel entrance (which can be uniquely identified as “Tunnel_40”), while packet 6 can be sent by a transmitter located at a toll booth (which can be uniquely identified as “Toll Booth_30”). Any suitable vehicle or infrastructure can include a LIDAR device for LIDAR communication, and can include a unique transmitter ID.

[0057] In some implementations, emergency vehicles, construction zones, and other devices associated with priority transmissions can include a transmitter ID that indicates that a packet is for priority transmission. If LIDAR device 100 decodes packets sequentially, LIDAR device 100 can first process transmitter ID 302 and determine that the packet is a priority transmission. In this way, LIDAR device 100 can determine that a packet is to be processed with priority before completing processing of the remainder of the packet. Other suitable implementations of transmitter ID 302 can be used, and the present disclosure is not limited to the examples provided. For example, transmitter IDs can be configured to distinguish between caravans, vehicle types (e.g., private cars, commercial trucks, school buses, etc.), and the like.

[0058] Location 304 can indicate a location of a vehicle or infrastructure associated with the transmitting device. For example, a vehicle or infrastructure can include a global positioning system (GPS) receiver to determine latitude and longitude. The latitude and longitude can then be provided in location 304, as shown for packets 1-6. Location 304 can alternatively include positioning information relative to LIDAR device 100 as determined by other vehicles or infrastructure via their own LIDAR devices. For example, a transmitting LIDAR device can determine a relative distance and location between the transmitting and receiving vehicles or infrastructure, and can transmit the device distance and location in location 304. In some implementations, if location information cannot be transmitted (e.g., a vehicle does not include a GPS receiver), location 304 can be padded with zeros or otherwise filled in to indicate that no location is provided.

[0059] The payload type 306 can indicate a type of information to be provided in the payload 308 of the packet. In the example packets 1-6, the payload type 306 of packet 1 indicates a "heartbeat" or "pulsate," which can be a signal periodically transmitted by the "vehicle 100." In some implementations, the "heartbeat" or "pulsate" can indicate that the packet is similar to a beacon, and basic information can be provided in the payload 308. In some examples providing basic information for a vehicle or infrastructure, the payload 1 can include a jitter rate or other operating parameters for the transmitter and / or trajectory information or other status information for the vehicle. Alternatively, no information can be provided in the payload 308 for such packets (e.g., the payload is zero-padded, and the packet is only used to provide location information via the location 304 and to inform other devices of the presence of the transmitter).

[0060] The payload type 306 of packet 2 is a "render_description." As described in the following example use case, the LIDAR device 100 can measure surfaces within the line of sight (LoS) of the LIDAR device 100, but surfaces not within the LoS of the LIDAR device 100 can not be sensed and detected. In this way, a rendering of the environment based on measurements from the LIDAR device 100 can not include portions of the vehicle that are not within the view of the LIDAR device 100. The "render_description" can indicate that basic information about rendering the vehicle or infrastructure can be provided in the payload 308. For example, the payload 2 can include dimensions of one or more shapes associated with the "vehicle 100" to be rendered. In one implementation, the dimensions can include an orientation and dimensions of a geometric shape (e.g., a rectangle for a two-dimensional rendering or a rectangular prism for a three-dimensional rendering). Other suitable rendering information can include textures, contours, or other features of an object rendering indicated by the payload 308. For example, an emergency vehicle or road work zone can be highlighted in a rendering displayed to a driver and / or passenger of the vehicle. The payload 308 can thus indicate a texture to highlight portions of the rendering associated with the emergency vehicle or work zone. In another example implementation, similar textures can be used for vehicles within the same fleet. In this way, a driver and / or passenger can easily identify fleet vehicles in the displayed rendering. However, any suitable rendering description can be used, and the present disclosure is not limited to the examples provided.

[0061] As an addition or alternative to "RENDER_DESCRIPTION," the payload type 306 can indicate "RENDER_ID," such as for packet 4. In some implementations, a list of standard vehicles (or infrastructure) and their associated rendering information can be stored in a database, such as a lookup table or other suitable organized group of data for access. For example, if the vehicle sending the packet is a 1994 Geo Metro, the payload 308, such as payload 4, can indicate "1994 Geo Metro." The database can be used in determining rendering details for the particular vehicle, such as the dimensions and textures of the rendering, and existing renderings can be augmented by additional rendering details. The database can also include information about the particular vehicle, such as length, acceleration capabilities, etc. The term "rendering" can refer herein to a line drawing, point cloud, depth map, image, texture, shading, or other information that can be visualized or displayed.

[0062] The payload type for packet 5 is "GATED-ENTRY," which can indicate that the payload 5 indicates the type of vehicle allowed to enter "Tunnel_40." For example, "Tunnel_40" can be limited to high- bearing or autonomous vehicles, which can be indicated in packet 5. Thus, the vehicle including the LIDAR device 100 can determine whether to allow entry into "Tunnel_40" based on the information in the payload 5.

[0063] The payload type for packet 6 is "TOLL_CHARGE," which can indicate that the payload 6 indicates the fee charged by "Toll Booth_30." For example, if the bridge toll is $5, the payload 6 can indicate that $5 will be automatically charged from the driver's or fleet's account when the vehicle passes "Toll Booth_30." In some implementations, the payload 308 can include a combination of the fee to be charged and a restriction on vehicle entry. For example, a transmitter for a parking garage space can indicate the parking rate and the vehicles allowed in that space (e.g., reserved for electric vehicle spaces, designated handicapped parking, etc.). When the vehicle is parked in the space, the transmitter can update the parking fee based on the rate so that the driver can know the current parking cost.

[0064] The payload type of packet 3 is“vehicle locator,” which can indicate that the payload 3 indicates a vehicle that is being sought. For example, if“vehicle_005” is offline, a vehicle in the fleet can send a packet that attempts to locate“vehicle_005.” In this way, the payload 3 can include a“vehicle_005” identifier to indicate which vehicle to locate. In some implementations, the LIDAR device 100 can keep a number of transmitter IDs of received packets. Thus, the LIDAR device 100 can be configured to search the kept transmitter IDs of received packets to determine whether“vehicle_005” is in communication with the LIDAR device 100. The LIDAR device 100 can then indicate this to“vehicle_102.” For example, the LIDAR device 100 can send the location received in the packet from“vehicle_005” to“vehicle_102.” In some other implementations, the LIDAR device 100 can be configured to propagate a message to other vehicles or infrastructure, indicating that“vehicle_005” is to be located. In this way, if“vehicle_005” receives the propagated message from other vehicles or infrastructure, then“vehicle_005” can determine to communicate its location with the fleet.

[0065] Other suitable payload types and payloads can exist (e.g., for different use cases of LIDAR communication as described herein), and the present disclosure is not limited to the provided examples. In some example implementations, a vehicle can include multiple sensors to collect information for packet generation. For example, the payload can include information about the number of passengers, the vehicle operating mode, etc., and sensors such as pressure sensors, occupancy sensors, engine sensors can be used to detect operating mode, etc., to collect such information.

[0066] A wide variety of information can be provided and received via LIDAR communication, and LIDAR communication can be applied to a variety of use cases as described herein.

[0067] LIDAR communication implementation

[0068] Vehicle locator

[0069] A vehicle can operate in areas without a cellular network for communication. For example, when a vehicle traverses a rural or sparsely populated area, the vehicle can not have consistent cellular connectivity or be able to communicate with a base station. In another example, a cellular modem of the vehicle can not be operational. If the vehicle is part of a fleet of vehicles (e.g., a fleet of taxis or rental cars), a dispatcher can not be able to identify the location of the vehicle, and the dispatcher can want to find the vehicle.

[0070] In some implementations, searching for a missing, stranded, or offline vehicle can be proactive, where a vehicle or infrastructure that finds the offline vehicle reports back to a dispatcher that the offline vehicle was found. In some other implementations, searching for an offline vehicle can be reactive, where a vehicle records the last known location of the offline vehicle. In this way, the vehicle reports its findings when it is synchronized with a dispatcher or other parts of the fleet, such as when the vehicle is charging or otherwise not operational.

[0071] Other vehicles in the fleet can be instructed to send vehicle locator messages through LIDAR communications. For example, each of the fleet vehicles can be instructed to periodically send a packet with a payload type of "vehicle_locator" that identifies the offline vehicle in the packet payload. The dispatcher can also instruct infrastructure (e.g., a fleet-owned electric vehicle charging station or parking space that includes or is coupled to a LIDAR device) to send a vehicle locator message. As described above, other vehicles can also propagate the message upon receiving such a message to expand the number of senders of the vehicle locator message. In this way, a vehicle that can be offline with respect to its primary method of communication (e.g., cellular) can receive the message from a passing vehicle or infrastructure, and the vehicle can determine to communicate its location to the dispatcher through LIDAR communications. In another example, a vehicle or infrastructure that finds the offline vehicle can communicate the location of the offline vehicle to the dispatcher.

[0072] For example, if another fleet vehicle comes into LIDAR communication with the offline vehicle, the offline vehicle can send a message through LIDAR communications to the other fleet vehicle that its cellular modem is inoperable, and contact the dispatcher, e.g., through the cellular modem of the other vehicle. In this way, the dispatcher can communicate with the offline vehicle through the other fleet vehicle.

[0073] As an additional or alternative to the dispatcher finding an offline fleet vehicle, a vehicle locator message can be used in emergency situations, such as for a stolen vehicle. The payload of the vehicle locator packet can include a description of the vehicle to be found, and such a packet can be propagated to other vehicles and infrastructure to find the stolen vehicle. In some implementations, a vehicle can display a notification to a driver in response to receiving the packet. In some other implementations, the stolen vehicle can be automatically identified based on the size or other attributes of the stolen vehicle. For example, surfaces of other vehicles detected using a LIDAR device can be used to identify a vehicle of the same make and model based on similar dimensions or other features of the stolen vehicle. In another example, a visible light camera can be used to capture an image of a license plate after detecting a missing vehicle, and the image can be analyzed to identify the missing vehicle based on the license plate number. The vehicle can report the location of the missing vehicle to a central office (e.g., through a cellular modem).

[0074] In some other implementations, a vehicle locator message can be initiated by an offline or disabled vehicle. For example, a cellular modem of a fleet vehicle can be inoperable, and the fleet vehicle is unable to communicate with the fleet over a cellular network. If a dispatcher provides information over a cellular communication about where the fleet vehicle is to go, the fleet vehicle can be unable to determine where the dispatcher wants to dispatch the vehicle. In an example of an autonomous fleet of taxi vehicles, a dispatcher can want to dispatch a vehicle to a particular address to pick up a customer and then have the customer ride in the taxi to a particular destination. However, the vehicle can not receive the instructions from the dispatcher. LIDAR communication can be used to provide any messages from the vehicle (e.g., a vehicle with a disabled cellular modem) to the dispatcher. For example, the vehicle can send a message for the dispatcher through LIDAR communication to other fleet vehicles. The other fleet vehicles can then send the message to the dispatcher. In this way, the dispatcher is informed that the vehicle can not be receiving messages from the dispatcher, and the dispatcher can update the handling of requests without the vehicle in the fleet. The dispatcher can also provide instructions to the vehicle through LIDAR communication (e.g., through other fleet vehicles) to wait for service at a specified location, to remove itself from the fleet operation, or any other suitable operation.

[0075] In some implementations, a vehicle can have a home location away from a taxi fleet dispatcher. For example, if a vehicle does not have a location to go to (e.g., a fare communicated to the dispatcher), the vehicle can return to a specified location in a service area, such as a particular parking space for the vehicle within the service area. The space can be associated with a LIDAR device that is communicable with the vehicle (e.g., allowing the dispatcher to communicate with the vehicle). The specified location can be a parking area for the vehicle until the vehicle is activated to collect a fare when the vehicle is fully operational and back in standard communication with the dispatcher. The vehicle can also return to the specified location (e.g., a parking space) when the vehicle is unable to communicate with the dispatcher and is not being used for a fare. Reference to an offline vehicle herein can refer to a vehicle that is stranded, lost, or otherwise not operational or can not be expected to operate in a default, typical, or standard mode of operation.

[0076] FIG. 4An environment 400 is shown that includes example parking spaces 410 for autonomous fleet vehicles 402 to return to when not in use. The autonomous vehicles 402 include one or more LIDAR devices, such as LIDAR device 404. A stand 408 can be located near the parking spaces 410. Examples of stands 408 include parking meters, charging stations, and taxi stands for walking customers. In some implementations, the stand 408 includes a LIDAR device that communicates with the vehicles 402 through LIDAR communication 406. The stand 408 also includes a wired or wireless backhaul, such as a cellular modem or a fiber connection. Vehicles 402 that are unable to directly communicate with a dispatcher can communicate with the stand 408 through LIDAR communication, and the stand 408 can communicate with the dispatcher through the backhaul. While the vehicles 402 are described as communicating with the stand 408 through LIDAR communication, additional or alternative communication systems can be used in some other implementations. Other systems can include wireless local area network systems (such as IEEE 802.11 based systems), Bluetooth® systems, cellular systems (such as 3G, 4G, 5G, etc.), visible light communication systems, near field communication (NFC) systems, etc.

[0077] While communicating with the stand 408, the vehicle 402 can provide the stand 408 with a vehicle locator packet. In some implementations, sending a vehicle locator packet can indicate to the stand 408 and to a dispatcher that the vehicle 402 is unable to directly communicate with the dispatcher (such as through a cellular modem). In some examples, the dispatcher can remove the vehicle 402 from service, dispatch a maintenance person to the vehicle 402, send the location of the vehicle 402 for maintenance, or communicate a new fare to be handled by the vehicle 402. In some implementations, the stand 408 can also communicate with the vehicle 402 while the vehicle 402 is on the road. If the vehicle 402 is unable to directly communicate with the dispatcher, the stand 408 can update the dispatcher with the location of the vehicle 402.

[0078] Ride hailing

[0079] In addition to locating vehicles, LIDAR communication can also be used to hail vehicles. Referring back to FIG. 4 , the stand 408 can be a walking taxi stand. A person can walk to the stand 408 and input a desired destination (such as through a smartphone application, a graphical user interface of the stand 408, or another suitable interface with the stand 408). Multiple autonomous vehicles can be queued at a location away from the stand 408 (such as in a parking lot at a corner) and communicate with another LIDAR transmitter (such as one or more stands in the parking lot). The parking space 410 can be the location where the next vehicle in the queue is hailed, and the parking space 410 can be the location where the user enters the vehicle and begins the requested ride.

[0080] The station 408 can handle multiple ride requests concurrently. As a result, the station 408 can call multiple vehicles (such as using one or more LIDAR transmitters in a staging area for queued vehicles). Each called vehicle can be assigned a particular customer or ride, and as the vehicle approaches the parking space 410, the vehicle can indicate the assigned ride to the station 408 (through the LIDAR communication 406). In this way, the station 408 can inform the customer of the assigned vehicle in the parking space 410 to speed up pickup and coordinate rides from multiple customers in the same parking space 410.

[0081] Notification

[0082] LIDAR communication can be used to provide notifications to vehicles or drivers. For example, a vehicle can be notified through LIDAR communication of, for example, an approaching emergency vehicle, a road hazard, a school crosswalk during school hours, a change in speed limit, a construction zone, etc. For emergency vehicles, an emergency broadcast can be transmitted by the infrastructure or a vehicle that previously received the broadcast to convey the notification to other vehicles through LIDAR that an emergency vehicle is present.

[0083] FIG. 5 An example environment 500 is shown for communicating the presence of an ambulance 502 to a vehicle 506. The vehicle 508 can detect the ambulance 502. For example, the ambulance 502 can broadcast an emergency signal indicating the presence of the ambulance 502. In another example, a LIDAR device or other sensor of the vehicle 508 can sense the ambulance 502, and the vehicle 508 can identify the ambulance 502 as an emergency vehicle.

[0084] In some examples other than an ambulance 502, the vehicle 508 can detect a person in the environment that would necessitate other vehicles to change their navigation, or can be notified of such a person. For example, the vehicle 508 can detect one or more bicyclists in a lane, children playing near a street, a child or other pedestrian crossing a street (e.g., before or after school), etc. In other examples, the vehicle 508 can detect a car with its engine off, a minor traffic accident (“fender bender”), a road blockage, or other traffic condition that can necessitate other vehicles to change their navigation or be notified of such an accident. Such a scenario can necessitate a vehicle to slow down its speed through the area, use one or more LIDAR devices to adjust scanning the environment to focus on a particular area, change lanes, etc., for example, to provide more space for the detected person, situation, etc.

[0085] Vehicle 508 can not be within LIDAR communication range of vehicle 506. For example, LIDAR communication can be based on LoS, and vehicle 506 can be blocked or otherwise occluded by a building to prevent vehicle 508 from directly communicating with vehicle 506. In some implementations, vehicle 508 can use one or more vehicles and / or infrastructure to perform LIDAR communication with vehicle 506 (e.g., “around the corner” communication). For example, vehicle 508 can use LIDAR communication 510 with vehicle 504 to indicate the presence of ambulance 502. Additionally or alternatively, vehicle 508 can communicate with station 514 through LIDAR communication 516 to indicate the presence of ambulance 502.

[0086] While FIG. 5 While direct LoS communication between LIDAR devices (510, 512, 516, and 518) is shown, LoS communication between LIDAR devices can include one or more reflections of a signal off a surface. For example, a focused beam of light signals from a transmitting LIDAR device can be intentionally reflected off one or more building walls, signs, or other objects in environment 500, and the reflections can be received by a receiving LIDAR device. If the beam is sufficiently focused, the transmission includes sufficient transmission power, and the one or more surfaces have sufficient reflectivity, the reflections received by the receiving LIDAR device can be processed to determine the communication transmitted by the transmitting LIDAR device (e.g., a notification of the presence of ambulance 502). In this way, LIDAR devices can communicate with each other without direct LoS. In this specification, LoS can include direct LoS or indirect LoS (which can include one or more reflections).

[0087] Vehicle 504 or station 514 can be configured to indicate the presence of ambulance 502 or other traffic conditions to other vehicles and infrastructure within LIDAR communication range. For example, vehicle 504 can use LIDAR communication 512 to indicate the presence of ambulance 502. In another example, station 514 can use LIDAR communication 518 to indicate the presence of ambulance 502. In this example, indicating the presence of ambulance 502 can include two hops from vehicle 508 to vehicle 506 (e.g., through LIDAR communications 510 and 512 or through LIDAR communications 516 and 518).

[0088] In some implementations, the indication (or other notification) of the emergency vehicle can be propagated for a defined number of hops or a defined distance. In this way, other vehicles are notified of the presence of the emergency vehicle and localize to an area around the emergency vehicle or traffic condition. In some examples, the vehicle 508 can also determine a trajectory, route, or other information for the ambulance 502 and indicate such information to the vehicle 504 or the station 514. The distance or number of hops for which the indication is propagated can increase, for example, in the direction of the route (or conversely, decrease in the opposite direction of the route). If the vehicle 506 is an autonomous vehicle, the vehicle 506 can proactively pull over to the side of the road, stop, change lanes, or alter its route in response to receiving the indication of the presence of the ambulance 502 or other traffic condition. In some other implementations, the vehicle 506 can provide a visual or audible notification to the driver (and / or passengers) to indicate the presence of the ambulance 502 or traffic condition. For example, a speaker or display of the vehicle can notify the driver and / or passengers of the presence of the ambulance 502 or traffic condition.

[0089] Another example notification can be available parking spaces in a parking lot or garage. In some implementations, an entry gate of the parking garage can communicate available spaces to an incoming vehicle via LIDAR communication. If the vehicle is an autonomous vehicle, the vehicle can be assigned a parking space and automatically proceed and park in the space. In this way, the parking garage or lot can efficiently organize the parking of incoming vehicles. In some other implementations, the entry gate can indicate available parking spaces to the vehicle, and the vehicle can notify the driver and / or passengers of the location of the available spaces. For example, the vehicle can display a map of the parking garage and indicate the location of the available spaces on the displayed map. The driver and / or passengers can be notified of the available spaces prior to the vehicle entering the parking garage (e.g., when the vehicle is approaching the garage), and the driver and / or passengers can select which spaces are preferred.

[0090] As described above, another example notification can be a traffic accident or other traffic obstacle. The notification can be communicated to vehicles ahead of the obstacle via LIDAR communication, and the vehicles can determine alternative routes and / or notify the driver and / or passengers of possible delays based on the received notification. Other suitable example notifications can also be sent via LIDAR communication, and the present disclosure is not limited to the examples described above.

[0091] Limited access area

[0092] Another use case for LIDAR communication is to restrict access to a defined area. Certain areas can be limited to specific vehicles or types of vehicles. For example, a hospital ambulance entrance can be limited to ambulances. In another example, a handicapped parking space can be limited to vehicles that include a handicapped placard. In another example, an HOV lane can be limited to vehicles that include, for example, 3 or more passengers. In another example, a portion of a city center can be limited to autonomous vehicles, low- or zero-emission vehicles, taxis, or other specific types of vehicles. In another example, a company parking garage can be limited to employee vehicles or vehicles with security clearance.

[0093] LIDAR communication can be used to indicate restrictions to a particular area and to authorize access to the area for specific vehicles. In some implementations, a LIDAR device for the area (e.g., an entrance gate, a station, or other infrastructure) can send the restrictions for the area to approaching vehicles. The vehicles can then communicate the necessary authentication to the LIDAR device (through their own LIDAR devices) that the vehicle is cleared to enter the area. The vehicle can then be allowed to enter and navigate the area (e.g., a door is opened or the vehicle is authorized to access the area). In some other implementations, a vehicle can store the necessary qualifications for the area. In this way, a vehicle can proactively communicate its credentials to a LIDAR device to gain the right of access. For example, when an ambulance approaches an ambulance entrance for a hospital, the ambulance can use a LIDAR device to identify itself as an ambulance to enter.

[0094] Access restrictions to an area can change over time. For example, an HOV lane can be restricted only during designated peak hours. In another example, a city center can be limited to zero-emission vehicles when the smoke level is above a threshold. In another example, an area around an accident or that requires police or emergency response personnel can be limited to such police or emergency response personnel vehicles. Because LIDAR communication does not require a centralized network, changes to access restrictions can be made and provided to vehicles more quickly than, for example, by other communication to remotely coordinate and attempt to establish access parameters for an area. In some examples, a vehicle can notify a driver and / or passengers that the vehicle is about to enter a restricted area. In this way, the driver, passengers, or the vehicle itself can adjust a route to navigate around the restricted area.

[0095] In another implementation, the zones can include different tolls or fees based on the time of day, day of the week, amount of congestion in the zone, special events, etc. LIDAR communications can be used to communicate such tolls or fees to the vehicle before the vehicle enters the zone. The driver and / or passengers can then determine (or the vehicle can automatically determine) to enter the zone based on this information. For example, some roads include a hot lane, where the price to enter the lane is based on the time of day and congestion on the road. The driver and / or passengers can be notified of the price to enter the hot lane, thereby deciding whether to enter. Alternatively, the vehicle can be configured to enter the hot lane based on, for example, whether the price is less than a threshold amount or the priority of the ride. For example, if a couple of aligned parents are going to the hospital, the priority of the ride can be set to override any price constraints associated with the hot lane. Other suitable restriction use cases can exist, and the present disclosure is not limited to the examples provided.

[0096] Object visualization or rendering

[0097] Another use case for LIDAR communications is to provide rendering or visualization information. As described with respect to FIG. 4, the packets can include “render_description” or “render_identification” information for rendering or visualizing the vehicle (or infrastructure, such as the station 408 in FIG. 4). Rendering or visualizing objects in the environment of the vehicle can be limited to the LoS of the LIDAR device coupled to the vehicle. Thus, surfaces that are not in the LoS of the LIDAR device can not be detected and rendered. FIG. 3 FIG. 4

[0098] FIG. 6A ​​An example environment 600 of the vehicle 602 is illustrated. The environment 600 includes buildings 604 and 606 and vehicles 608 and 610. As illustrated, a view of the vehicle 608 from the perspective of the LIDAR device 612 of the vehicle 602 includes a front surface and a driver side surface of the vehicle 608. A view of the vehicle 610 from the perspective of the LIDAR device 612 is occluded by the building 606. As a result, the view of the vehicle 610 includes a front surface and only a portion of a driver side surface. Further, the views of the buildings 604 and 606 from the perspective of the LIDAR device 612 are surfaces facing the vehicle 602. The LIDAR device 612 can collect information about the depths of the surfaces within the view of the LIDAR device 612 and the vehicle 602 can visualize or render the environment 600 including the detected surfaces. As used herein, a vehicle visualizing or rendering an environment can include one or more of the vehicle or infrastructure providing instructions to a display coupled to the vehicle or infrastructure (e.g., a tablet communicably coupled to a processing system of the vehicle, a dashboard infotainment system coupled to a processing system of the vehicle, a display remote from the vehicle, etc.) to render at least a portion of the environment on the display; or the vehicle rendering at least a portion of the environment on an integrated display (e.g., through an integrated entertainment system in a center console of the vehicle). As used herein, a particular vehicle or infrastructure performing one or more operations (such as a vehicle visualizing or rendering an environment) can refer to a device or system coupled to the vehicle or infrastructure performing the operation (such as generating a rendering to be displayed, and providing the rendering to a display coupled to the vehicle or infrastructure).

[0099] FIG. 6B An example rendering 650 of the environment 600 sensed by the LIDAR device 612 is illustrated. As described above, the example rendering 650 can be displayed on an integrated display of the vehicle or a display coupled to a processing system of the vehicle. The rendering 650 includes a rendering of a surface 654 corresponding to the building 604, a surface 656 corresponding to the building 606, a surface 658 corresponding to the vehicle 608, and a surface 660 corresponding to the vehicle 610. The example rendering 650 also includes a representation 652 of the vehicle 602 to provide perspective. For example, the representation can be an icon, a stock image, a block, or other saved representation of the vehicle to be used for the rendering.

[0100] The example rendering 650 is simplified to explain aspects of the disclosure, and the rendering 650 can include additional information or detail. For example, the lines or markings on the road can be reflective, and thus can be sensed by the LIDAR. In another example, additional features of the buildings 604 and 606 and the vehicles 608 and 610 within the LoS of the LIDAR device 612 can be sensed by the LIDAR. While the example rendering 650 is shown as two-dimensional from a bird’s eye view, the rendering can be any suitable orientation or from any suitable perspective. For example, the rendering can be a three-dimensional point cloud from the perspective of the LIDAR device 612, can be a two-dimensional view of the environment from the perspective of the LIDAR device 612, or can be any other suitable rendering.

[0101] The example rendering 650 can be provided to a driver and / or passenger by the vehicle 602 via a vehicle display. For example, the example rendering 650 can be displayed on an integrated display of the vehicle, or the vehicle can provide instructions for rendering to a display coupled to a processing system of the vehicle. As shown in the example rendering 650, depth information can not be determined for hidden surfaces of objects and buildings. For example, the rendering 650 does not include information for the passenger side and rear surfaces of the vehicle 608, does not include information for the passenger side, rear, and a portion of the driver side surface of the vehicle 610, and does not include information for the sides of the buildings 604 and 606 that are not directed toward the vehicle 602.

[0102] In some implementations, another LIDAR device can send rendering information for one or more objects in the environment 600 to the vehicle 602. For example, the LIDAR device 614 of the vehicle 608 can send information about the vehicle 608 for rendering, and the LIDAR device 616 of the vehicle 610 can send information about the vehicle 610 for rendering. In some implementations, dimensions for the shape of the vehicle can be sent for rendering. For example, dimensions for one or more rectangles used to render the vehicle 608 in the rendering 650 can be sent from the LIDAR device 614 to the LIDAR device 612. Dimensions for one or more rectangles used to render the vehicle 610 in the rendering 650 can be sent from the LIDAR device 616 to the LIDAR device 612. Additional features about rendering the vehicles 608 and 610 can also be sent. Example features include textures, such as colors, shading, highlighting, etc. for the vehicle rendering.

[0103] In some other implementations, the LIDAR devices 614 and 616 can transmit an identification of the respective vehicle. For example, the LIDAR device 614 can transmit information about the make and model of the vehicle 608, a vehicle identification number of the vehicle 608, a license plate number of the vehicle 608, whether the vehicle is part of a particular vehicle fleet (e.g., the same vehicle fleet as the vehicle 602), a unique identifier within the vehicle fleet, or other identifying information.

[0104] The vehicle 602 (or memory coupled to a display to display the rendering) can store rendering information for particular vehicles, and the vehicle 602 can use vehicle identification to look up the stored rendering information for the particular vehicle. For example, a vehicle memory (or other memory) can store rendering information for multiple makes and models of vehicles (including the vehicle 608 and the vehicle 610), and such rendering information can be retrieved and used to augment the rendering 650 of the environment 600. If the rendering 650 is to be displayed on a display coupled to the vehicle 602, the vehicle 602 can provide the rendering information retrieved from the vehicle memory, or the vehicle 602 can provide information to identify the rendering information in an external memory (such as providing information about the make and model to an entertainment system coupled to the vehicle). Example rendering information can include inventory images or drawings of particular vehicles. In some examples, the rendering information can be used to replace or combine with corresponding portions of the rendering 650. In some other examples, adjusting the rendering can include highlighting object renderings. For example, the rendering can be updated to notify a driver or passenger of an emergency vehicle or situation in the vehicle’s environment.

[0105] FIG. 6C An example environment 670 of a vehicle 672 is shown. The example environment 670 includes buildings 684 and 686 and vehicles 674 and 676. The environment 670 also includes an emergency vehicle 678. FIG. 6D An example top-down rendering 690 of the environment 670 of the vehicle 672 in FIG. 6C The rendering 690 includes a rendering of a surface 694 corresponding to the building 684, a surface 696 corresponding to the building 686, and a surface 698 corresponding to the vehicle 674. The example rendering 690 also includes a depiction 692 of the vehicle 672 to provide a perspective. The vehicle 676 and the emergency vehicle 678 can not be within the LoS of the LIDAR device coupled to the vehicle 672. As a result, the rendering 690 can not include a rendering of the vehicle 676 or a rendering of the emergency vehicle 678.

[0106] FIG. 7 A flowchart showing an example operation 700 to adjust a rendering of an environment, e.g., of a vehicle, LIDAR device, infrastructure, etc., is shown. The following references FIG. 6A and 6CThe example operations 700 are described herein as being performed by the vehicles 602 and 672 for illustrative purposes. One of ordinary skill in the art will recognize that the example operations 700 can be performed by any suitable device (e.g., a control device coupled to a vehicle), infrastructure, and / or vehicle in accordance with various implementations, and that the example operations 700 described herein can be performed with additional steps, fewer steps, different order of steps, parallel steps, or any combination thereof. In this document, the vehicle 602 performing one or more steps can refer to a processing system coupled to the vehicle performing the one or more steps, a processing system integrated into the vehicle 602 performing the one or more steps, or other suitable embodiments for performing the described methods. For example, a control device separate from the vehicle can be installed or otherwise coupled to the vehicle. Such a control device can also be coupled to one or more LIDAR devices, to a display (for rendering or displaying other information), and / or to one or more other input / output components (e.g., speakers, keyboard, etc.). The vehicle 602 performing one or more steps can refer to the control device performing the one or more steps in some implementations.

[0107] From 702, the vehicle 602 can receive data associated with a modulated light signal emitted by a transmitter of a first LIDAR device. For example, the device can be coupled to the LIDAR device 612, and the modulated light signal can be transmitted to the LIDAR device 612 from the LIDAR device 614 or the LIDAR device 616. The modulated light signal includes a light carrier signal modulated to include a data signal. The LIDAR device 612 or the device coupled to the LIDAR device 612 can extract the data signal and determine data included in the data signal.

[0108] The vehicle 602 can also generate a rendering of a vehicle environment based on information from one or more LIDAR devices coupled to the vehicle (704). For example, the vehicle 602 can include a processing system to receive measurements from the LIDAR device 612 while the LIDAR device 612 is in a detection mode. The measurements can be used to generate a rendering, such as a point cloud, a depth map, or other suitable representation of the environment 600. For example, the processing system can generate an example rendering 650 of the environment 600 (or an example rendering 690 of the environment 670) based on received measurements from the LIDAR device 612 or other LIDAR devices coupled to the vehicle, and the rendering can include a representation of surfaces detected in the environment 600 (or the environment 670).

[0109] Based on the received data, the vehicle 602 can update the rendering (706). For example, the vehicle 602 can update the rendering of the objects in the environment (708). In some example implementations, the vehicle 602 can receive an object identifier in the received data. For example, the LIDAR device 612 in the communication mode can receive a packet including an identifier of the vehicle 608 from the LIDAR device 614 through the modulated light signal. The identifier can be a specific vehicle ID in the packet, a vehicle identification number, a make and model of the vehicle, or other suitable identifier.

[0110] The vehicle 602 can determine rendering information for the object based on the identifier. Example rendering information can include dimensions of one or more shapes in the rendered object (e.g., one or more rectangles or other shapes in the rendered vehicle), a representative image of the object (such as a representative drawing of the vehicle based on, for example, a type of the vehicle, a make and model of the vehicle, etc.), a stock image of the object (e.g., a stock picture of the vehicle), or a texture to be applied to the rendering of the object (such as a texture to be applied to the rendering of the vehicle). The rendering information can also indicate whether the object rendering is to be highlighted or adjusted (e.g., changes in size, dimensions, stretching, etc.). In some implementations, if the vehicle 602 receives an identifier (e.g., a make and model) of the vehicle 608, the vehicle 602 can scan a database indexed by identifiers (e.g., makes and models) associated with rendering information (such as reference images of vehicles, stored dimensions of vehicles, highlighting information if an emergency vehicle, etc.) to determine rendering information for the vehicle 608 (such as a stored image of the make and model of the vehicle 608). The vehicle 602 can then update the object rendering based on the determined rendering information. For example, the rendered surface 658 in the rendering 650 can be combined with a stored image or other rendering information of the vehicle 608 to update the rendering 650. In another example, the rendered surface can be highlighted. In another example of updating the object rendering, the object rendering can be added to the rendering of the environment. For example, a rendering of the vehicle 676 and / or a rendering of the emergency vehicle 678 can be added to the rendering 690.

[0111] In some other implementations, the LIDAR device can be configured to transmit a data signal including a specific description of how an object should be rendered. For example, the LIDAR device 614 of the vehicle 608 can transmit a rendering model, a texture, and any other features for rendering the vehicle. In this way, no memory including lookup tables, databases, etc. can be used in determining rendering information for an object when updating a rendering of an environment.

[0112] Referring back to FIG. 7After updating the rendering, the vehicle 602 can provide the updated rendering for display (710). For example, a control device coupled to the vehicle 602 can provide the updated rendering to a display coupled to the vehicle 602, and the display can display the updated rendering to a driver and / or passengers. In another example, the rendering can be provided to a remote display for others to see the environment of the vehicle 602.

[0113] FIG. 8 An example rendering 800 of the environment 600 in FIG. 6 is shown. The rendering 800 can be an adjusted rendering of the rendering 650 based on the rendering information of the vehicles 608 and 610. For example, the vehicle 602 can store a representative image (e.g., a stock image, a line drawing, etc.) or other suitable rendering information for the received identifiers of the vehicles 608 and 610. The LIDAR devices 614 and 616 can transmit the vehicle identifiers to the LIDAR device 612, and the vehicle 602 can use the received vehicle identifiers to determine the rendering information of the vehicles 608 and 610 to adjust the rendering 650. FIG. 6B

[0114] The rendering surface 658 associated with the vehicle 608 can be combined with the determined rendering information of the vehicle 608 (e.g., a stock image or stored drawing of the vehicle 608). For example, the image of the vehicle 608 can be aligned with the rendered surface 658 (e.g., by resizing and / or orienting the image of the vehicle 608 to align with the rendered surface 658). In the example rendering 800, the object 808 corresponds to the adjusted rendering of the vehicle 608. Similarly, the object 810 in the rendering 800 corresponds to the adjusted rendering of the vehicle 610.

[0115] Although not shown, other objects that can be represented in the rendering 650 can include infrastructure, buildings, road markings, pedestrians, or other non-vehicles that can have their renderings adjusted. In some examples, the LIDAR devices coupled to the vehicles can transmit information about objects in the environment other than the vehicles. For example, one or more LIDAR devices of the vehicle 608 (e.g., the LIDAR device 614) can be used to determine depths of objects in the environment of the vehicle 608, and one or more LIDAR devices of the vehicle 610 (e.g., the LIDAR device 616) can be used to determine depths of objects in the environment of the vehicle 610. In one example, each of the vehicles 602, 608, and 610 can generate a rendering based on the measurements provided by the LIDAR devices of the respective vehicle. In some implementations, the vehicles 602, 608, and 610 can share the LIDAR measurements with each other (over LIDAR communication), and the vehicle 602 can use the combined LIDAR measurements to generate a more comprehensive rendering of the environment 600 than the rendering 650.

[0116] ​In some implementations, any suitable updated rendering can be displayed on a display of the vehicle 602. For example, an updated rendering can be generated for an area of the environment that is not yet within range or LoS of the LIDAR device of the vehicle 602. The vehicle 610 can detect objects near the street behind the vehicle 610 (e.g., using the LIDAR device 616), which can be used to render the environment 600 that is not within the field of view of the vehicle 612. The vehicle 610 can send rendering information for at least a portion of that environment (e.g., rendering information for pedestrians, bicyclists, parked cars, traffic cones, obstacles, blockages, etc.) to the vehicle 602. In this way, the vehicle 602 can use the rendering information to include additional information about the environment 600 to the right of the rendering 650 (which is blocked from the field of view of the LIDAR device 612 of the vehicle 602 by the building 606). In some aspects, rendering information can be shared between multiple vehicles and infrastructure, and a rendering for a particular vehicle (e.g., the vehicle 602) can include areas far beyond the vehicle. For example, rendering information about the environment of a vehicle multiple hops away from the vehicle 610 can be sent through those hops to the vehicle 610, and then to the vehicle 602. The vehicle 602 can then use the rendering information to extend the area covered by the rendering generated by the vehicle 602.

[0117] In addition to updating the rendering, the received data can be used for navigation or other automation operations of the autonomous vehicle. For example, the updated rendering can indicate additional obstacles or blockages, and thus the vehicle can navigate (e.g., update its navigation) to avoid the additional blockages during operation. Returning to the renderings 650 and 800 in FIGS. 6 and 8, respectively, the rendering 650 does not show the back of the vehicle 610, but the updated rendering 800 shows the entirety of the vehicle 610. If the route of the vehicle 602 is determined to collide with the end of the vehicle 610 based on the received data, the navigation of the vehicle 602 can be updated to avoid the vehicle 610. In another example, the driver and / or passengers can be notified of a possible collision or obstacle based on the updated rendering. For example, the updated rendering including the obstacle or blockage can be displayed to the driver and / or passengers, or the driver and / or passengers can be notified of the obstacle or blockage. In this way, the occupants of the vehicle can be informed of a route change, vehicle slowdown, or other change in navigation of an autonomous vehicle before the obstacle or blockage appears in front of the occupants. Additionally, or alternatively, the obstacle or blockage can be notified to a dispatcher or central office that includes a fleet of vehicles to account for a reason to adjust the navigation of the vehicle. FIG. 6B FIG. 8 In addition to updating the rendering, the received data can be used for navigation or other automation operations of the autonomous vehicle. For example, the updated rendering can indicate additional obstacles or blockages, and thus the vehicle can navigate (e.g., update its navigation) to avoid the additional blockages during operation. Returning to the renderings 650 and 800 in FIGS. 6 and 8, respectively, the rendering 650 does not show the back of the vehicle 610, but the updated rendering 800 shows the entirety of the vehicle 610. If the route of the vehicle 602 is determined to collide with the end of the vehicle 610 based on the received data, the navigation of the vehicle 602 can be updated to avoid the vehicle 610. In another example, the driver and / or passengers can be notified of a possible collision or obstacle based on the updated rendering. For example, the updated rendering including the obstacle or blockage can be displayed to the driver and / or passengers, or the driver and / or passengers can be notified of the obstacle or blockage. In this way, the occupants of the vehicle can be informed of a route change, vehicle slowdown, or other change in navigation of an autonomous vehicle before the obstacle or blockage appears in front of the occupants. Additionally, or alternatively, the obstacle or blockage can be notified to a dispatcher or central office that includes a fleet of vehicles to account for a reason to adjust the navigation of the vehicle.

[0118] ​In addition to updating the rendering 650 to include a model or other representative image of the vehicle (or another object in the environment), the rendering 650 can be updated to include a texture for the model. For example, the rendering of vehicles within the same team can be adjusted to have similar textures to indicate that they belong to the same team.

[0119] FIG. 9A An example rendering 900 of the environment 600 in FIG. 6A is shown, where the vehicle 608 and the vehicle 602 belong to the same team. As shown, objects 902 and 908 in the rendering 900 can be similarly textured (e.g., shaded). The object 910 corresponding to the vehicle 610, which is not part of the same team, can be textured differently than the objects 902 and 908. For example, the object 910 can include different shading than the objects 902 and 908. Objects such as infrastructure, buildings, roads, vehicles, etc. can each be textured differently depending on the type of object. For example, stations can be rendered using the same texture, emergency vehicles rendered using the same texture, etc. In this way, the driver and / or passengers can easily identify similar objects in the environment based on the textures in the displayed rendering. In some implementations, updating the rendering can include highlighting the object rendering. For example, the rendering of an emergency vehicle can be highlighted to notify the occupants of the vehicle of the presence of the emergency vehicle. Similarly, construction zones, safety zones, or other areas of the environment can be highlighted to notify the occupants of the vehicle of their presence in the environment.

[0120] FIG. 9B An example updated rendering 950 of the environment 670 FIG. 6C is shown. The updated rendering 950 includes a rendering 952 of the emergency vehicle 678. The rendering 952 can be highlighted to indicate that the vehicle 678 is an emergency vehicle. The rendering 952 can also include representative images 954 and 956 for the vehicle 674 and 676, respectively, that were not originally included in the rendering 670.

[0121] Referring back to the environment 670 in FIG. 6C , the LoS blocking vehicle 676 and the emergency vehicle 678 can be blocked from the LIDAR device coupled to the vehicle 672. As a result, FIG. 6DThe rendering 690 in FIG. 6B (based on measurements from a LIDAR device) does not include a rendering of the vehicle 676 and the emergency vehicle 678. In some implementations, a LIDAR device coupled to the vehicle 672 can receive reflections of the modulated light signal from a LIDAR device coupled to the vehicle 676 or the emergency vehicle 678. The modulated light signal can include data indicating the presence of the emergency vehicle 678. The data can also indicate other information about the emergency vehicle, such as the type of emergency vehicle, whether the emergency vehicle 678 is responding to an emergency, and / or the travel path, speed, etc. of the emergency vehicle. The data can also include information about other vehicles or objects. For example, the data can indicate the presence of the vehicle 676.

[0122] In some other implementations, the vehicle 672 can receive data about the emergency vehicle 678 from an intermediary device. The vehicle 676 can sense the presence of the emergency vehicle 678 (such as via a LIDAR device coupled to the vehicle 676). The vehicle 676 can transmit data about the emergency vehicle 678 to the vehicle 674 within the LoS via the modulated light signal 680. The vehicle 674 can then transmit the data about the emergency vehicle 678 to the vehicle 672 within the LoS via the modulated light signal 682. Although one intermediary node is shown for transmitting the data, any number of hops can be performed by the data. When the vehicle 672 receives the data about the emergency vehicle 678 (and the data about the vehicle 676), the vehicle 672 can update the rendering 690 to fill in the portion of the occlusion caused by the building 684 and to highlight or otherwise indicate the presence of the emergency vehicle 678. In addition to or instead of highlighting the emergency vehicle, the vehicle 672 can notify the driver and / or passengers by, for example, flashing a display, providing a sound notification, applying a special texture to the rendering, and / or providing a text notification on a display. Although examples are described with reference to emergency vehicles (such as ambulances), the operations can also apply to any vehicle, object, or area of interest (such as a construction zone, school zone, accident zone, etc.). FIG. 6C 、 6D and 9B, the operations can also apply to any vehicle, object, or area of interest (such as a construction zone, school zone, accident zone, etc.).

[0123] In some implementations, rendering information for objects in the environment can be provided via a software as a service (SaaS) model. For example, a unique identifier for an object can be received by the vehicle 602. The vehicle 602 can then communicate with a service (e.g., through a cellular modem or other communication device) to obtain rendering information for the object. For example, the vehicle 602 can receive a vehicle identification number for the vehicle 608, and the vehicle 602 can communicate with a remote server that stores rendering information for a plurality of objects, including the vehicle 608. The remote server can provide the rendering information for the vehicle 608 in response to the request, and the rendering information can be used by the vehicle 602 to update the rendering.

[0124] In some other implementations, the vehicles 602 can store rendering information for objects, and a remote server can provide rendering information for objects that the vehicles 602 do not have locally stored. Additionally or alternatively, the remote server can be used to update locally stored rendering information. For example, when a new make and model of vehicle is released, rendering information for the vehicle can be created that has not yet been stored in the memory of the vehicles 602. LIDAR communication can be used to update the locally stored rendering information of the vehicles 602 to include the new rendering information. In some implementations, the vehicles 602 can update their stored rendering information when coupled to a station (such as the station 408 of FIG. 4) over a dedicated backhaul. FIG. 4

[0125] There are other use cases for LIDAR communication, and the present disclosure is not limited to the above examples. For example, devices (such as vehicles) can include software or firmware that requires periodic updates. LIDAR devices can be used to receive software updates from other devices. For example, a software update including rendering information for a new make and model of vehicle can be provided to a first subset of vehicles. These vehicles can then send the update to other vehicles in the environment via LIDAR communication. In this way, updates can be distributed quickly without requiring all vehicles to connect to a central location (such as a server storing the updates). For example, communication between devices (such as vehicles and infrastructure) can be similar to a peer-to-peer network, and updates can be propagated across devices without requiring a central repository to download the updates. Another example use case can include audio and video transmission between devices (such as for voice calls, messaging, or video conferencing).

[0126] In general, one or more operations can be based on data received via modulated light signals for LIDAR communication. As described herein, LIDAR communication can be used to provide data to, for example, affect navigation of an autonomous vehicle, provide notifications to a driver or passenger, adjust access to a location, etc.

[0127] FIG. 10 A flow diagram depicting example operations 1000 for performing one or more operations based on data received from transmissions of a LIDAR transmitter is shown. Although discussed in terms of devices performing one or more operations for purposes of explaining aspects of the present disclosure, FIG. 10 The operations 1000 can be performed by a vehicle, infrastructure, a dispatcher or central office coordinating a fleet of vehicles, a processing system coupled to a vehicle (such as an entertainment system), an appropriate control system (such as a control system coupled to a vehicle for level 4 self-driving), etc. The example operations 1000 are not limited to being performed by or performing operations of a particular device.

[0128] ​From 1002, the device can receive data associated with a modulated light signal transmitted by a transmitting LIDAR device. For example, a device (e.g., a vehicle, a processing system, etc.) can be coupled to a LIDAR device configured to receive a modulated light signal from a separate transmitting LIDAR device. A receiver of the receiving LIDAR device can receive the modulated light signal, which can include a light carrier signal modulated to carry a data signal including data. The carrier signal can be a signal typically used for detection and ranging. The receiving LIDAR device (or a device coupled to the receiving LIDAR device) can demodulate the modulated light signal to generate and provide the data provided in the modulated light signal. In this way, the device (e.g., a processing system or a vehicle) can receive data associated with the modulated light signal. In some implementations, the data can include one or more packets. Referring back FIG. 3 , an example packet can include information specific to, for example, positioning a vehicle, using a vehicle to access a particular area or location, object rendering, etc.

[0129] Referring back FIG. 10 , the device can determine one or more operations to perform based on the received data (1004). For example, the device can determine one or more operations based on the received one or more packets. In some implementations, the device can determine one or more vehicle positioner operations (1006). In one example, if a vehicle receives information in the data that the vehicle is to be positioned, the vehicle can determine to return to a specified location (e.g., a parking space including a station, as shown in FIG. 4 , and communicate (through LIDAR communication) to the station the location and current status of the vehicle. In another example, if a device receives information about a lost vehicle and the device recently communicated with the lost vehicle, the device can communicate to the vehicle, infrastructure, a dispatcher, etc. the last received location and other information about the lost vehicle. Additionally or alternatively, the device can propagate a lost vehicle message to other vehicles and devices through LIDAR communication.

[0130] In some other implementations, the device can determine one or more vehicle hail operations (1008). Referring back FIG. 4If station 408 is a walk-up station for an automated taxi fleet, station 408 can communicate with a vehicle via LIDAR communication that the vehicle is assigned to a particular customer and ride. The vehicle can receive data from station 408 via LIDAR communication between a LIDAR device coupled to the vehicle and a LIDAR device coupled to the station and located in a separate location from where vehicles are queued. Based on the received data, the vehicle can approach a space near station 408 and communicate with station 408 information about the ride or customer assigned to the vehicle. Station 408 can notify the customer that the vehicle has arrived so that the customer can get in. In another example, the vehicle or device can determine to notify a driver (e.g., a taxi driver) of a customer and / or ride assigned to the vehicle. The taxi driver can approach the customer based on the notification to begin the ride.

[0131] Referring back FIG. 10 In some further implementations, the device can determine one or more notification operations (1010). In some examples, referring back FIG. 5 In another example of a notification operation, the vehicle can receive data that an ambulance 502 or other emergency vehicle is nearby. The vehicle 506 can pull over to the side of the road, stop, or otherwise adjust the operation or navigation of the vehicle to allow the ambulance 502 to pass safely. Additionally or alternatively, the vehicle 506 or another device can determine to notify the driver and / or passenger of the presence of the ambulance 502. In some other examples, the vehicle 506 or device can determine to indicate the presence of the ambulance 502 to other vehicles, infrastructure, or other devices via LIDAR communication (thereby propagating the indication to other vehicles and infrastructure within the area of the ambulance 502).

[0132] In another example of a notification operation, the vehicle can receive data about available parking spaces in a parking garage. Accordingly, the vehicle can drive to one of the available spaces and park therein. The vehicle or device can determine to notify the driver and / or passenger of the available space (e.g., indicate the available space on a displayed map of the parking garage), etc. Other example notification operations can include updating a route based on a notification of an accident or traffic condition along the current route, notifying other vehicles of the accident or traffic condition via LIDAR communication, notifying the driver and / or passenger of a change in route or traffic conditions, etc.

[0133] Referring back FIG. 10In some other implementations, the device can determine one or more limited access region operations (1012). For example, if a downtown area is limited to autonomous vehicles, zero emission vehicles, or other specific types of vehicles, the vehicle can receive data associated with the access limitation from a LIDAR device coupled to the vehicle. The LIDAR device can receive data in a modulated light signal transmitted by a LIDAR device coupled to infrastructure or other suitable device. For example, a gate, toll booth, station, etc. can include a LIDAR device to transmit an indication of an access limitation to the vehicle. The vehicle can determine whether the vehicle is allowed to enter the area based on the indication. If the vehicle is allowed, the vehicle can also determine one or more navigation operations to access the area. For example, if access is limited to autonomous vehicles, an autonomous vehicle can enter the downtown area. Additionally, or as an alternative, the vehicle can determine to notify the driver and / or passenger of the approaching limited access region and the limitation on access. In some examples, the driver can determine whether to access the area based on the notification. In some further examples, the vehicle can be notified of a toll or fee to access a particular area. For example, a bridge, lane, or area of a city can be associated with an access fee. The vehicle can provide account or other payment information through LIDAR communication prior to accessing the infrastructure of the limited area (e.g., provide payment to a toll booth prior to crossing a bridge). In some examples, the driver and / or passenger can be notified of the fee.

[0134] As the limitations of an area can change based on current conditions (e.g., time of day, day of week, current traffic congestion, smoke level, situations requiring the area to be limited to emergency vehicles, etc.), such changes in limitations can be communicated. In response, the vehicle can, for example, determine to notify the driver and / or passenger of the change in access limitations of the area, adjust a route to navigate around the newly limited area, or notify other vehicles of the change through LIDAR communication.

[0135] Referring back to FIG. 10 In some further implementations, the device can determine one or more visualization or rendering operations (1014). For example, an environment of the vehicle, LIDAR device, infrastructure, or other suitable device can be rendered (e.g., generate a point cloud, a mapping of the environment, a depth map, etc.) and the rendering can be displayed to the driver and / or passenger. Referring back to FIG. 6A and 6Bfrom the data provided by the LIDAR device 612. The vehicle 602 (or other suitable device) can also receive one or more identifiers of the vehicles 608 and 610 in transmissions from the respective LIDAR devices 614 and 616. The vehicle 602 (or other suitable device) can therefore determine rendering information for the vehicles 608 and 610 based on the received identifiers (e.g., look up the information in a memory of the vehicle or device, communicate with a remote service to provide rendering information for one or more objects, determine rendering information encoded in the transmissions from the respective LIDAR devices, etc.). The rendering operation can therefore include updating the rendering 650 using the determined rendering information. For example, FIG. 6B The rendering 650 in FIG. 8 may be updated to the rendering 800 in FIG. 3 Other example rendering operations can include applying a defined texture to emergency vehicles, applying similar textures to similar types of objects, applying unique textures specific to each vehicle or object in the environment and associated with a unique identifier such as the unique transmitter ID 302 in

[0136] Referring back to FIG. 10 , the device can then instruct performance of one or more operations (1016). For example, if one or more rendering operations were determined, the device can instruct a display or another device coupled to the display to display the rendering and display the updated rendering. In another example, if the device is a vehicle and the one or more operations are limited access area operations, the vehicle can instruct one or more vehicle components to navigate the vehicle into or out of the area. The device can also provide instructions to a driver and / or passenger for displaying information about the limited access area.

[0137] Although described from the perspective of a vehicle, FIG. 10Some example processes of example operations 1000 are described above, however any suitable device can perform one or more associated operations. For example, a station (or a processing system coupled to a station) can perform a vehicle locator operation (such as contacting a dispatcher) in response to receiving a LIDAR transmission from a lost vehicle. In another example, a station can perform a vehicle hailing operation (such as notifying a customer that a vehicle has arrived) in response to receiving a LIDAR transmission from a hailed vehicle. In another example, a station can perform a notification operation (such as sending a notification of an ambulance in an area) in response to receiving a notification in a LIDAR transmission. In another example, a toll booth or gate can perform a limited access area operation (such as changing an access fee or access restriction for an area) in response to receiving an update of traffic congestion from a LIDAR device of infrastructure or a vehicle in an area. As described above, example operations 1000 are not limited to being performed by a particular device (such as a vehicle).

[0138] Additionally, although example operations (such as ride hailing, access restriction, updated rendering, etc.) are described separately, however a device can be configured to perform any number of operations and combinations of operations. For example, a device can be configured to update a rendering regarding an emergency vehicle, notify a vehicle rider of the presence of an emergency vehicle, update an area restricted to vehicles based on the presence of an emergency vehicle, and update a navigation of a vehicle to avoid the restricted area.

[0139] Those skilled in the art will understand that information and signals can be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that can be referenced throughout the above description can be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof. Additionally, unless otherwise specified, the terms "first," "second," "third," etc. are used herein to denote different instances of an embodiment, and do not imply a particular ordering, unless otherwise specified. Further, the terms "if," "as if," and the like, are used herein to denote conditional nature of an event occurring with regard to an instance of an embodiment, and do not imply logic based decisions, unless otherwise specified. The disclosure of aspects of the disclosure can be implemented by any suitable device, such as an autonomous vehicle, a fleet of autonomous vehicles, infrastructure, a control system of a vehicle or infrastructure, an entertainment system coupled to a vehicle, or other device, and are not limited to the particular examples described herein.

[0140] Furthermore, those skilled in the art will appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the aspects disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure. For example, a processing system of a vehicle or other suitable device can include one or more processors and memory coupled to the one or more processors. The memory can include instructions that are executable by the one or more processors to cause the device to perform the operations described herein. The processing system can also be coupled to one or more LIDAR devices for LIDAR communication and detection and ranging of surfaces. In some aspects, the processing system can include dedicated hardware, such as one or more integrated circuits, configured to perform one or more operations.

[0141] Accordingly, the methods, sequences, or algorithms described in connection with the aspects disclosed herein can be embodied directly in hardware, in software executed by a processor, or in a combination of the two. The software modules can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An example storage medium is coupled to the one or more processors such that the one or more processors can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the one or more processors. The processor and the storage medium can reside in an ASIC.

[0142] In the foregoing specification, the example implementations have been described with reference to specific examples of implementations thereof. It will, however, be evident that various modifications and changes can be made thereto without departing from the broader scope of the disclosure as set forth in the appended claims. For example, while the vehicle is generally illustrated as a car, any suitable vehicle can be used, such as a motorcycle, drone, airplane, boat, helicopter, etc. Furthermore, the vehicle can include one or more LIDAR devices, or the vehicle can be coupled to LIDAR devices manufactured separately from the vehicle. The vehicle can also include one or more processing systems, entertainment systems, control systems, etc., or the vehicle can be coupled to such processing systems, entertainment systems, control systems manufactured separately from the vehicle. For example, one or more LIDAR devices can be attached to different portions of the car to ensure coverage of the environment surrounding the car. The LIDAR devices can be coupled to the processing system of the car (or to the control system of the vehicle), and provide data sent by other LIDAR devices for processing, or send data provided by the processing system to other LIDAR devices via modulated light signals.

[0143] The description and drawings are thus to be regarded as illustrative in nature and not as restrictive.

Claims

1. An apparatus comprising: One or more processors; as well as A memory coupled to one or more processors, the memory including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, including: The transmitter sends a first modulated optical signal to a first optical detection and ranging LIDAR device coupled to a vehicle, wherein the transmitter is coupled to the infrastructure, and wherein the first modulated optical signal comprises an optical carrier signal modulated to include a data signal.

2. The device according to claim 1, wherein, The infrastructure includes at least one of the following: toll booths, road construction areas, tunnel entrances, traffic signals, charging stations, entrance gates, and stations.

3. The device according to claim 1, wherein, The transmitter is part of a second LIDAR device coupled to the infrastructure.

4. The device according to claim 1, wherein, The data signal includes at least one of the following: transmitter identifier, identifier of the location of the infrastructure, and transmitter operating parameters.

5. The device according to claim 1, wherein, The infrastructure controls access to a limited access area, wherein the data signal indicates restrictions on entering the limited access area.

6. The device according to claim 1, wherein, The infrastructure is a toll booth, wherein the data signal indicates the toll for entering the area.

7. The device according to claim 1, wherein, The data signal includes information about at least one of the following: lost vehicle, stolen vehicle, offline vehicle.

8. The device according to claim 1, wherein, The data signal calls a car in order to arrange the requested ride.

9. The device according to claim 1, wherein, The data signal includes a notification to the vehicle, wherein the notification is associated with at least one of the following: emergency vehicle, road hazard, school crossing area, speed limit change, construction area, traffic accident, or traffic obstacle.

10. The device according to claim 1, wherein, The infrastructure is associated with a parking area, and the data signal indicates at least one of the following: available parking space in the parking area or a fee to be charged for parking in the parking area.

11. An apparatus comprising: One or more processors; as well as A memory coupled to one or more processors, the memory including instructions that, when executed by the one or more processors, cause the device to: Receive data associated with a modulated optical signal emitted by a first optical detection and ranging LIDAR device and received by a detector, wherein the first LIDAR device is coupled to a vehicle, wherein the detector is coupled to an infrastructure, and wherein the modulated optical signal comprises an optical carrier signal modulated to include a data signal containing data. Based on the received data, determine one or more operations to be performed; and Send communications related to one or more operations.

12. The device according to claim 11, wherein, The infrastructure includes at least one of the following: toll booths, road construction areas, tunnel entrances, traffic signals, charging stations, entrance gates, and stations.

13. The device according to claim 11, wherein, The detector is part of a second LIDAR device coupled to the infrastructure.

14. The device according to claim 11, wherein, The communications related to one or more operations include information about at least one of the following: a lost vehicle, a stolen vehicle, or an offline vehicle.

15. The device according to claim 11, wherein, The received data indicates that the vehicle cannot communicate with the destination via a specific communication channel, and wherein sending communications related to one or more operations includes: Send vehicle-related communications to the destination.

16. The device according to claim 11, wherein, The received data includes information about customers assigned to the vehicle and / or rides, wherein the one or more operations to be performed include one or more ride-hailing operations.

17. The device according to claim 16, wherein, The communications related to one or more operations include notifying the customer that the vehicle has arrived.

18. The device according to claim 11, wherein, The received data includes notifications, and wherein sending communications related to one or more operations includes: Send a notification to one or more other vehicles.

19. The device according to claim 18, wherein, The notification relates to at least one of the following: emergency vehicles, road hazards, school crossing areas, speed limit changes, construction areas, traffic accidents, and traffic obstacles.

20. The device according to claim 11, wherein, The received data indicates that the vehicle is authorized to enter the restricted access area, and wherein the communication associated with one or more operations is related to authorizing the vehicle to access the restricted access area.

Images