Communication based on optical detection and ranging

Abstract

Systems and methods for performing operations based on LiDAR communication are described. Example devices may include one or more processors; and memory coupled to the one or more processors. The memory includes instructions that, when executed by the one or more processors, cause the device to: receive data associated with a modulated optical signal transmitted by a transmitter of a first LiDAR device and received by a receiver of a second LiDAR device coupled to a vehicle and that device; generate a rendering of the vehicle's environment based on information from the one or more LiDAR devices coupled to the vehicle; and update the rendering based on the received data. Updating the rendering includes updating object rendering of objects in the vehicle's environment. The instructions also cause the device to provide the updated rendering for display on a display coupled to the vehicle.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Patent Application No. 16 / 502,008, filed July 2, 2019, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] This disclosure generally relates to LIDAR devices, and more specifically to the use of LIDAR as a communication channel in self-driving applications. Background Technology

[0004] Light detection and ranging (LIDAR) devices can be used to detect objects in an environment. For example, a vehicle may include or be coupled to one or more LIDAR devices to detect, for example, vehicles, pedestrians, traffic signals, obstacles, etc. The LIDAR device emits light at a specific frequency (e.g., in the 800-1000 nm range or at 1550 nm), and receives the reflections of the emitted light. The LIDAR device then determines the time-of-flight (ToF) of the light to estimate the distances to multiple reflective surfaces while scanning the environment. The estimated distances can be used to generate a point cloud representation of the environment or otherwise for rendering the environment or assisting vehicle operation.

[0005] For example, a vehicle can be configured to operate in an autonomous mode, in which it navigates through its environment with little or no input from the driver. An autonomous vehicle may include one or more LiDAR devices to determine the distances to objects in the environment, and these distances can be used to control the vehicle's navigation. In another example, a vehicle may include one or more LiDAR devices to assist the driver, for example, by performing adaptive cruise control, providing cross-traffic warnings, providing lane departure warnings, and so on, all the way to performing all safety-critical driving functions and monitoring road conditions in a fully autonomous system. Summary of the Invention

[0006] This overview is provided to introduce a range of concepts in a simplified form, which will be further described in the detailed description below. This overview is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0007] The innovative aspects of the subject matter described in this disclosure can be implemented for devices coupled to one or more LIDAR devices. In some implementations, example devices include one or more processors and memory coupled to the one or more processors. The memory includes instructions that, when executed by the one or more processors, cause the device to: receive data associated with a modulated optical signal transmitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to a vehicle and the device; generate a rendering of the environment of the vehicle based on information from the one or more LIDAR devices coupled to the vehicle; and update the rendering based on the received data. Updating the rendering includes updating object rendering of objects in the environment of the vehicle. The instructions cause the device to provide an updated rendering for display on a display coupled to the vehicle.

[0008] The innovative aspects of the subject matter described in this disclosure can be implemented in a computer-readable medium. The computer-readable medium stores instructions that, when executed by one or more processors of a device, cause the device to: receive data associated with a modulated optical signal transmitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to a vehicle and the device; generate a rendering of the environment of the vehicle based on information from the one or more LIDAR devices coupled to the vehicle; and update the rendering based on the received data. Updating the rendering includes updating object rendering of objects in the environment of the vehicle. The instructions cause the device to provide an updated rendering for display on a display coupled to the vehicle.

[0009] The innovative aspects of the subject matter described in this disclosure can be implemented as a method. An example method includes: receiving data associated with a modulated optical signal transmitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to a vehicle and the device; generating a rendering of the environment of the vehicle based on information from one or more LIDAR devices coupled to the vehicle; and updating the rendering based on the received data. Updating the rendering includes updating object rendering of objects in the environment of the vehicle. The method also includes providing the updated rendering for display on a display coupled to the vehicle.

[0010] Another example device may include one or more processors and memory coupled to the one or more processors. The memory includes instructions that, when executed by the one or more processors, cause the device to: receive data associated with a modulated optical signal transmitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to infrastructure (e.g., a toll booth, road construction zone, tunnel entrance, etc.) and the device. The device may generate a rendering of the infrastructure environment based on information from the one or more LIDAR devices coupled to the infrastructure. The device may also update the rendering based on the received data. Updating the rendering may include updating object rendering of objects in the infrastructure environment. The device may also provide updated renderings for display. For example, renderings may be displayed for toll booth attendants, central transportation office auditors, construction site foremen, etc.

[0011] The data may include indications of objects in the environment, and updating object rendering may include highlighting object rendering during display, adjusting the texture of object rendering, including a representative image of the object in the environment rendering, and / or adjusting the dimension of object rendering. Displaying updated rendering may include informing the viewer that an object in the infrastructure environment is an emergency vehicle.

[0012] The device can also determine navigation operations for one or more vehicles in the infrastructure environment based on the received data, and the device can provide adjusted navigation operations to one or more vehicles. Modulated light signals can be received by a second LIDAR device operating in communication mode, and objects in the infrastructure environment and the line of sight of the second LIDAR device can be sensed by the second LIDAR device operating in detection mode.

[0013] The device can also receive second data associated with a second modulated optical signal received by a second LIDAR device, and update one or more entries in its local database based on the received second data. In some implementations, the device can send updated data to other LIDAR devices to update their local databases.

[0014] Details of one or more implementations of the subject matter described in this disclosure are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages will be apparent from the description, drawings, and claims. Note that the relative dimensions in the following figures may not be drawn to scale. Attached Figure Description

[0015] Figure 1 This is a block diagram of an example LIDAR device.

[0016] Figure 2A Example LIDAR devices are shown in the environment.

[0017] Figure 2B This is an example timing diagram corresponding to the waveforms of the transmitted and received light pulses of the example LIDAR device.

[0018] Figure 3 This illustrates an example packet format for multiple received packets transmitted from a LiDAR based on an example communication protocol.

[0019] Figure 4 The example parking space shown includes autonomous fleet vehicles returning to their parking spaces when not in use.

[0020] Figure 5 This example environment is shown to communicate the presence of an ambulance to another vehicle in the environment.

[0021] Figure 6A An example environment for the vehicle is shown.

[0022] Figure 6B This shows an example rendering of the vehicle's environment as sensed by a LIDAR device.

[0023] Figure 6C An example environment is shown, including emergency vehicles.

[0024] Figure 6D Showing the data sensed by the LIDAR device Figure 6C Example rendering of the vehicle's environment.

[0025] Figure 7 A flowchart depicting an example operation of rendering for adjusting the environment of a LIDAR device is shown.

[0026] Figure 8 Show Figure 6A An example of an adjusted rendering of the environment.

[0027] Figure 9A Show Figure 6A An example rendering of an environment where two vehicles belong to the same convoy.

[0028] Figure 9B Show Figure 6C Example rendering of an environment, where the environment includes an emergency vehicle.

[0029] Figure 10 A flowchart is shown depicting an example operation for performing one or more operations based on data received from a transmission from a LIDAR transmitter.

[0030] The same reference numerals and names in the various figures indicate the same elements. Detailed Implementation

[0031] Efforts have been made to couple vehicles and infrastructure via cellular communications. For example, some vehicles include cellular modulators to communicate with other vehicles or equipment via the 5 GHz radio spectrum. In another example, a vehicle may be equipped with dedicated short-range communication (DSRC) equipment to communicate with other vehicles or equipment via the 5.9 GHz radio spectrum. One problem with cellular or radio technologies used for communications is that vehicles may require cellular connectivity or require another vehicle or infrastructure equipped with consistent communication technology. For example, some rural or saturated urban environments may not have available cellular connectivity, and vehicles may not be able to communicate with other vehicles via their cellular modems. Another problem with cellular technologies used for communications is the inherent latency associated with communications. For example, cellular communication between vehicles may require communication through one or more base stations in a cellular network, which can delay communication. Furthermore, the wavelength of the radio signals used for communication limits the communication bandwidth. In addition, cellular and radio signals are typically omnidirectional, and transmitting multiple omnidirectional signals over the air can quickly saturate an environment.

[0032] Many vehicles and infrastructures (such as toll booths, traffic signals, charging stations, etc.) can include or be coupled to one or more LiDAR devices. Furthermore, as autonomous vehicles become more prevalent, the LiDAR devices they carry will also become more common. In some respects, in addition to performing Time-of-Flight (ToF) measurements (for detecting surfaces in the environment), LiDAR devices can be configured to communicate with other LiDAR devices. Because the emitted light signals have higher frequencies than radio signals, communication between LiDAR devices can potentially have higher bandwidth than cellular communication. Furthermore, communication between LiDAR devices does not require cellular networks or other infrastructure. Additionally, the light signals (e.g., signals with wavelengths close to 1000 nm) can be emitted in a focused dispersion mode to prevent saturation and interference from multiple light signals simultaneously transmitted in the air.

[0033] The implementation of the subject matter described herein allows a LiDAR device to communicate with another LiDAR device (referred to herein as "LIDAR communication"). LiDAR communication can be used for vehicle-to-vehicle (V2V) communication or vehicle-to-infrastructure (V2X) communication, and it can occur between any compatible vehicles and / or infrastructure (e.g., within a fleet or between unrelated vehicles, including configured LiDAR devices). LiDAR communication can be used in a variety of situations and use cases, as described herein.

[0034] For the purpose of describing the innovative aspects of this disclosure, the following description is directed to a particular implementation. However, those skilled in the art will readily recognize that the teachings herein can be applied in a variety of different ways. The described implementations can be implemented in any device, system, or vehicle that includes or is coupled to one or more LiDAR devices. In some implementations, “device” for performing the operations described herein can refer to a control device or system coupled to a vehicle and one or more LiDAR devices; a vehicle that includes a control device or system and is coupled to one or more LiDAR devices; a control device or system coupled to infrastructure or another non-vehicle system; or other suitable implementations. Similarly, “vehicle” can refer to a control device or system coupled to that vehicle; a vehicle separate from the control device or system coupled to the vehicle; a combination of a control device or system and a coupled vehicle; or other suitable implementations.

[0035] In the following description, numerous specific details, such as examples of specific components, systems, and processes, are set forth to provide a thorough understanding of this disclosure. Furthermore, specific nomenclature and / or details are set forth in the following description and for illustrative purposes to provide a thorough understanding of the exemplary embodiments. However, it will be apparent to those skilled in the art that these specific details may not be necessary to practice the exemplary embodiments. In other instances, well-known circuits, systems, and devices are illustrated in block diagram form to avoid obscuring this disclosure. Any signal provided via the various buses described herein may be time-division multiplexed with other signals and provided via one or more general-purpose buses. Additionally, interconnections between components or software blocks may be shown as buses or single signal lines. Each bus may alternatively be a single signal line, and each of the single signal lines may alternatively be a bus, and a single line or bus may represent any one or more of a number of physical or logical mechanisms for communication between components. Furthermore, the term "coupling" as used herein means direct coupling to or through one or more intermediate components or devices.

[0036] Figure 1This is a block diagram of an example Light Detection and Ranging (LIDAR) device 100. The LIDAR device 100 can be used to detect object surfaces in the environment by emitting light pulses that illuminate the surface of an object and by detecting the light pulses reflected from the object surface. The LIDAR device 100 can determine the distance to the object based on the time delay between the emission of the light pulse and the reception of the corresponding light pulse reflected from the selected object. This time delay, referred to as the Time-of-Flight (ToF) of the light pulse, can be multiplied by the speed of light to determine the distance between the LIDAR device 100 and the object. Multiple pulses can be used to determine distance information for multiple points associated with an object in the environment. These points can be used to generate point clouds or to determine the object's position, size, shape, orientation, and motion. In some implementations, information from the LIDAR device 100 can be used to control, for example, autonomous vehicles, enabling them to navigate to a destination in the environment while avoiding obstacles. The LIDAR device 100 can also be used to measure distances for driver-assisted operations.

[0037] LIDAR device 100 is shown to include a transmitter 110, a receiver 120, and a LIDAR controller 130. The transmitter 110 may include a transmission controller 111, one or more light emitters 112, and a transmission aperture 113. The light emitters 112 may emit one or more light pulses 125, which can be used to detect objects in the surrounding environment. The light emitters 112 may include any number of suitable light sources, such as (but not limited to) laser diodes, light-emitting diodes (LEDs), vertical-cavity surface-emitting lasers (VCSELs), organic light-emitting diodes (OLEDs), polymer light-emitting diodes (PLEDs), light-emitting polymers (LEPs), liquid crystal displays (LCDs), microelectromechanical systems (MEMS), or any other device configured to selectively transmit or emit light pulses 125 of defined wavelengths. The source wavelength may include, for example, the ultraviolet, visible, and / or infrared portions of the electromagnetic spectrum. In some aspects, the light emitters 112 may be disposed on one or more substrates (e.g., printed circuit boards (PCBs), flexible PCBs, etc.). Although the optical emitter 112 is described herein as emitting optical pulses 115, those skilled in the art will readily understand that the optical emitter 112 can send or emit optical signals, light beams, photons, etc. Therefore, the terms optical pulse, optical signal, light beam, and photon are used interchangeably herein.

[0038] The transmitting aperture 113 is coupled to the light emitter 112 and may include any suitable component (e.g., mirror, lens, diffraction grating, exit aperture, etc.) capable of focusing, guiding, and / or conditioning the light pulse 115 to be emitted into the surrounding environment. In some implementations, the transmitting aperture 113 may be configured to guide the light pulse 115 in one or more designated directions relative to the LIDAR device 100. The designated directions may span a range of directions, for example, such that the distance between the LIDAR device 100 and multiple objects (e.g., vehicles, people, roads, traffic signals, traffic signs, obstacles, etc.) can be determined based on the reflection of the light pulse 115 caused by objects.

[0039] The transmit controller 111 controls the operation of the optical emitter 112 and the transmit aperture 113, and can adjust multiple parameters or settings of the optical emitter 112 and the transmit aperture 113, or both. In some implementations, the transmit controller 111 may respond to one or more control signals provided by the LIDAR controller 130. For example, the transmit controller 111 may adjust the width, timing, frequency, and / or amplitude (intensity) of the optical pulses 115 emitted by the optical emitter 112 based on one or more control signals. In other implementations, the transmit controller 111 may be omitted, or may be included within the LIDAR controller 130.

[0040] Receiver 120 may include a plurality of photodetectors 121, detector circuitry 122, and analog-to-digital converter (ADC) 123. Photodetectors 121 may receive light pulses 125 (e.g., photons) from the surrounding environment. In some implementations, the received light pulses 125 may include components of emitted light pulses 115 reflected from one or more objects in the surrounding environment. Photodetectors 121 may be configured to convert the received light pulses 125 into photodetector signals (e.g., analog current signals) indicating the intensity level of the received light pulses 125. Photodetectors 121 may be any suitable component or device capable of receiving or sensing light, including, for example, photodiodes (e.g., avalanche photodiodes), silicon photomultiplier tubes (SiPMs), phototransistors, cameras (e.g., CMOS sensors), active pixel sensors (APS), charge-coupled devices (CCDs), cryogenic detectors, etc. In some implementations, photodetectors 121 are reverse-biased photodiodes that, for example, generate current in response to a received light pulse, such that the amount of current through each photodiode is proportional to the intensity of the light pulse received by the photodiode.

[0041] Although not shown for simplicity, receiver 120 may include optics to filter the wavelength of the received light, such that photodetector 121 primarily receives light corresponding to the wavelength of the light pulse 115 emitted by transmitter 110 (and receives a minimum of light corresponding to other wavelengths). For example, receiver 120 may include a bandpass filter to filter optical signals outside the wavelength range centered on the fundamental wavelength of the light pulse emitted by transmitter 110.

[0042] Detector circuit 122 can use any suitable technique to sample the photodetector signal provided by photodetector 121 to determine the intensity level of the received light pulse 125. In some implementations, detector circuit 122 can sample the photodetector signal at multiple intervals or sampling times. In other implementations, detector circuit 122 can sample the photodetector signal continuously. Detector circuit 122 can provide the determined intensity level to ADC 123, for example, as an analog signal having an amplitude (e.g., voltage amplitude or current amplitude) indicating the light information contained in the photodetector signal. In some aspects, detector circuit 122 can amplify and / or filter the photodetector signal.

[0043] ADC 123 can receive an analog signal from detector circuitry 122 indicating the intensity level of the received light pulse 125, and can convert the analog signal into digital data that can be processed by LIDAR controller 130. ADC 123 can be any suitable ADC, such as (but not limited to) a flash ADC, a successive approximation register (SAR) ADC, or a delta-sigma ADC. In some implementations, each photodetector 121 may correspond to a corresponding ADC. In other implementations, multiple photodetectors 121 may correspond to a single ADC (e.g., to reduce the size, cost, and / or power consumption of LIDAR device 100). In some other implementations, ADC 123 may be omitted. Each photodetector 121 (and its corresponding ADC 123) may be associated with a specific light emitter 112. In this way, multiple Time-of-Flight (ToF) measurements can be performed, thus allowing multiple distances to be determined during a one-pass of LIDAR device 100.

[0044] The LIDAR controller 130 may include a processor 131, a memory 132, and a digital signal processor (DSP) 133. The DSP 133 may process digital data provided by one or more ADCs 123 to determine information about light pulses received by any number of photodetectors 121. In some implementations, the intensity and / or arrival time of the light pulses can be used to determine the size, shape, and location of multiple detected objects in the surrounding environment. For example, if the departure time of the light pulse is known, the DSP 133 can use the arrival time to determine the Time of Flight (ToF). In another example, if the intensity of the emitted light pulse is known, the DSP 133 can use the measured intensity to determine the energy loss of the reflected light pulse. Objects relatively close to the LIDAR device 100 may reflect the emitted light pulse 115 before objects relatively far from the LIDAR device 100. Additionally, light reflected from objects relatively close to the LIDAR device 100 has less pulse broadening than light reflected from objects relatively far from the LIDAR device 100 (assuming similar surface reflectivity between objects at different distances). Therefore, in some implementations, the distance between the LIDAR device 100 and the object can be estimated based on the rising and falling edges of the received optical pulse 125.

[0045] Processor 131 may be any suitable one or more processors capable of executing scripts or instructions of one or more software programs stored in LIDAR device 100 (e.g., within memory 132). In some implementations, processor 131 may include one or more microprocessors and memory providing at least a portion of a machine-readable medium in which program instructions or scripts may be stored. In other implementations, processor 131 may be an application-specific integrated circuit (ASIC). In some other implementations, processor 131 may be or include one or more field-programmable gate arrays (FPGAs) or programmable logic devices (PLDs).

[0046] Memory 132 may store information relating to transmitter 110, receiver 120, the surrounding environment, or any combination thereof. Memory 132 may also include a non-transitory computer-readable medium (e.g., one or more non-transitory memory elements, such as EPROM, EEPROM, flash memory, hard drives, etc.) capable of storing numerous software (SW) modules, each including instructions that, when executed by processor 131, cause LIDAR device 100 to perform all or some of the operations described herein. In some other implementations, LIDAR controller 130 may be instructed by one or more processors external to LIDAR device 100 (e.g., included in a vehicle processing system coupled to LIDAR device 100) to perform one or more operations relating to the transmission or reception of light pulses. For example, LIDAR device 100 may be coupled to a vehicle’s processing hub (e.g., via a Controller Area Network (CAN) bus) or another processing system of the vehicle, and the processing system may instruct LIDAR device 100 to perform one or more operations, and receive information from LIDAR device 100 in response (e.g., retrieving ToF or intensity information measured by LIDAR device 100, which is used by the processing system to generate point clouds or other types of depth maps or environmental renderings).

[0047] Figure 2A This shows a sample LiDAR device in the environment. Figure 2A In the example, the LIDAR device 100 is located in an environment 200 including a vehicle 201 and an overhang 202. In a simplified example, the LIDAR device 100 is shown as including three light emitters 112A-112C that emit corresponding light pulses 115A-115C into the environment 200. The LIDAR device 100 is also shown as including three photodetectors 121A-121C that receive corresponding light pulses 125A-125C reflected from the surfaces of objects in the environment 200. A first light pulse 115A illuminates a surface 205A of the overhang 202, and the first photodetector 121A receives the corresponding reflected light pulse 125A. A second light pulse 115B illuminates a surface 205B of the vehicle 201, and the second photodetector 121 receives the corresponding reflected light pulse 125B. A third light pulse 115C illuminates another surface 205C of the vehicle 201, and the third photodetector 121C receives the corresponding reflected light pulse 125C. The LIDAR device 100 can use one or more attributes (such as timing, amplitude, pulse width, etc.) of the received optical pulses 125A-125C to determine the distance between the LIDAR device 100 and each of the surfaces 205A-205C in the environment 200.

[0048] Figure 2BExample timing diagram 210 shows the waveforms corresponding to the transmitted and received optical pulses of the example LIDAR device. Transmitted waveforms 215A-215C can indicate the source... Figure 2A The intensity level of the corresponding light pulses 115A-115C emitted by the LIDAR device 100, and the received waveforms 225A-225C can indicate... Figure 2A The intensity levels of the corresponding optical pulses 125A-125C received by the LIDAR device 100. Optical pulses 115A-115C are emitted from the LIDAR device 100 at the same time t0 (or at least substantially simultaneously), and at different times t... A -t C (For example, due to the varying distances between each surface 205A-205C of the LIDAR device 100 and the environment 200) reflected light pulses 125A-125C are received by the LIDAR device 100. Transmitted waveforms 215A-215C include corresponding pulses 216A-216C representing time t0, at which time corresponding light pulses 115A-115C are emitted from the LIDAR device 100. Received waveforms 225A-225C include those representing time t... A -t C The corresponding pulses 216A-216C, at time t A -t C The corresponding reflected light pulses 125A-125C are received by the LIDAR device 100. The reception time t from the corresponding pulses 226A-226C can be determined using any suitable peak detection technique (e.g., determining peak amplitude, determining centroid, determining the average time between threshold crossovers, etc.). A -t C The determined reception time t A -t C It can be used to determine the distance between the LIDAR device 100 and the corresponding surfaces 205A-205C of the environment 200.

[0049] Many LIDAR devices allow adjustment of one or more characteristics of the emitted light pulse. For example, LIDAR device 100 ( Figure 1The transmitter 110 can be configured to adjust one or more of the timing, frequency, or intensity of the light pulses used to transmit the light 115. In some examples, the LIDAR device 100 allows dithering of the light pulses of the transmitted light 115 to control timing. Some LIDAR devices may allow adjustment of the power supplied to the light transmitter 112 (e.g., an LED) to control intensity. Some LIDAR devices may allow adjustment of the reference signal frequency (e.g., a light source) to adjust the frequency of the light pulses of the transmitted light 115. The receiver 120 can also be configured to sense differences in timing, frequency, or intensity of the light pulses of the received light 125. For example, the sensitivity of the photodiode array of the photodetector 121 may be sufficient to determine intensity differences, and the sampling rate of the photodiode array and / or the ADC may be sufficient to determine differences in timing or frequency of the light pulses of the received light 125.

[0050] In some aspects, LIDAR device 100 (e.g., LIDAR controller 130) can be configured to encode information into transmitted light 115 by frequency adjustment, intensity adjustment, and / or timing jitter of the pulse to be transmitted, and transmitter 110 can be configured to convey such encoded information through the transmitted light 115. For example, LIDAR device 100 can receive a data signal to be transmitted, and LIDAR device 100 can use the light to be transmitted (which may be referred to herein as an optical signal) as a carrier signal. In this way, LIDAR device 100 can modulate the optical signal to include the data signal, and LIDAR device 100 can transmit the modulated optical signal. For the first LIDAR device, receiver 120 can be configured to receive the modulated optical signal transmitted by transmitter 110 of the second LIDAR device. The modulated optical signal is encoded with the data signal from the second LIDAR device, and LIDAR controller 130 of the first LIDAR device can be configured to extract the data signal from the received modulated optical signal. For example, LIDAR controller 130 can demodulate the received optical signal to generate a data signal. Because this communication between LIDAR devices is point-to-point (and therefore does not require a central network), and the signal frequency is higher than that of cellular communication, the throughput may be higher and the latency may be lower than that of traditional cellular communication. "Modulated optical signal" here refers to an optical signal modulated to include data signals (e.g., light emitted by LIDAR device 100).

[0051] In some implementations, the LIDAR device 100 can be configured to switch between a distance measurement mode (which may be referred to herein as a "detection mode") and a communication mode for transmitting modulated optical signals, including data signals. For example, the LIDAR controller 130 can determine when the LIDAR device 100 transmits or receives information via modulated optical signals, and when the LIDAR device 100 operates to detect the surface of objects in the environment. For example, the LIDAR controller 130 can determine to place the LIDAR device 100 in communication mode in a first time period and in detection mode in a second time period. The LIDAR device 100 can thus switch between communication mode and detection mode. In some other implementations, the emitted light 115 can be used to convey information to another receiver (e.g., by adjusting the frequency) and also to detect the surface of objects (e.g., by sensing intensity differences in the received light 125), and the LIDAR device 100 can be configured to perform both modes simultaneously. For example, the LIDAR device 100 can emit modulated optical signals (including data signals). LIDAR device 100 can receive the reflection of a modulated light signal and use the reflection to determine the depth of an object from LIDAR device 100. Additionally, a second LIDAR device can receive the modulated light signal from LIDAR device 100 and demodulate the light signal to generate a data signal.

[0052] Devices such as vehicles or infrastructure may include, or be coupled to, one or more LIDAR devices (e.g., LIDAR device 100) configured to transmit data signals (provided by, for example, vehicles or infrastructure) via emitted light 115. In this way, the device can communicate with vehicles or infrastructure using the configured LIDAR device. In some aspects, LIDAR device 100 may be configured to use communication protocols adopted by other vehicles and infrastructure including LIDAR devices. The communication protocol may be self-organizing or managed, and any suitable packetization of information can be used for the communication protocol. For example, LIDAR communication within a vehicle convoy may be based on a specific protocol. In some implementations, standardized protocols (or multi-party adopted protocols) can more easily allow the integration of LIDAR communication between vehicles and infrastructure. Such protocols may include defined packet formats for sending and receiving information. Protocols from other communication media, such as cellular communication, Wi-Fi communication, digital subscriber line (DSL) communication, fiber optic communication, etc., can be utilized to create protocols for communication between LIDAR devices.

[0053] Figure 3An example packet format for multiple received packets 300 based on an example communication protocol is shown. In some implementations, the LIDAR device 100 may include a buffer or other suitable memory (e.g., memory 132) for queuing one or more received packets 300 (e.g., packets 1-6). The LIDAR device 100 may process the buffered packets for a vehicle or infrastructure to perform one or more operations. In another example, the buffered packets may be provided to the device's processing system for processing. As shown, the format of packets 1-6 includes fields 302-308, which include transmitter ID 302, location 304, payload type 306, and payload 308. Additional fields, fewer fields, different fields, or different field organization may be included in the packets, as any suitable packet format can be used. For example, packets 1-6 may include a Cyclic Redundancy Check (CRC) field after payload 308 to correct for any errors in the received packets.

[0054] Transmitter ID 302 can indicate the device sending the packets. In some implementations, each vehicle may include a unique identifier to identify that particular vehicle. For example, if a convoy of 200 autonomous vehicles is communicating with each other, each vehicle may include a unique identifier relative to other vehicles in the convoy (e.g., “Vehicle_1” through “Vehicle_200”). If a LiDAR device mounted on a vehicle associated with “Vehicle_100” sends packets 1 and 2 to LiDAR device 100, the transmitter ID 302 of the packets may include the unique identifier “Vehicle_100”. Similarly, if a LiDAR device mounted on a vehicle associated with “Vehicle_102” sends packet 3 to LiDAR device 100, the transmitter ID 302 of the packet may include the unique identifier “Vehicle_102”. Some vehicles may not include a unique identifier or may not be identified, but the vehicle may still transmit packets to LiDAR device 100. In one example, transmitter ID 302 may include a transmitter ID value specific to a previously unidentified vehicle or device (referred to as “Unidentified_Vehicle” for group 4). The transmitter ID value may be, for example, a field with zero padding, a null value, or otherwise appropriately padded to indicate that there is no unique identifier for the vehicle or infrastructure associated with the group.

[0055] Infrastructure such as toll booths, tunnels, entrance gates for high-load vehicle (HOV) lanes, etc., may include or be coupled to one or more LiDAR devices to send packets to and receive packets from LiDAR device 100. For example, packet 5 may be sent by a transmitter located at the tunnel entrance (which may be uniquely identified as "Tunnel_40"), while packet 6 may be sent by a transmitter located at the toll booth (which may be uniquely identified as "Toll Booth_30"). Any suitable vehicle or infrastructure may include LiDAR devices for LiDAR communication and may include a unique transmitter ID.

[0056] In some implementations, the emergency vehicle, construction zone, and other equipment associated with priority transmission may include a transmitter ID indicating that the packet is for priority transmission. If the LIDAR device 100 decodes the packets sequentially, the LIDAR device 100 may process the transmitter ID 302 first and determine that the packet is for priority transmission. In this way, the LIDAR device 100 can determine that the packet will be prioritized before completing processing of the rest of the packet. Other suitable implementations of the transmitter ID 302 may be used, and this disclosure is not limited to the examples provided. For example, the transmitter ID may be configured to distinguish between fleets, vehicle types (e.g., private cars, commercial trucks, school buses, etc.), etc.

[0057] Location 304 may indicate the location of a vehicle or infrastructure associated with the transmitting device. For example, the vehicle or infrastructure may include a Global Positioning System (GPS) receiver to determine latitude and longitude. The latitude and longitude can then be provided in location 304, as shown, for example, for groups 1-6. Alternatively, location 304 may include positioning information relative to LiDAR device 100, as determined by other vehicles or infrastructure via their own LiDAR devices. For example, the transmitting LiDAR device may determine the relative distance and location between the transmitting and receiving vehicles or infrastructure, and may transmit the device distance and location in location 304. In some implementations, if location information cannot be transmitted (e.g., the vehicle does not include a GPS receiver), location 304 may be padded with zeros or otherwise filled to indicate that no location is provided.

[0058] Payload type 306 can indicate the type of information to be provided in the payload 308 of a packet. In example packets 1-6, payload type 306 of packet 1 indicates "heartbeat" or "beat," which can be a signal periodically transmitted by "vehicle_100." In some implementations, "heartbeat" or "beat" can indicate that the packet resembles a beacon and can provide basic information in payload 308. In some examples of providing basic information about a vehicle or infrastructure, payload 1 may include jitter rate or other operating parameters for the transmitter and / or trajectory information or other status information for the vehicle. Alternatively, no information may be provided in payload 308 for such packets (e.g., the payload is zero-padded, and the packet is only used to provide location information via location 304 and to notify other devices of the presence of a transmitter).

[0059] Payload type 306 in group 2 is “Render_Description”. As described in the following example use case, LIDAR device 100 may measure surfaces within its line of sight (LoS), but surfaces outside its LoS will not be sensed and detected. In this way, a rendering of the environment based on measurements from LIDAR device 100 may not include vehicle portions not within its field of view. “Render_Description” may indicate basic information about the rendered vehicle or infrastructure that can be provided in payload 308. For example, payload 2 may include the dimensions of one or more shapes associated with the “Vehicle_100” to be rendered. In one implementation, dimensions may include the orientation and size of the geometry (e.g., a rectangle for 2D rendering or a rectangular prism for 3D rendering). Other suitable rendering information may include the texture, contours, or other features of the object being rendered as indicated by payload 308. For example, an emergency vehicle or road construction zone may be highlighted in a rendering of the vehicle as it is shown to the driver and / or passengers. Payload 308 can therefore be used to indicate textures to highlight the rendered portions associated with emergency vehicles or construction zones. In another example implementation, a similar texture could be used for vehicles within the same convoy. In this way, drivers and / or passengers can easily identify convoy vehicles in the displayed rendering. However, any suitable rendering description can be used, and this disclosure is not limited to the examples provided.

[0060] As an addition to or alternative to “Render_Description”, payload type 306 may indicate “Render_Identification”, for example, for group 4. In some implementations, a list of standard vehicles (or infrastructures) and their associated rendering information may be stored in a database (such as a lookup table or other suitable group of data organized for access). For example, if the vehicle sending the group is 1994 Geo Metro, payload 308 (such as payload 4) may indicate “1994 Geo Metro”. The database (such as the size and texture of the render) may be used to determine rendering details for a particular vehicle, and existing renders may be enhanced with additional rendering details. The database may also include information about a particular vehicle, such as length, acceleration capabilities, etc. The term “render” as used herein may refer to line drawings, point clouds, depth maps, images, textures, shading, or other information that can be visualized or displayed.

[0061] The payload type in group 5 is “GATED-ENTRY,” which indicates the type of vehicle permitted to enter “Tunnel_40.” For example, “Tunnel_40” may be restricted to high-load or autonomous vehicles, which can be indicated in group 5. Therefore, vehicles including LIDAR device 100 can determine whether they are permitted to enter “Tunnel_40” based on the information in payload 5.

[0062] The payload type for group 6 is "Fee_Charge," which indicates that payload 6 instructs "Toll Booth_30" to collect a fee. For example, if the toll is $5, payload 6 could instruct that $5 will be automatically charged from the driver's or fleet's account when a vehicle passes through "Toll Booth_30." In some implementations, payload 308 may include a combination of the fee to be collected and restrictions on vehicle entry. For example, a transmitter for a parking garage space could indicate the parking rate and the vehicles permitted in that space (e.g., space reserved for electric vehicles, designated disabled parking spaces, etc.). When a vehicle is parked in that space, the transmitter can update the parking fee based on the rate, allowing the driver to know the current parking cost.

[0063] The payload type of packet 3 is "vehicle_locator," which indicates that payload 3 indicates the vehicle being sought. For example, if "vehicle_005" is offline, vehicles in the convoy might send packets attempting to locate "vehicle_005." In this way, payload 3 can include a "vehicle_005" identifier to indicate which vehicle to locate. In some implementations, LIDAR device 100 can retain multiple transmitter IDs for received packets. Therefore, LIDAR device 100 can be configured to search the retained transmitter IDs of received packets to determine if "vehicle_005" is communicating with LIDAR device 100. LIDAR device 100 can then indicate this to "vehicle_102." For example, LIDAR device 100 can send the location received in a packet from "vehicle_005" to "vehicle_102." In some other implementations, LIDAR device 100 can be configured to propagate the message to other vehicles or infrastructure, thereby indicating that "vehicle_005" should be located. In this way, if "Vehicle_005" receives a transmitted message from other vehicles or infrastructure, "Vehicle_005" can determine and communicate its location to the convoy.

[0064] Other suitable payload types and payloads may exist (e.g., for different use cases of LIDAR communication as described herein), and this disclosure is not limited to the examples provided. In some example implementations, the vehicle may include multiple sensors to collect information for packet generation. For example, the payload may include information about passenger numbers, vehicle operating modes, etc., and such information can be collected using sensors such as pressure sensors, occupancy sensors, engine sensors, etc., to detect operating modes.

[0065] A wide variety of information can be provided and received via LIDAR communication, and LIDAR communication can be applied to a variety of use cases as described in this article.

[0066] LIDAR communication implementation

[0067] Vehicle locator

[0068] Vehicles may operate in areas without a cellular network for communication. For example, when a vehicle travels through rural or sparsely populated areas, it may lack a consistent cellular connection or be unable to communicate with a base station. In another example, the vehicle's cellular modem may be inoperable. If the vehicle is part of a fleet of vehicles (such as taxis or rental cars), a dispatcher may be unable to identify the vehicle's location and may need to locate it.

[0069] In some implementations, searching for lost, stranded, or offline vehicles can be proactive, where the vehicle or infrastructure that finds an offline vehicle reports its location to the dispatcher. In other implementations, searching for offline vehicles can be reactive, where the vehicle records the last known location of the offline vehicle. In this way, the vehicle reports its discovery while synchronizing with the dispatcher or other parts of the fleet (e.g., when the vehicle is charging or otherwise not operating).

[0070] Other vehicles in the convoy can be instructed to send vehicle locator messages via LiDAR communication. For example, each vehicle in the convoy can be periodically instructed to send packets with a payload type of "vehicle_locator," identifying an offline vehicle within the packet payload. The dispatcher can also instruct infrastructure (e.g., convoy-owned electric vehicle charging stations or parking spaces that include or are coupled to LiDAR equipment) to send vehicle locator messages. As mentioned above, other vehicles can also propagate such messages upon receipt, increasing the number of transmitters sending vehicle locator messages. In this way, a vehicle that may be offline relative to its primary communication method (e.g., cellular) can receive messages from passing vehicles or infrastructure, and that vehicle can determine its location and communicate it to the dispatcher via LiDAR communication. In another example, a vehicle or infrastructure that locates an offline vehicle can communicate the offline vehicle's location to the dispatcher.

[0071] For example, if another vehicle in a convoy is communicating with an offline vehicle via LiDAR, the offline vehicle can send a message to the other convoy vehicle via LiDAR communication that its cellular modem is inoperable, and contact the dispatcher, for example, through the other vehicle's cellular modem. In this way, the dispatcher can communicate with the offline vehicle through other convoy vehicles.

[0072] As a dispatcher, vehicle locator messages can be used to locate additional or replacement vehicles in an offline fleet, such as for stolen vehicles, in emergency situations. The payload of a vehicle locator packet can include a description of the vehicle to be located, and such packets can be propagated to other vehicles and infrastructure to find the stolen vehicle. In some implementations, the vehicle can display a notification to the driver in response to receiving the packet. In other implementations, stolen vehicles can be automatically identified based on their size or other attributes. For example, the surface of other vehicles detected using LIDAR equipment can be used to identify vehicles of the same brand and model based on similar size or other characteristics of the stolen vehicle. In another example, a visible light camera can be used to capture an image of the license plate after a lost vehicle is detected, and the image can be analyzed to identify the lost vehicle based on the license plate number. Vehicles can report the location of lost vehicles to a central office (e.g., via a cellular modem).

[0073] In some other implementations, vehicle locator messages can be initiated by offline or disabled vehicles. For example, a fleet vehicle's cellular modem may be unavailable, and the fleet vehicle may not be able to communicate with the fleet via the cellular network. If a dispatcher provides information about where a fleet vehicle is going via cellular communication, the fleet vehicle may not be able to determine where the dispatcher wants to send the vehicle. In the example of an autonomous vehicle taxi fleet, the dispatcher may want to send a vehicle to pick up a customer at a specific address and then have the customer take a taxi to a specific destination. However, the vehicle cannot receive instructions from the dispatcher. LiDAR communication can be used to provide the dispatcher with any messages from vehicles (such as vehicles with disabled cellular modems). For example, a vehicle can send a message to the dispatcher from other fleet vehicles via LiDAR communication. Other fleet vehicles can then forward the message to the dispatcher. In this way, the dispatcher is informed that a vehicle may not be able to receive messages from the dispatcher, and the dispatcher can update the processing request even if the vehicle is not in the fleet. Dispatchers can also give instructions to vehicles via LIDAR communication (e.g., via other fleet vehicles) to wait for service at a designated location, remove themselves from fleet operations, or any other suitable operations.

[0074] In some implementations, a vehicle may have a home location remote from the taxi fleet dispatcher. For example, if the vehicle has no destination (e.g., for a fare communicated to the dispatcher), it may return to a designated location within the service area, such as a specific parking space within that service area. This space may be associated with a LiDAR device that can communicate with the vehicle (e.g., allowing the dispatcher to communicate with the vehicle). The designated location may be the vehicle's parking area when it is fully operational and returns to standard communication with the dispatcher, until the vehicle is activated to collect the fare. The vehicle may also return to a designated location (such as a parking space) when it is unable to communicate with the dispatcher and is not used for fare collection. The term "offline vehicle" as used herein can refer to a stationary, lost, or otherwise inactive vehicle or one that may not be expected to operate in the default, typical, or standard operating mode.

[0075] Figure 4An environment 400 is shown, including an example parking space 410 for autonomous fleet vehicles 402 to return to when not in use. Autonomous vehicles 402 include one or more LiDAR devices, such as LiDAR device 404. A stand 408 may be located near the parking space 410. Examples of stands 408 include parking meters, charging stations, and taxi stands for walking customers. In some implementations, stand 408 includes a LiDAR device that communicates with vehicle 402 via LiDAR communication 406. Stand 408 also includes wired or wireless backhaul (e.g., cellular modem or fiber optic connection). Vehicle 402, which cannot communicate directly with a dispatcher, can communicate with stand 408 via LiDAR communication, and stand 408 can communicate with the dispatcher via backhaul. While vehicle 402 is described as communicating with stand 408 via LiDAR communication, additional or alternative communication systems may be used in some other embodiments. Other systems may include wireless local area network systems (such as systems based on IEEE 802.11), Bluetooth® systems, cellular systems (such as 3G, 4G, 5G, etc.), visible light communication systems, near field communication (NFC) systems, etc.

[0076] When communicating with station 408, vehicle 402 can provide vehicle locator packets to station 408. In some implementations, sending vehicle locator packets can indicate to station 408 and the dispatcher that vehicle 402 cannot communicate directly with the dispatcher (e.g., via a cellular modem). In some examples, the dispatcher can remove vehicle 402 from service, dispatch maintenance personnel to vehicle 402, send the location of vehicle 402 for maintenance, or communicate new fares to be handled by vehicle 402. In some implementations, station 408 can also communicate with vehicle 402 while passing through the road. If vehicle 402 cannot communicate directly with the dispatcher, station 408 can update the dispatcher with the location of vehicle 402.

[0077] Call a car

[0078] Besides locating vehicles, LiDAR communication can also be used to hail vehicles. (Return to reference) Figure 4 Station 408 can be a pedestrian taxi stand. A person can walk to station 408 and enter their desired destination (e.g., via a smartphone app, the graphical user interface of station 408, or another suitable interface with station 408). Multiple autonomous vehicles can queue at a location away from station 408 (e.g., in a parking lot at a corner) and communicate with another LIDAR transmitter (e.g., one or more stations in the parking lot). Parking space 410 can be the location where the next vehicle in the queue is hailed, and parking space 410 can also be the location where a user enters a vehicle and begins the requested ride.

[0079] Station 408 can handle multiple ride requests simultaneously. As a result, station 408 can call multiple vehicles (e.g., using one or more LiDAR transmitters in a queued vehicle storage area). Each called vehicle can be assigned a specific customer or ride, and as the vehicle approaches parking space 410, it can indicate its assigned ride to station 408 (via LiDAR communication 406). In this way, station 408 can notify customers of their assigned rides in parking space 410, expediting pick-up and coordinating rides from multiple customers in the same parking space 410.

[0080] notify

[0081] LiDAR communication can be used to provide notifications to vehicles or drivers. For example, vehicles can be notified via LiDAR of an approaching emergency vehicle, road hazards, school crossings during school hours, changes in speed limits, construction zones, etc. For emergency vehicles, emergency broadcasts can be transmitted by infrastructure or vehicles that have previously received broadcasts to notify other vehicles of the presence of an emergency vehicle via LiDAR.

[0082] Figure 5 An example environment 500 is shown for transmitting the presence of ambulance 502 to vehicle 506. Vehicle 508 can detect ambulance 502. For example, ambulance 502 can broadcast an emergency signal indicating its presence. In another example, a LiDAR device or other sensor of vehicle 508 can sense ambulance 502, and vehicle 508 can identify ambulance 502 as an emergency vehicle.

[0083] In some examples besides ambulance 502, vehicle 508 may detect people in the environment who would necessitate changes in the navigation of other vehicles, or people who could be notified of such changes. For example, vehicle 508 may detect one or more cyclists in a lane, children playing near the street, children crossing the street, or other pedestrians (e.g., before or after school). In other examples, vehicle 508 may detect a vehicle stalled, a minor traffic accident (“minor crash”), traffic congestion, or other traffic conditions that might require other vehicles to change their navigation or be notified of such an incident. Such scenarios would necessitate that vehicles reduce their speed through the area, adjust their scanning environment using one or more LiDAR devices to focus on a specific area, change lanes, etc., for example, to provide more space for detected people, situations, etc.

[0084] Vehicle 508 may be outside the LiDAR communication range of vehicle 506. For example, LiDAR communication may be based on LoS, and vehicle 506 may be obstructed by buildings or otherwise blocked to prevent direct communication between vehicle 508 and vehicle 506. In some implementations, vehicle 508 may use one or more vehicles and / or infrastructure to perform LiDAR communication with vehicle 506 (e.g., "around a corner" communication). For example, vehicle 508 may use LiDAR communication 510 with vehicle 504 to indicate the presence of ambulance 502. Additionally or alternatively, vehicle 508 may communicate with station 514 via LiDAR communication 516 to indicate the presence of ambulance 502.

[0085] Although Figure 5 Direct Loss of Signal (LoS) communication between LIDAR devices (510, 512, 516, and 518) is illustrated, but LoS communication between LIDAR devices may include one or more signal reflections from surfaces. For example, a focused beam of light from a transmitting LIDAR device may be intentionally reflected from one or more building walls, signs, or other objects in environment 500, and the reflection may be received by a receiving LIDAR device. If the beam is sufficiently focused, the transmission includes sufficient transmission power, and one or more surfaces have sufficient reflectivity, the reflection received by the receiving LIDAR device can be processed to determine the communication transmitted by the transmitting LIDAR device (e.g., notification of the presence of ambulance 502). In this way, LIDAR devices can communicate with each other without direct LoS. In this specification, LoS may include direct LoS or indirect LoS (which may include one or more reflections).

[0086] Vehicle 504 or station 514 can be configured to indicate the presence of ambulance 502 or other traffic conditions to other vehicles and infrastructure within LiDAR communication range. For example, vehicle 504 can use LiDAR communication 512 to indicate the presence of ambulance 502. In another example, station 514 can use LiDAR communication 518 to indicate the presence of ambulance 502. In this example, indicating the presence of ambulance 502 may include two hops from vehicle 508 to vehicle 506 (e.g., via LiDAR communications 510 and 512 or via LiDAR communications 516 and 518).

[0087] In some implementations, the indication (or other notification) of an emergency vehicle can be propagated up to a defined number of hops or a defined distance. In this way, notification of the presence of an emergency vehicle to other vehicles is localized to the area surrounding the emergency vehicle or traffic situation. In some examples, vehicle 508 may also determine the trajectory, route, or other information of ambulance 502 and indicate such information to vehicle 504 or station 514. The distance or number of hops propagating this indication may, for example, increase in the direction of the route (or conversely, decrease in the direction opposite to the route). If vehicle 506 is an autonomous vehicle, vehicle 506 may proactively pull over, stop, change lanes, or change its route in response to receiving an indication of the presence of ambulance 502 or other traffic situation. In some other implementations, vehicle 506 may provide visual or auditory notification to the driver (and / or passengers) to indicate the presence of ambulance 502 or traffic situation. For example, the vehicle's speakers or displays may notify the driver and / or passengers of the presence of ambulance 502 or traffic situation.

[0088] Another example notification could be of available parking spaces in a parking lot or garage. In some implementations, the garage entrance gate can communicate available spaces to entering vehicles via LiDAR communication. If the vehicle is autonomous, a parking space can be assigned to it, and the vehicle automatically moves forward and parks in that space. In this way, the parking garage or parking lot can efficiently organize the parking of vehicles entering and exiting. In some other implementations, the entrance gate can indicate available parking spaces to the vehicle, and the vehicle can notify the driver and / or passengers of the location of available spaces. For example, the vehicle can display a map of the parking garage and indicate the location of available spaces on the displayed map. The driver and / or passengers can be notified before the vehicle enters the parking garage (e.g., when the vehicle approaches the garage), and the driver and / or passengers can choose which spaces are preferred.

[0089] As described above, another example notification could be a traffic accident or other traffic obstruction. The notification can be communicated to the vehicle ahead of the obstruction via LiDAR communication, and the vehicle can determine an alternative route and / or notify the driver and / or passengers of potential delays based on the received notification. Other suitable example notifications can also be sent via LiDAR communication, and this disclosure is not limited to the examples described above.

[0090] Limited access area

[0091] Another use case for LIDAR communication is restricting access to designated areas. Certain areas may be restricted to specific vehicles or vehicle types. For example, hospital ambulance entrances may be restricted to ambulances. In another example, disabled parking spaces may be restricted to vehicles with disability tags. In yet another example, HOV lanes may be restricted to vehicles carrying, for example, three or more passengers. In yet another example, a section of a city center may be restricted to autonomous vehicles, low-emission or zero-emission vehicles, taxis, or other specific types of vehicles. In yet another example, a company parking garage may be restricted to employee vehicles or vehicles with security clearance.

[0092] LiDAR communication can be used to indicate restrictions on specific areas and authorize access to those areas for specific vehicles. In some implementations, the LiDAR device for the area (e.g., an entrance gate, station, or other infrastructure) can send the area's restrictions to an approaching vehicle. The vehicle can then (via its own LiDAR device) communicate the necessary authentication to the LiDAR device, indicating that the vehicle has been authorized to enter the area. The vehicle can then be allowed to enter and navigate the area (e.g., open a door or be authorized to access the area). In some other implementations, the vehicle can store the credentials required for the area. In this way, the vehicle can preemptively communicate its credentials to the LiDAR device to gain access rights. For example, when an ambulance approaches a hospital's ambulance entrance, the ambulance can use the LiDAR device to identify itself as an ambulance to enter.

[0093] Access restrictions on areas can change over time. For example, HOV lanes may be restricted only during designated peak hours. In another example, city centers may be restricted to zero-emission vehicles when smoke levels exceed a threshold. In yet another example, areas around accidents or requiring police or emergency responders may be restricted to such police or emergency responder vehicles. Because LIDAR communications do not require a centralized network, changes to access restrictions can be formulated and provided to vehicles much faster than, for example, by remotely coordinating and attempting to establish access parameters for an area via other communications. In some examples, vehicles can notify drivers and / or passengers that they are about to enter a restricted area. In this way, drivers, passengers, or the vehicle itself can adjust their routes to navigate around the restricted area.

[0094] In another implementation, the area may include different tolls or fees based on time of day, day of the week, congestion level in the area, special events, etc. LIDAR communication can be used to communicate such tolls or fees to the vehicle before it enters the area. The driver and / or passenger can then determine (or the vehicle can determine automatically) whether to enter the area based on this information. For example, some roads include hot lanes, where the price to enter the lane is based on the time of day and road congestion. The driver and / or passenger can be informed of the price to enter the hot lane and thus decide whether to enter. Alternatively, the vehicle can be configured to enter the hot lane based on, for example, whether the price is less than a threshold amount or a priority for riding. For example, if a couple is going to the hospital, a priority for riding can be set to override any price constraints associated with the hot lane. Other suitable use cases for limiting the lanes may exist, and this disclosure is not limited to the examples provided.

[0095] Object visualization or rendering

[0096] Another use case for LIDAR communication is providing rendering or visualization information. (For example, regarding...) Figure 3 As described, the grouping may include "rendering_description" or "rendering_identification" information for rendering or visualizing vehicles (or infrastructure, such as...) Figure 4 (Station 408 in the text). Rendering or visualizing objects in a vehicle environment may be limited to the LoS of the LiDAR device coupled to the vehicle. Therefore, surfaces not in the LoS of the LiDAR device may not be detected and rendered.

[0097] Figure 6AAn example environment 600 of vehicle 602 is illustrated. Environment 600 includes buildings 604 and 606 and vehicles 608 and 610. As shown, the view of vehicle 608 from the perspective of the LIDAR device 612 of vehicle 602 includes the front surface and driver's side surface of vehicle 608. The view of vehicle 610 from the perspective of the LIDAR device 612 is obscured by building 606. As a result, the view of vehicle 610 includes the front surface and only a portion of the driver's side surface. Furthermore, the views of buildings 604 and 606 from the perspective of the LIDAR device 612 are surfaces facing vehicle 602. The LIDAR device 612 can collect information about the depth of surfaces within the view of the LIDAR device 612, and vehicle 602 can visualize or render environment 600 including detected surfaces. As used herein, vehicle visualization or rendering of the environment may include one or more of the following: the vehicle or infrastructure provides instructions to a display coupled to the vehicle or infrastructure (e.g., a tablet computer communicatively coupled to the vehicle's processing system, an in-dashboard entertainment system coupled to the vehicle's processing system, a display remote from the vehicle, etc.) to render at least a portion of the environment on the display; or the vehicle renders at least a portion of the environment on an integrated display (e.g., via an integrated entertainment system in the vehicle's central console). As used herein, a particular vehicle or infrastructure performing one or more operations (such as vehicle visualization or rendering of the environment) may refer to: a device or system coupled to the vehicle or infrastructure performing operations (such as generating a rendering to be displayed and providing the rendering to a display coupled to the vehicle or infrastructure).

[0098] Figure 6B The illustration shows an example rendering 650 of an environment 600 sensed by a LIDAR device 612. As described above, the example rendering 650 can be displayed on an integrated display of the vehicle or on a display coupled to a processing system of the vehicle. Rendering 650 includes renderings of surfaces 654 corresponding to building 604, surface 656 corresponding to building 606, surface 658 corresponding to vehicle 608, and surface 660 corresponding to vehicle 610. The example rendering 650 also includes a representation 652 of vehicle 602 to provide a viewpoint. For example, the representation can be an icon, a stock image, a block, or another saved representation of the vehicle to be used for rendering.

[0099] The example render 650 is simplified to illustrate aspects of this disclosure, and render 650 may include additional information or details. For example, lines or markings on a road may be reflective and thus can be sensed by a LiDAR. In another example, additional features of buildings 604 and 606 and vehicles 608 and 610 within the LoS of LiDAR device 612 may be sensed by LiDAR. Although example render 650 is shown as two-dimensional from a bird's-eye view, the render can be from any suitable orientation or from any suitable perspective. For example, the render may be a three-dimensional point cloud from the perspective of LiDAR device 612, a two-dimensional view of the environment from the perspective of LiDAR device 612, or any other suitable render.

[0100] Example rendering 650 may be provided to the driver and / or passenger by vehicle 602 via a vehicle display. For example, example rendering 650 may be displayed on an integrated display of the vehicle, or the vehicle may provide rendering instructions to a display coupled to the vehicle's processing system. As shown in example rendering 650, depth information may not be available for hidden surfaces of objects and buildings. For example, rendering 650 does not include information on the passenger side and rear surfaces of vehicle 608, does not include information on the passenger side, rear, and a portion of the driver side surfaces of vehicle 610, and does not include information on the sides of buildings 604 and 606 that do not point towards the vehicle 602.

[0101] In some implementations, another LiDAR device can send rendering information about one or more objects in environment 600 to vehicle 602. For example, LiDAR device 614 of vehicle 608 can send information about vehicle 608 for rendering, and LiDAR device 616 of vehicle 610 can send information about vehicle 610 for rendering. In some implementations, dimensions for rendering the shape of the vehicle can be sent. For example, dimensions for rendering one or more rectangles of vehicle 608 in rendering 650 can be sent from LiDAR device 614 to LiDAR device 612. Dimensions for rendering one or more rectangles of vehicle 610 in rendering 650 can be sent from LiDAR device 616 to LiDAR device 612. Other features about rendering vehicles 608 and 610 can also be sent. Example features include textures, such as colors, shadows, highlighting, etc., used for vehicle rendering.

[0102] In some other implementations, LIDAR devices 614 and 616 can send identification information for the corresponding vehicle. For example, LIDAR device 614 can send information about the brand and model of vehicle 608, the vehicle identification number of vehicle 608, the license plate number of vehicle 608, whether the vehicle is part of a specific vehicle fleet (e.g., the same vehicle fleet as vehicle 602), a unique identifier within the vehicle fleet, or other identification information.

[0103] Vehicle 602 (or a memory coupled to a display to show the rendering) may store rendering information for a specific vehicle, and vehicle 602 may use vehicle identification to locate the stored rendering information for a specific vehicle. For example, vehicle memory (or other memory) may store rendering information for vehicles of multiple brands and models (including vehicles 608 and 610), and such rendering information may be retrieved and used to enhance the rendering 650 of environment 600. If rendering 650 is to be displayed on a display coupled to vehicle 602, vehicle 602 may provide the rendering information retrieved from vehicle memory, or vehicle 602 may provide information for identifying rendering information in external memory (e.g., providing information about the brand and model to an entertainment system coupled to the vehicle). Example rendering information may include stock images or drawings of a specific vehicle. In some examples, rendering information may be used to replace or combine with corresponding portions of rendering 650. In some other examples, adjusting the rendering may include highlighting object renderings. For example, the rendering may be updated to notify the driver or passengers of an emergency vehicle or situation in the vehicle environment.

[0104] Figure 6C An example environment 670 is shown for vehicle 672. Example environment 670 includes buildings 684 and 686 and vehicles 674 and 676. Environment 670 also includes an emergency vehicle 678. Figure 6D It shows Figure 6C Example rendering 690 of the environment 670 of vehicle 672 is shown from top to bottom. Rendering 690 includes renderings of surface 694 corresponding to building 684, surface 696 corresponding to building 686, and surface 698 corresponding to vehicle 674. Example rendering 690 also includes a depiction 692 of vehicle 672 to provide a viewpoint. Vehicle 676 and emergency vehicle 678 may not be within the LoS of the LIDAR device coupled to vehicle 672. As a result, rendering 690 may not include renderings of vehicle 676 or emergency vehicle 678.

[0105] Figure 7 A flowchart depicting an example operation 700 of rendering for adjusting an environment such as a vehicle, LiDAR device, infrastructure, etc., is shown. (See below for reference.) Figure 6A and 6CVehicles 602 and 672 in the text are for illustrative purposes only in describing example operation 700. Those skilled in the art will recognize that example operation 700 can be performed by any suitable device (e.g., a control device coupled to the vehicle), infrastructure, and / or vehicle according to various implementations, and that example operation 700 described herein can be performed with additional steps, fewer steps, steps in a different order, steps in parallel, or any combination thereof. In this document, vehicle 602 performing one or more steps can mean that a processing system coupled to the vehicle performs one or more steps, a processing system integrated into vehicle 602 performs one or more steps, or other suitable embodiments for performing the described methods. For example, a control device separate from the vehicle can be mounted or otherwise coupled to the vehicle. Such a control device can also be coupled to one or more LiDAR devices, to a display (for rendering or displaying other information), and / or to one or more other input / output components (e.g., speakers, keyboards, etc.). Vehicle 602 performing one or more steps in some implementations can refer to a control device performing one or more steps.

[0106] Starting at 702, the carrier 602 can receive data associated with a modulated optical signal transmitted by the transmitter of the first LIDAR device. For example, the device can be coupled to LIDAR device 612, and the modulated optical signal can be transmitted from LIDAR device 614 or LIDAR device 616 to LIDAR device 612. The modulated optical signal includes an optical carrier signal modulated to include a data signal. LIDAR device 612 or a device coupled to LIDAR device 612 can extract the data signal and determine the data included in the data signal.

[0107] Vehicle 602 may also generate a rendering (704) of the vehicle environment based on information from one or more LiDAR devices coupled to the vehicle. For example, vehicle 602 may include a processing system to receive measurements from LiDAR device 612 when LiDAR device 612 is in detection mode. These measurements may be used to generate a rendering, such as a point cloud, depth map, or other suitable representation of environment 600. For example, the processing system may generate an example rendering 650 of environment 600 (or an example rendering 690 of environment 670) based on measurements received from LiDAR device 612 or other LiDAR devices coupled to the vehicle, and the rendering may include a representation of surfaces detected in environment 600 (or environment 670).

[0108] Based on the received data, vehicle 602 can update the rendering (706). For example, vehicle 602 can update the rendering of objects in the environment (708). In some example implementations, vehicle 602 can receive object identifiers in the received data. For example, LIDAR device 612 in communication mode can receive a packet containing the identifier of vehicle 608 from LIDAR device 614 via a modulated light signal. The identifier can be a specific vehicle ID, vehicle identification number, vehicle brand and model, or other suitable identifier in the packet.

[0109] Vehicle 602 can determine the rendering information of an object based on an identifier. Example rendering information may include the dimensions of one or more shapes in the rendered object (e.g., one or more rectangles or other shapes in the rendered vehicle), a representative image of the object (e.g., a representative drawing of the vehicle based on, for example, the vehicle's type, brand, and model), a stock image of the object (e.g., a stock image of the vehicle), or a texture to be applied to the rendering of the object (e.g., a texture to be applied to the rendering of the vehicle). Rendering information may also indicate whether the object rendering should be highlighted or adjusted (e.g., changes in size, dimensions, stretching, etc.). In some implementations, if vehicle 602 receives an identifier (e.g., brand and model) of vehicle 608, vehicle 602 can scan a database indexed by identifiers (e.g., brand and model) associated with rendering information (e.g., a reference image of the vehicle, stored dimensions of the vehicle, or, in the case of an emergency vehicle, highlighted information), to determine the rendering information of vehicle 608 (e.g., a stored image of vehicle 608's brand and model). Vehicle 602 can then update the object rendering based on the determined rendering information. For example, surface 658 rendered in render 650 can be combined with stored images or other rendering information of vehicle 608 to update render 650. In another example, the rendered surface can be highlighted. In another example of updating object rendering, object rendering can be added to the rendering of the environment. For example, the rendering of vehicle 676 and / or the rendering of emergency vehicle 678 can be added to render 690.

[0110] In some other implementations, the LiDAR device can be configured to send data signals that include a specific description of how objects should be rendered. For example, the LiDAR device 614 of vehicle 608 can send the rendering model, textures, and any other features used for rendering the vehicle. In this way, memory including lookup tables, databases, etc., can be avoided when determining the rendering information of objects during the rendering of the updated environment.

[0111] Return to reference Figure 7After the updated rendering, vehicle 602 can provide an updated rendering for display (710). For example, a control device coupled to vehicle 602 can provide an updated rendering to a display coupled to vehicle 602, and the display can show the updated rendering to the driver and / or passengers. In another example, the rendering can be provided to a remote display so that others can be aware of the environment of vehicle 602.

[0112] Figure 8 An example rendering 800 of environment 600 in Figure 6 is shown. Rendering 800 can be based on rendering information of vehicles 608 and 610. Figure 6B The following is an example of adjusting the rendering of rendering 650. For example, vehicle 602 can store representative images (e.g., stock images, line drawings, etc.) or other suitable rendering information for the received identifiers of vehicles 608 and 610. LiDAR devices 614 and 616 can send the vehicle identifiers to LiDAR device 612, and vehicle 602 can use the received vehicle identifiers to determine the rendering information of vehicles 608 and 610 to adjust rendering 650.

[0113] The rendering surface 658 associated with vehicle 608 can be combined with specific rendering information of vehicle 608 (e.g., a stock image or stored drawing of vehicle 608). For example, an image of vehicle 608 can be aligned with the rendered surface 658 (e.g., by resizing and / or orienting the image of vehicle 608 to align with the rendered surface 658). In example rendering 800, object 808 corresponds to an adjusted rendering of vehicle 608. Similarly, object 810 in rendering 800 corresponds to an adjusted rendering of vehicle 610.

[0114] Although not shown, other objects that can be represented in rendering 650 may include infrastructure, buildings, road markings, pedestrians, or other non-vehicles whose rendering can be adjusted. In some examples, a LiDAR device coupled to a vehicle may send information about objects in the environment other than the vehicle. For example, one or more LiDAR devices of vehicle 608 (e.g., LiDAR device 614) may be used to determine the depth of objects in the environment of vehicle 608, and one or more LiDAR devices of vehicle 610 (e.g., LiDAR device 616) may be used to determine the depth of objects in the environment of vehicle 610. In one example, each of vehicles 602, 608, and 610 may generate a rendering based on measurements provided by the LiDAR devices of the respective vehicle. In some implementations, vehicles 602, 608, and 610 may share LiDAR measurements with each other (via LiDAR communication), and vehicle 602 may use combined LiDAR measurements to generate a more comprehensive rendering of environment 600 than rendering 650.

[0115] In some implementations, any suitable updated render can be displayed on the monitor of vehicle 602. For example, an updated render can be generated for an environmental area that is not yet within the range or LoS of the LiDAR device of vehicle 602. Vehicle 610 can detect objects near the street behind vehicle 610 (e.g., using LiDAR device 616), which can be used to render environment 600 that is not within the field of view of vehicle 612. Vehicle 610 can send rendering information of at least a portion of this environment (e.g., rendering information of pedestrians, cyclists, parked cars, traffic cones, obstacles, blockages, etc.) to vehicle 602. In this way, vehicle 602 can use the rendering information to include additional information about environment 600 to the right of render 650 (which is obscured by building 606 from the field of view of LiDAR device 612 of vehicle 602). In some aspects, rendering information can be shared among multiple vehicles and infrastructure, and the rendering of a particular vehicle (e.g., vehicle 602) can include areas far beyond the vehicle. For example, rendering information about the environment of a vehicle that is more than hops away from vehicle 610 can be sent to vehicle 610 via these hops, and then to vehicle 602. Vehicle 602 can then use the rendering information to expand the area covered by the rendering generated by vehicle 602.

[0116] Besides updating the rendering, the received data can also be used for autonomous vehicle navigation or other automated operations. For example, an updated rendering can indicate additional obstacles or obstructions, and the vehicle can therefore navigate (e.g., update its navigation) to avoid these additional obstructions during operation. (See separate references.) Figure 6B and Figure 8 Renderings 650 and 800 are used in the above examples. Rendering 650 does not show the rear of vehicle 610, but the updated rendering 800 shows the entire vehicle 610. If the route of vehicle 602 is determined to collide with the end of vehicle 610 as determined based on received data, the navigation of vehicle 602 can be updated to avoid vehicle 610. In another example, the driver and / or passengers can be notified of potential collisions or obstacles based on the updated rendering. For example, the updated rendering, including obstacles or blockages, can be shown to the driver and / or passengers, or the driver and / or passengers can be notified of obstacles or blockages. In this way, the occupants of the vehicle can be informed of route changes, vehicle deceleration, or other changes in autonomous vehicle navigation before obstacles or blockages appear. Alternatively, or as an alternative, the dispatcher or central office of the fleet including the vehicles can be notified of obstacles or blockages to explain the reason for adjusting vehicle navigation.

[0117] In addition to updating render 650 to include models or other representative images of vehicles (or another object in the environment), render 650 can be updated to include the textures of the models. For example, the renders of vehicles within the same convoy can be adjusted to have similar textures to indicate that they belong to the same convoy.

[0118] Figure 9A It shows Figure 6A Example rendering 900 of environment 600 shows vehicles 608 and 602 belonging to the same convoy. As shown, objects 902 and 908 in rendering 900 can be textured similarly (e.g., shaded). Object 910, corresponding to vehicle 610 which is not part of the same convoy, can be textured differently from objects 902 and 908. For example, object 910 can include different shadows than objects 902 and 908. Objects such as infrastructure, buildings, roads, and vehicles can be textured differently depending on their type. For example, the same texture can be used to render a station, and the same texture can be used to render an emergency vehicle, etc. In this way, drivers and / or passengers can easily identify similar objects in the environment based on the textures in the displayed rendering. In some implementations, updating the rendering can include highlighting object renderings. For example, the rendering of an emergency vehicle can be highlighted to notify the vehicle's occupants of its presence. Similarly, construction zones, safety zones, or other areas of the environment can be highlighted to notify vehicle occupants of their presence in the environment.

[0119] Figure 9B Environment 670 is shown. Figure 6C Example updated render 950. Updated render 950 includes render 952 of emergency vehicle 678. Render 952 can be highlighted to indicate that vehicle 678 is an emergency vehicle. Render 952 may also include representative image 954 included for vehicle 674 and representative image 956 included for vehicle 676, which were not originally included in render 670.

[0120] Return to reference Figure 6C In the environment 670, the LIDAR equipment coupled to vehicle 672 can be blocked by a Loss blocking vehicle 676 and an emergency vehicle 678. As a result, Figure 6DThe rendering 690 (based on measurements from the LIDAR device) does not include rendering of vehicle 676 and emergency vehicle 678. In some implementations, the LIDAR device coupled to vehicle 672 may receive reflections of modulated light signals from the LIDAR device coupled to vehicle 676 or emergency vehicle 678. The modulated light signals may include data indicating the presence of emergency vehicle 678. This data may also indicate other information about the emergency vehicle, such as the type of emergency vehicle, whether emergency vehicle 678 is responding to an emergency, and / or the emergency vehicle's travel path, speed, etc. This data may also include information about other vehicles or objects. For example, the data may indicate the presence of vehicle 676.

[0121] In some other implementations, vehicle 672 may receive data about emergency vehicle 678 from an intermediate device. Vehicle 676 may sense the presence of emergency vehicle 678 (e.g., via a LIDAR device coupled to vehicle 676). Vehicle 676 may transmit data about emergency vehicle 678 to vehicle 674 within the LoS via modulated optical signal 680. Vehicle 674 may then transmit data about emergency vehicle 678 to vehicle 672 within the LoS via modulated optical signal 682. Although an intermediate node for transmitting data is shown, any number of hops can be performed via data. When vehicle 672 receives data about emergency vehicle 678 (and data about vehicle 676), vehicle 672 may update rendering 690 to fill in occlusions caused by building 684, and highlight or otherwise indicate the presence of emergency vehicle 678. In addition to highlighting the emergency vehicle, or as an alternative, vehicle 672 may notify the driver and / or passengers by, for example, flashing the display, providing an audible notification, applying special textures to the rendering, and / or providing a text notification on the display. Although the description of emergency vehicles (such as ambulances) is similar. Figure 6C , 6D Similar to the example of 9B, but the operation can also be applied to any vehicle, object, or area of ​​interest (such as construction zones, school zones, accident zones, etc.).

[0122] In some implementations, rendering information for objects in the environment can be provided via a Software as a Service (SaaS) model. For example, a unique identifier for an object can be received by vehicle 602. Vehicle 602 can then communicate with a service (e.g., via a cellular modem or other communication device) to obtain the object's rendering information. For example, vehicle 602 can receive the vehicle identification number of vehicle 608, and vehicle 602 can communicate with a remote server that stores rendering information for multiple objects (including vehicle 608). The remote server can respond to a request by providing the rendering information for vehicle 608, and the rendering information can be used by vehicle 602 to update the rendering.

[0123] In some other implementations, vehicle 602 can store rendering information for objects, and a remote server can provide rendering information for objects that vehicle 602 does not have locally stored. Additionally or alternatively, the remote server can be used to update locally stored rendering information. For example, when a new brand and model of vehicle is released, rendering information for the vehicle can be created but not yet stored in vehicle 602's memory. LiDAR communication can be used to update the locally stored rendering information of vehicle 602 to include the new rendering information. In some implementations, vehicle 602 can be coupled to a station via a dedicated backhaul (e.g., Figure 4 When the site (408) updates its stored rendering information.

[0124] Other use cases exist for LIDAR communication, and this disclosure is not limited to the examples above. For example, devices (such as vehicles) may include software or firmware that periodically requires updates. LIDAR devices can be used to receive software updates from other devices. For example, a software update, including rendering information of a new brand and model of vehicle, can be provided to a first subset of vehicles. These vehicles can then send the update to other vehicles in the environment via LIDAR communication. In this way, updates can be distributed quickly without all vehicles needing to be connected to a central location (such as a server storing updates). For example, communication between devices (such as vehicles and infrastructure) can be similar to a peer-to-peer network, and updates can be propagated across devices without requiring a central repository to download them. Another example use case may include audio and video transmission between devices (such as for voice calls, messaging, or video conferencing).

[0125] Generally, one or more operations may be based on data received via modulated optical signals used for LIDAR communication. As described herein, LIDAR communication can be used to provide data that, for example, influences the navigation of autonomous vehicles, provides notifications to drivers or passengers, adjusts access to locations, etc.

[0126] Figure 10 A flowchart depicting example operation 1000 is shown, which is used to perform one or more operations based on data received from transmissions from a LIDAR transmitter. Although the execution of one or more operations by the device is discussed for the purpose of explaining aspects of this disclosure... Figure 10 Operation 1000 can be performed by vehicles, infrastructure, dispatchers or central offices coordinating vehicle fleets, processing systems coupled to vehicles (such as entertainment systems), appropriate control systems (such as control systems coupled to vehicles for Level 4 autonomous driving), etc. Example Operation 1000 is not limited to being performed by a specific device or performing operations on a specific device.

[0127] Starting at 1002, the device can receive data associated with a modulated optical signal emitted by a transmitting LIDAR device. For example, the device (e.g., a vehicle, processing system, etc.) can be coupled to a LIDAR device configured to receive modulated optical signals from a separate transmitting LIDAR device. The receiver of the receiving LIDAR device can receive the modulated optical signal, which may include an optical carrier signal modulated to carry a data signal including data. The carrier signal may be a signal typically used for detection and ranging. The receiving LIDAR device (or a device coupled to the receiving LIDAR device) can demodulate the modulated optical signal to generate and provide the data provided in the modulated optical signal. In this way, the device (e.g., a processing system or vehicle) can receive data associated with the modulated optical signal. In some implementations, the data may include one or more packets. Return to Reference Figure 3 Example groups may include information specific to things such as locating vehicles, using vehicles to access specific areas or locations, object rendering, etc.

[0128] Return to reference Figure 10 The device can determine one or more operations to be performed based on the received data (1004). For example, the device can determine one or more operations based on one or more received packets. In some implementations, the device can determine one or more vehicle locator operations (1006). In one example, if the vehicle receives information in the data that the vehicle will be located, the vehicle can determine to return to a designated location (e.g., including parking spaces at stations, such as...). Figure 4 (As shown) and communicates with the station (via LiDAR communication) the vehicle's location and current status. In another example, if the device receives information about a lost vehicle and the device recently communicated with the lost vehicle, the device can communicate the last received information about the lost vehicle's location and other information to the vehicle, infrastructure, dispatchers, etc. Alternatively, the device can propagate the lost vehicle message to other vehicles and devices via LiDAR communication.

[0129] In some other implementations, the device can determine one or more vehicle hailing operations (1008). Return to Reference Figure 4If station 408 is a pedestrian station for an autonomous taxi fleet, station 408 can communicate with vehicles via LiDAR communication to assign vehicles to specific customers and for rides. Vehicles can receive data from station 408 via LiDAR communication between a LiDAR device coupled to the vehicle and a LiDAR device coupled to the station and located in a separate area where vehicles are queuing. Based on the received data, the vehicle can approach the space near station 408 and communicate with station 408 about the rides or customers assigned to the vehicle. Station 408 can notify the customer that the vehicle has arrived, allowing the customer to board. In another example, the vehicle or device can determine to notify the driver (e.g., a taxi driver) of the customers and / or rides assigned to the vehicle. The taxi driver can approach the customer based on the notification to initiate the ride.

[0130] Return to reference Figure 10 In some further implementations, the device may determine one or more notification operations (1010). In some examples, a reference is returned. Figure 5 Vehicle 506 can receive data on the presence of ambulance 502 or other emergency vehicles in the vicinity. Vehicle 506 may pull over to the side of the road, stop, or otherwise adjust its operation or navigation to allow ambulance 502 to pass safely. Alternatively or concurrently, vehicle 506 or another device may determine to notify the driver and / or passengers of the presence of ambulance 502. In some other examples, vehicle 506 or the device may determine to indicate the presence of ambulance 502 to other vehicles, infrastructure, or other equipment via LiDAR communication (thereby propagating that indication to other vehicles and infrastructure within the area of ​​ambulance 502).

[0131] In another example of a notification operation, the vehicle may receive data about available parking spaces in a parking garage. Therefore, the vehicle may drive to one of the available spaces and park there. The vehicle or device may determine and notify the driver and / or passengers of available spaces (e.g., indicating available spaces on a displayed parking garage map), etc. Other example notification operations may include: updating the route based on incidents or traffic conditions along the current route, notifying other vehicles of incidents or traffic conditions via LiDAR communication, notifying the driver and / or passengers of changes in route or traffic conditions, etc.

[0132] Return to reference Figure 10In some other implementations, the device may determine one or more restricted access area operations (1012). For example, if the city center is restricted to autonomous vehicles, zero-emission vehicles, or other specific types of vehicles, the vehicle may receive data associated with the access restrictions from a LIDAR device coupled to the vehicle. The LIDAR device may receive data in modulated optical signals transmitted by a LIDAR device coupled to infrastructure or other suitable equipment. For example, gates, toll booths, stations, etc., may include LIDAR devices to send an indication of the access restrictions to the vehicle. The vehicle may determine, based on this indication, whether it is permitted to enter the area. If the vehicle is permitted, it may also determine one or more navigation operations to access the area. For example, if access is restricted to autonomous vehicles, the autonomous vehicle may enter the city center. Alternatively, or as an alternative, the vehicle may determine to notify the driver and / or passengers of the approaching restricted access area and the access restrictions. In some examples, the driver may determine whether to access the area based on the notification. In some further examples, the vehicle may be notified of a toll or charge for accessing a specific area. For example, a bridge, lane, or area in the city may be associated with an access fee. Vehicles can provide account or other payment information via LIDAR communication before accessing infrastructure in a limited area (e.g., making payment to a toll booth before crossing a bridge). In some examples, toll notifications can be sent to the driver and / or passengers.

[0133] Since area restrictions may change based on current conditions (e.g., time of day, day of the week, current traffic congestion, smoke levels, situations requiring the area to be restricted to emergency vehicles, etc.), such changes in restrictions can be communicated. In response, a vehicle can, for example, determine and notify the driver and / or passengers of the change in access restrictions for the area, adjust its route to navigate around the new restricted area, or notify other vehicles of the change via LIDAR communication.

[0134] Return to reference Figure 10 In some further implementations, the device can determine one or more visualization or rendering operations (1014). For example, it can render the vehicle's environment, LIDAR equipment, infrastructure, or other suitable equipment (e.g., generating point clouds, environment mappings, depth maps, etc.), and the rendering can be displayed to the driver and / or passengers. Return to Reference Figure 6A and 6BVehicle 602 (or another suitable device coupled to vehicle 602, such as an entertainment system, processing system, etc.) can generate rendering 650 from data provided by LIDAR device 612. Vehicle 602 (or other suitable device) can also receive one or more identifiers of vehicles 608 and 610 in transmissions from the respective LIDAR devices 614 and 616. Vehicle 602 (or other suitable device) can therefore determine rendering information for vehicles 608 and 610 based on the received identifiers (e.g., looking up information in the memory of the vehicle or device, communicating with a remote service to provide rendering information for one or more objects, determining rendering information encoded in transmissions from the respective LIDAR devices, etc.). Rendering operations can therefore include updating rendering 650 using the determined rendering information. For example, Figure 6B The render 650 in the middle can be updated based on representative images (e.g., models) of vehicles 608 and 610 used for updating the render. Figure 8 Rendering 800 in the example. Other example rendering operations may include applying a defined texture to an emergency vehicle, applying a similar texture to objects of a similar type, applying an environment-specific texture to each vehicle or object and using a unique identifier (e.g., ...). Figure 3 The unique transmitter ID (302) is associated with a unique texture, etc. In some examples, vehicle 602 or other suitable devices may determine the updated rendering of the environment to be displayed to the driver and / or passengers. Rendering operations may also include contacting a remote service to update a stored database of rendering information, updating the database based on receiving new rendering information (such as a new brand and model of the vehicle), or propagating updates to the rendering information through LIDAR communication with other vehicles or devices in the environment.

[0135] Return to reference Figure 10 The device can then instruct the execution of one or more operations (1016). For example, if one or more rendering operations are determined, the device can instruct a display or another device coupled to the display to display the rendering and show an updated rendering. In another example, if the device is a vehicle and one or more operations are limited access area operations, the vehicle can instruct one or more vehicle components to navigate the vehicle into or out of the area. The device can also provide instructions to the driver and / or passengers for displaying information about the limited access area.

[0136] Although described from the perspective of the vehicle Figure 10The example operation 1000 describes some example procedures; however, any suitable device can perform one or more associated operations. For example, a station (or a processing system coupled to the station) may perform a vehicle locator operation (such as contacting a dispatcher) in response to receiving a LiDAR transmission from a lost vehicle. In another example, a station may perform a vehicle hailing operation (such as notifying a customer that a vehicle has arrived) in response to receiving a LiDAR transmission from a hailed vehicle. In another example, a station may perform a notification operation (such as sending a notification for an ambulance in the area) in response to receiving a notification in a LiDAR transmission. In yet another example, a toll booth or gate may perform a limited access area operation (such as changing the access fee or access restrictions for the area) in response to receiving a traffic congestion update from the LiDAR device of the infrastructure or vehicle in the area. As mentioned above, example operation 1000 is not limited to being performed by a specific device (such as a vehicle).

[0137] Furthermore, although example operations (such as hailing a ride, access restrictions, and updated rendering) are described individually, the device can be configured to perform any number of operations and combinations thereof. For example, the device can be configured to update rendering regarding emergency vehicles, notify vehicle occupants of the existence of an emergency vehicle, update the area restricted to the vehicle based on the existence of an emergency vehicle, and update the vehicle's navigation to avoid the restricted area.

[0138] Those skilled in the art will understand that information and signals can be represented using any of a variety of different techniques and skills. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be mentioned throughout the foregoing description can be represented by voltage, current, electromagnetic waves, magnetic fields or particles, light fields or particles, or any combination thereof. Furthermore, those skilled in the art will understand that examples have been provided for illustrative purposes in interpreting aspects of this disclosure. Aspects of this disclosure can be implemented by any suitable device (such as autonomous vehicles, autonomous vehicle fleets, infrastructure, control systems of vehicles or infrastructure, entertainment systems coupled to vehicles, or other devices), and are not limited to the specific examples herein.

[0139] Furthermore, those skilled in the art will understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the aspects disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the foregoing has generally described various components, blocks, modules, circuits, and steps in terms of their functionality. Whether this functionality is implemented in hardware or software depends on the specific application and design constraints imposed on the overall system. Skilled artisans can implement the functionality in various ways for each specific application; however, such implementation decisions should not be construed as a departure from the scope of this disclosure. For example, a processing system for a vehicle or other suitable device may include one or more processors and memory coupled to the one or more processors. The memory may include instructions executed by the one or more processors to cause the device to perform the operations described herein. The processing system may also be coupled to one or more LiDAR devices for LiDAR communication and surface detection and ranging. In some aspects, the processing system may include dedicated hardware, such as one or more integrated circuits, configured to perform one or more operations.

[0140] Therefore, the methods, sequences, or algorithms described in conjunction with the aspects disclosed herein can be implemented directly in hardware, as a software module executed by a processor, or a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. Example storage media are coupled to one or more processors (e.g., directly connected or remotely coupled) such that one or more processors can read information from and write information to the storage medium. Alternatively, the storage medium may be an integrated portion of one or more processors.

[0141] In the foregoing description, exemplary embodiments have been described with reference to specific example embodiments. However, it will be apparent that various modifications and variations can be made thereto without departing from the broader scope of this disclosure as set forth in the appended claims. For example, although the vehicle is generally shown as an automobile, any suitable vehicle, such as a motorcycle, drone, aircraft, boat, helicopter, etc., can be used. Furthermore, the vehicle may include one or more LiDAR devices, or the vehicle may be coupled to LiDAR devices manufactured separately from the vehicle. The vehicle may also include one or more processing systems, entertainment systems, control systems, etc., or the vehicle may be coupled to such processing systems, entertainment systems, control systems, etc., manufactured separately from the vehicle. For example, one or more LiDAR devices may be attached to different parts of the automobile to ensure coverage of the environment around the automobile. LiDAR devices may be coupled to the automobile's processing system (or coupled to the vehicle's control system) and provide data transmitted by other LiDAR devices for processing, or transmit data provided by the processing system to other LiDAR devices via modulated optical signals.

[0142] Therefore, the specification and drawings should be understood in an illustrative sense rather than a limiting sense.

Claims

1. An apparatus comprising: One or more processors; as well as A memory coupled to the one or more processors, the memory including instructions that, when executed by the one or more processors, cause the device to: Receive data associated with a modulated optical signal transmitted by a transmitter of a first optical detection and ranging LIDAR device and received by a receiver of a second LIDAR device coupled to a vehicle and the device, wherein the modulated optical signal includes an optical carrier signal modulated to include a data signal, the data signal including the data; A rendering of the vehicle's environment is generated based on information from one or more LiDAR devices coupled to the vehicle; The rendering is updated based on the received data, wherein updating the rendering includes updating the object rendering of objects in the environment of the vehicle; as well as Provide updated rendering for display on the monitor coupled to the vehicle. The update of the rendering further includes updating the rendering of the environment based on information from one or more LiDAR devices coupled to one or more other vehicles in the environment, and the updated rendering of the environment also includes rendering of environmental areas outside the range of one or more LiDAR devices of the vehicle.

2. The device according to claim 1, wherein, The data includes indications of objects in the environment, and updating object rendering includes at least one of the following: Highlight object rendering during display; Adjust the texture rendered on the object; The rendering of the environment includes representative images of the objects; and Adjust the size of the rendered object.

3. The device according to claim 2, wherein, Displaying updated rendering includes notifying at least one of the drivers or passengers in the vehicle that an object in the environment is an emergency vehicle.

4. The device according to claim 3, wherein, The execution of the instruction also causes the device to: Determine one or more navigation operations of the vehicle to be adjusted based on the emergency vehicle in the environment; as well as Provide one or more adjusted navigation operations to be performed by the vehicle.

5. The device according to claim 3, wherein, The data originates from a second device that is not coupled to the first LIDAR device.

6. The device according to claim 2, wherein, The execution of the instruction also causes the device to: Search a database indexed by vehicle identifier for an indication of the object, wherein the indication of the object includes a vehicle identifier; and The database identifies indications for updates to be performed on the object rendering associated with the vehicle identifier, wherein updating the object rendering is based on the identified indications for updates to be performed.

7. The device according to claim 6, wherein, The execution of the instruction also causes the device to: Receive second data associated with the second modulated optical signal received by the second LIDAR device; and Update one or more entries in the database based on the received second data.

8. The device according to claim 1, wherein: One or more LIDAR devices coupled to the vehicle include a second LIDAR device operating in detection mode; and The second LIDAR device operates in communication mode when receiving the modulated optical signal.

9. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of the device, cause the device to: Receive data associated with a modulated optical signal transmitted by a transmitter of a first optical detection and ranging LIDAR device and received by a receiver of a second LIDAR device coupled to a vehicle, wherein the modulated optical signal includes an optical carrier signal modulated to include a data signal, the data signal including the data; A rendering of the vehicle's environment is generated based on information from one or more LiDAR devices coupled to the vehicle; The rendering is updated based on the received data, wherein updating the rendering includes updating the object rendering of objects in the environment of the vehicle; as well as Provide updated rendering for display on the monitor coupled to the vehicle. The update of the rendering further includes updating the rendering of the environment based on information from one or more LiDAR devices coupled to one or more other vehicles in the environment, and the updated rendering of the environment also includes rendering of environmental areas outside the range of one or more LiDAR devices of the vehicle.

10. The computer-readable medium according to claim 9, wherein, The data includes indications of objects in the environment, and updating object rendering includes at least one of the following: Highlight object rendering during display; Adjust the texture rendered on the object; The rendering of the environment includes representative images of the objects; and Adjust the size of the rendered object.

11. The computer-readable medium of claim 10, wherein, Displaying updated rendering includes notifying at least one of the drivers or passengers in the vehicle that an object in the environment is an emergency vehicle.

12. The computer-readable medium of claim 11, wherein, The execution of the instruction also causes the device to: Determine one or more navigation operations of the vehicle to be adjusted based on the emergency vehicle in the environment; as well as Provide one or more adjusted navigation operations to be performed by the vehicle.

13. The computer-readable medium of claim 11, wherein, The data originates from a second device that is not coupled to the first LIDAR device.

14. The computer-readable medium of claim 10, wherein, The execution of the instruction also causes the device to: Search the database indexed by vehicle identifier for an indication of the object, wherein the indication of the object includes a vehicle identifier; and The database identifies indications for updates to be performed on the object rendering associated with the vehicle identifier, wherein updating the object rendering is based on the identified indications for updates to be performed.

15. The computer-readable medium according to claim 14, wherein, The execution of the instruction also causes the device to: Receive second data associated with the second modulated optical signal received by the second LIDAR device; and Update one or more entries in the database based on the received second data.

16. A method comprising: One or more processors of the device receive data associated with a modulated optical signal transmitted by a transmitter of a first optical detection and ranging LIDAR device and received by a receiver of a second LIDAR device coupled to a vehicle and the device, wherein the modulated optical signal includes an optical carrier signal modulated to include a data signal, the data signal including the data; The one or more processors generate a rendering of the vehicle's environment based on information from one or more LiDAR devices coupled to the vehicle; The rendering is updated based on the received data, wherein updating the rendering includes updating the object rendering of objects in the environment of the vehicle; as well as Provide updated rendering for display on the monitor coupled to the vehicle. The update of the rendering further includes updating the rendering of the environment based on information from one or more LiDAR devices coupled to one or more other vehicles in the environment, and the updated rendering of the environment also includes rendering of environmental areas outside the range of one or more LiDAR devices of the vehicle.

17. The method according to claim 16, wherein, The data includes indications of objects in the environment, and updating object rendering includes at least one of the following: Highlight object rendering during display; Adjust the texture rendered on the object; The rendering of the environment includes representative images of the objects; and Adjust the size of the rendered object.

18. The method according to claim 17, wherein, Displaying updated rendering includes notifying at least one of the drivers or passengers in the vehicle that an object in the environment is an emergency vehicle.

19. The method of claim 17, wherein, The data originates from a second device that is not coupled to the first LIDAR device.

20. The method of claim 17, further comprising: Search the database indexed by vehicle identifier for an indication of the object, wherein the indication of the object includes the vehicle identifier; The database identifies indications for updates to be performed on the object rendering associated with the vehicle identifier, wherein updating the object rendering is based on the identified indications for updates to be performed; Receive second data associated with a second modulated optical signal received by the second LIDAR device; as well as Update one or more entries in the database based on the received second data.

Images