Autonomous driving vehicle control system and method using depolarization ratio of return signal

Abstract

To enable effective autonomous driving decision making in response to an object.SOLUTION: The autonomous vehicle control system may be configured to cause the transmitter to transmit the transmit signal from the laser source, cause the receiver to receive the return signal reflected by the object, cause the one or more optics to generate a first polarized signal of the return signal having a first polarization, cause the one or more optics to generate a second polarized signal of the return signal having a second polarization orthogonal to the first polarization, and calculate a ratio between an average signal-to-noise ratio (SNR) value of the first polarized signal and an average SNR value of the second polarized signal, One or more processors configured to calculate a ratio of reflectivity between the first polarized signal and the second polarized signal. The one or more processors are configured to operate the vehicle based on a ratio of reflectance between the first polarized signal and the second polarized signal.SELECTED DRAWING: Figure 1a

Description

[Technical Field]

[0001] Cross-reference to related art This application claims the benefit of and priority to U.S. Patent Application No. 16 / 916,981, filed June 30, 2020, the entire disclosure of which is incorporated herein by reference. [Background technology]

[0002] One of the challenges in autonomous vehicle technology relates to gathering and interpreting information about the vehicle's surrounding environment, as well as planning and executing instructions to appropriately control the vehicle's motion to navigate the vehicle through that environment. For example, measurement data is acquired from one or more sensors on the autonomous vehicle (or a vehicle equipped with autonomous vehicle sensors) and used to track and / or determine dynamic objects within the vehicle's surrounding environment. It is difficult to sense and track the shape of objects with sufficient precision and accuracy to enable effective autonomous driving decision-making in response to the objects. Summary of the Invention [Means for solving the problem]

[0003] Embodiments of the present disclosure relate to a system and method for controlling a vehicle using Light Detection and Ranging (LIDAR), and more particularly to a system and method for controlling a vehicle by detecting an object using the depolarization ratio of a return signal reflected by the object.

[0004] In some embodiments of the present disclosure, a light detection and ranging (LIDAR) system includes a transmitter configured to transmit a transmission signal from a laser source, a receiver configured to receive a return signal reflected by an object, one or more optical systems, and a processor. The one or more optical systems are configured to generate a first polarization signal of the return signal having a first polarization and a second polarization signal of the return signal having a second polarization orthogonal to the first polarization. The processor is configured to calculate a ratio of reflectivities of the first polarization signal and the second polarization signal.

[0005] In some embodiments of the present disclosure, an autonomous vehicle control system includes one or more processors. The one or more processors are configured to cause a transmitter to transmit a transmission signal from a laser source. The one or more processors are configured to cause a receiver to receive a return signal reflected by an object. The one or more processors are configured to cause one or more optical systems to generate a first polarization signal of the return signal having a first polarization and a second polarization signal of the return signal having a second polarization orthogonal to the first polarization. The one or more processors are configured to operate the vehicle based on a reflectivity ratio between the first polarization signal and the second polarization signal.

[0006] In some embodiments of the present disclosure, a method includes transmitting a transmission signal from a laser source and receiving a return signal reflected by an object. The method includes generating, by one or more optical systems, a first polarization signal of the return signal having a first polarization. The method includes generating, by one or more optical systems, a second polarization signal of the return signal having a second polarization orthogonal to the first polarization. The method includes operating, by one or more processors, a vehicle based on a reflectivity ratio between the first polarization signal and the second polarization signal.

[0007] In one aspect, the present disclosure relates to a Light Detection and Ranging (LIDAR) system. The LIDAR system includes a transmitter configured to transmit a transmission signal from a laser source, a receiver configured to receive a return signal reflected by an object, one or more optical systems, and a processor. The one or more optical systems are configured to generate a first polarization signal for the return signal having a first polarization and a second polarization signal for the return signal having a second polarization. The processor is configured to calculate a reflectance value based on a signal-to-noise ratio (SNR) value of the first image and a SNR value of the second image.

[0008] In some embodiments, the one or more optical systems are configured to polarize the return signal to a first polarization to generate a first polarized signal. The one or more optical systems are configured to polarize the return signal to a second polarization to generate a second polarized signal. The one or more optical systems are configured to detect the first polarized signal and the second polarized signal. The one or more optical systems are further configured to detect the second polarized signal by shifting the phase of the second polarized signal and detecting the phase-shifted second polarized signal.

[0009] In some implementations, the processor is further configured to calculate the reflectivity value by calculating a ratio between an average SNR value of the first polarization signal and an average SNR value of the second polarization signal.

[0010] In another aspect, the present disclosure relates to an autonomous vehicle control system including one or more processors. The one or more processors are configured to cause a transmitter to transmit a transmission signal from a laser source. The one or more processors are configured to cause a receiver to receive a return signal reflected by an object. The one or more processors are configured to cause one or more optical systems to generate a first polarization signal of the return signal having a first polarization and a second polarization signal of the return signal having a second polarization. The first polarization signal represents a first image of the object having the first polarization, and the second polarization signal represents a second image of the object having the second polarization. The one or more processors are driven to calculate a reflectance value based on a signal-to-noise ratio (SNR) value of the first image and the SNR value of the second image, and to operate the vehicle based on the reflectance value.

[0011] In some embodiments, the one or more processors are configured to cause the one or more optical systems to polarize the return signal to a first polarization to generate a first polarized signal, cause the one or more optical systems to polarize the return signal to a second polarization to generate a second polarized signal, and cause the one or more optical systems to detect the first polarized signal and the second polarized signal. The one or more processors are further configured to cause the one or more optical systems to detect the second polarized signal by shifting the phase of the second polarized signal and the one or more optical systems to detect the phase-shifted second polarized signal.

[0012] In some embodiments, the one or more processors are further configured to determine one or more characteristics of the object based on the calculated reflectance value and control a vehicle trajectory based on the one or more characteristics of the object. The one or more characteristics of the object include a type of the object. The one or more processors are further configured to determine the type of the object as one of an asphalt road, lane markings, rough concrete road, grass, or gravel, and determine that the object is an asphalt road based on the reflectance value. The one or more processors are further configured to determine the type of the object as one of a metal pole, a tree, or a utility pole, and determine that the object is a metal pole based on the calculated reflectance value. The one or more processors are further configured to determine the type of the object as one or more people, and determine respective areas of skin and clothing of the one or more people based on the calculated reflectance value.

[0013] In another aspect, the present disclosure relates to an autonomous vehicle control system. The autonomous vehicle includes a light detection and ranging (LIDAR) system. The LIDAR system includes a transmitter, one or more optical systems, and one or more processors. The transmitter is configured to transmit a transmission signal from a vehicle controller to a laser source, a receiver configured to receive a return signal reflected by an object, the one or more optical systems, at least one of a steering system or a braking system, and a vehicle controller. The one or more optical systems are configured to generate a first polarization signal for the return signal having a first polarization and a second polarization signal for the return signal having a second polarization. The vehicle controller includes one or more processors. The one or more processors are configured to calculate a reflectance value based on a signal-to-noise ratio (SNR) value of the first image and the SNR value of the second image. The one or more processors are configured to control operation of at least one of the steering system or the braking system based on the reflectance value.

[0014] In some embodiments, the one or more processors are further configured to cause the one or more optical systems to polarize the return signal to a first polarization to generate a first polarized signal, cause the one or more optical systems to polarize the return signal to a second polarization to generate a second polarized signal, and cause the one or more optical systems to detect the first polarized signal and the second polarized signal. The one or more processors are further configured to cause the one or more optical systems to detect the second polarized signal by shifting the phase of the second polarized signal and the one or more optical systems to detect the phase-shifted second polarized signal.

[0015] In some embodiments, the one or more processors are further configured to determine one or more characteristics of the object based on the calculated reflectance value and control a path of the autonomous vehicle based on the one or more characteristics of the object. The one or more characteristics of the object include a type of the object. The one or more processors are further configured to determine the type of the object as one of an asphalt road, lane markings, rough concrete road, grass, or gravel, and determine the object is an asphalt road based on the calculated reflectance value. The one or more processors are further configured to determine the type of the object as one of a metal pole, a tree, or a utility pole, and determine the object is a metal pole based on the calculated reflectance value. The one or more processors are further configured to determine the type of the object as one or more people, and determine respective areas of skin and clothing of the one or more people based on the calculated reflectance value. [Brief explanation of the drawings]

[0016] The patent or application file contains at least one color drawing. Copies of this patent application publication with color drawing(s) will be provided by the appropriate office upon request and payment of the necessary fee.

[0017] These and other aspects and features of the present embodiments will become apparent to those skilled in the art upon review of the following description of specific embodiments in conjunction with the accompanying drawings.

[0018] [Figure 1a] 1 is a block diagram illustrating an example system environment for an autonomous vehicle, according to some embodiments.

[0019] [Figure 1b] 1 is a block diagram illustrating an example system environment for an autonomous commercial trucking vehicle, according to some embodiments.

[0020] [Figure 1c] 1 is a block diagram illustrating an example system environment for an autonomous commercial trucking vehicle, according to some embodiments.

[0021] [Figure 1d] 1 is a block diagram illustrating an example system environment for an autonomous commercial trucking vehicle, according to some embodiments.

[0022] [Figure 2] 1 is a block diagram illustrating an example of a computing system according to some embodiments.

[0023] [Figure 3a] 1 is a block diagram illustrating an example of a LIDAR system according to some embodiments.

[0024] [Figure 3b] 1 is a block diagram illustrating another example of a LIDAR system according to some embodiments.

[0025] [Figure 4a] 10A-10C are images illustrating various examples of depolarization ratio data according to some embodiments. [Figure 4b] 10A-10C are images illustrating various examples of depolarization ratio data according to some embodiments. [Figure 4c] 10A-10C are images illustrating various examples of depolarization ratio data according to some embodiments. [Figure 4d]10A-10C are images illustrating various examples of depolarization ratio data according to some embodiments. [Figure 4e] 10A-10C are images illustrating various examples of depolarization ratio data according to some embodiments. [Figure 4f] 10A-10C are images illustrating various examples of depolarization ratio data according to some embodiments. [Figure 4g] 10A-10C are images illustrating various examples of depolarization ratio data according to some embodiments. [Figure 4h] 10A-10C are images illustrating various examples of depolarization ratio data according to some embodiments. [Figure 4i] 10A-10C are images illustrating various examples of depolarization ratio data according to some embodiments. [Figure 4j] 10A-10C are images illustrating various examples of depolarization ratio data according to some embodiments.

[0026] [Figure 5] 1 is a flowchart illustrating an example methodology for controlling a vehicle's path based on a depolarization ratio, according to some implementations.

[0027] [Figure 6] 1 is a flowchart illustrating an example methodology for operating a vehicle based on a depolarization ratio, according to some implementations. DETAILED DESCRIPTION OF THE INVENTION

[0028] According to certain aspects, embodiments of the present disclosure relate to a system and method for controlling a vehicle using light detection and ranging (LIDAR), and more particularly, to a system and method for controlling a vehicle by detecting an object using the depolarization ratio of a return signal reflected by the object.

[0029] According to certain aspects, an autonomous vehicle control system includes one or more processors. The one or more processors are configured to cause a transmitter to transmit a transmission signal from a laser source. The one or more processors are configured to cause a receiver to receive a return signal reflected by an object. The one or more processors are configured to cause one or more optical systems to generate a first polarization signal of the return signal having a first polarization and a second polarization signal of the return signal having a second polarization orthogonal to the first polarization. The one or more processors are configured to operate the vehicle based on a reflectivity ratio between the first polarization signal and the second polarization signal.

[0030] In conventional LIDAR systems, the laser signal (LS) is linearly polarized. When reflecting the signal transmitted from the LIDAR system, many objects in the world depolarize the return signal. For example, each object's return signal may return with the same polarization state as the polarized LS or with a different polarization state than the polarized LS. However, the LIDAR system only detects the polarization portion of the return signal that matches the polarized LS. As a result, other polarizations of the return signal are not measured or utilized.

[0031] To address this issue, in some embodiments, polarization-sensitive LIDAR is implemented by splitting the return light into two polarization states so that the two polarization states of the return light can be detected independently. The two polarization state signals are then compared to estimate how much the object (or target) depolarized the return signal. The LIDAR system includes a polarization beam splitter (PBS). PBSs are polarization beam splitters / combiners (PBSCs). The split return signal represents separate images of the object, each with a different polarization state. The LIDAR system calculates the ratio between the separate images of the object with the different polarization states (referred to as the "depolarization ratio").

[0032] In some implementations, the LIDAR system includes two detectors configured to detect respective polarization signals from the return signal using a splitter, where the splitter is a polarizing beam splitter (PBS). The LIDAR system includes a single detector configured to detect two polarization signals from the return signal using a splitter and a phase shifter by multiplexing the return signal onto the single detector.

[0033] In some embodiments, in response to detecting the two polarization signals, one or more detectors generate two corresponding electrical signals from separate channels, which are independently processed by a processing system. The two electrical signals represent respective separate images of the object with different polarization states. The LIDAR system calculates a depolarization ratio between the separate images of the object with different polarization states.

[0034] In some implementations, the system has two beams of polarization-sensitive LIDAR, each with two receive / digitizer channels for independent signal processing. The system processes independent streams from points in the cloud on two independent channels.

[0035] In some embodiments, the processing system performs simple post-processing on the electrical signals in each channel to generate an image of the object in each polarization. The processing system calculates the average reflectance of the image of the object across multiple samples. The processing system performs spatial averaging per voxel. For example, the processing system includes a hash-based voxelizer to efficiently generate multiple voxels representing the object's polarization state. Using the hash-based voxelizer, the processing system quickly searches for voxels. The processing system calculates the average reflectance within each of the multiple voxels across the multiple samples. As a result, the calculated average reflectance may differ from the measured reflectance. For example, the individual measurements are uncorrelated, but the average is correlated. The number of samples for averaging is less than 100. For example, 5 or 12 samples are used. The processing system calculates the ratio of the average reflectance within each voxel between the two channels.

[0036] In some embodiments, the average reflectance over a region of space (e.g., over a subset of voxels) is meaningful in understanding the polarization of an object. In some embodiments, the use of average reflectance effectively reduces variance in reflectance measurements caused by laser speckle, for example, when a coherent LIDAR system is used. The average reflectance of voxels over multiple samples is influenced by the selection of parameters, such as spatial resolution or precision (e.g., 5 cm x 10 cm voxels). In general, averaging smaller voxels contributes more to image contrast (making objects more distinguishable) while having a smaller contribution to averaging the overall image of the object. Appropriate parameter selection provides another dimension from measuring pure water reflectance, thereby enhancing the value of the data.

[0037] According to certain aspects, the present disclosure relates to a light detection and ranging (LIDAR) system including a transmitter configured to transmit a transmission signal from a laser source, a receiver configured to receive a return signal reflected by an object, one or more optical systems, and a processor. The one or more optical systems are configured to generate a first polarization signal of the return signal having a first polarization and a second polarization signal of the return signal having a second polarization orthogonal to the first polarization. The processor is configured to calculate a ratio of reflectivities of the first polarization signal and the second polarization signal.

[0038] According to certain aspects, some embodiments of the present disclosure relate to a method including transmitting a transmission signal from a laser source and receiving a return signal reflected by an object. The method includes generating, by one or more optical systems, a first polarization signal of the return signal having a first polarization. The method includes generating, by one or more optical systems, a second polarization signal of the return signal having a second polarization orthogonal to the first polarization. The method includes operating, by one or more processors, a vehicle based on a reflectivity ratio between the first polarization signal and the second polarization signal.

[0039] Various embodiments of the present disclosure have one or more of the following advantages and benefits.

[0040] First, embodiments of the present disclosure provide a useful technique for improving the disambiguation of different objects using a depolarization ratio in addition to a conventionally used signal, such as a single reflectance signal, or other unique signals mapped onto a point cloud. In some embodiments, object detection based on the depolarization ratio clarifies several major surfaces, such as (1) asphalt (vs. grass, rough concrete, or gravel), (2) metal poles (vs. trees or telephone / utility poles), (3) retro signs (vs. metal surfaces), (4) lane markings (vs. road surfaces), and (5) vehicle license plates (vs. vehicle surfaces). This disambiguation technique is useful for recognizing specific road signs, signs, pedestrians, etc. Although the depolarization ratio detects or recognizes sparse features (e.g., features with relatively smaller areas), such sparse features are easily registered for disambiguation of different objects. Distinguishing different objects or materials based on depolarization ratios can help a perception system more accurately detect, track, determine, and / or classify objects (e.g., using artificial intelligence techniques) within the vehicle's environment.

[0041] Second, embodiments of the present disclosure provide a useful technique for improving the stability and accuracy of object detection by utilizing differential measurements (e.g., the ratio of reflectance between signals with different polarization states) that should lead to low variance across changing conditions (e.g., weather changes - snow, ice, rain, etc.).

[0042] Third, embodiments of the present disclosure provide a useful technique for making LIDAR systems more interchangeable than specific data products. Metallic or specular surfaces sometimes produce stronger "glint" returns in radar data. This "glint" effect exhibits less dispersion at other common LIDAR wavelengths (i.e., 905 nm vs. 1550 nm). This means that reflectivity-based features often vary significantly at different wavelengths. Because polarization ratio measurements are less sensitive to the exact wavelength, LIDAR systems can be more interchangeable than specific data products.

[0043] Fourth, the depolarization ratio relates to the size of the angle of incidence between the LIDAR beam and the object, which is useful for determining surface normals, which are commonly used in localization and mapping. In some embodiments, one or more types of data or pieces of information related to the size of the angle of incidence are obtained from the depolarization ratio measurement, which can be useful, for example, for localization and mapping.

[0044] Fifth, embodiments of the present disclosure provide useful techniques for improving other technical fields, such as localization (e.g., spatial relationships between vehicles and stationary objects), camera simulation, LIDAR / radar simulation, and radar measurement. For example, contrast detection of common building materials is utilized for localization. In some embodiments, the depolarization ratio indicates whether a surface has diffuse or specular reflection characteristics. Surface characteristic information for diffuse or specular reflection is obtained based on the depolarization ratio and represented as a high-definition (HD) map, and such surface characteristic information is extracted by an autonomous vehicle control system. Such surface characteristic information is used in camera simulation to model various lighting conditions. Similarly, the depolarization ratio is utilized in integrated LIDAR / radar simulation. Such surface characteristic information is also utilized in analyzing radar data or camera data. 1. System environment for autonomous vehicles

[0045] FIG. 1a is a block diagram illustrating an example system environment for an autonomous vehicle, according to some embodiments.

[0046] 1a, an exemplary autonomous vehicle 110A is shown that may embody various technologies disclosed herein. Vehicle 110A may include, for example, a powertrain 192 including a prime mover 194 powered by an energy source 196 to power a drivetrain 198, and a control system 180 including a directional control device 182, a powertrain control device 184, and a brake control device 186. While vehicle 110A may be embodied as any number of different vehicle types, including vehicles that transport people and / or cargo and operate in a variety of environments, it should be understood that the components 180-198 described above may vary widely based on the type of vehicle in which they are utilized.

[0047] For simplicity, the embodiments discussed below focus on wheeled land vehicles such as cars, vans, trucks, buses, etc. In such embodiments, prime mover 194 includes (among other things) one or more electric motors and / or internal combustion engines. Energy sources include, for example, a fuel system (e.g., providing gasoline, diesel, hydrogen, etc.), a battery system, solar panels, or other renewable energy sources, and / or a fuel cell system. Drivetrain 198 includes not only the wheels and / or tires, along with a transmission and / or any other mechanical drive components for converting the power output of prime mover 194 into vehicle motion, but also one or more brakes configured to controllably stop or slow vehicle 110A, and direction or steering components adapted to control the path of vehicle 110A (e.g., a rack and pinion steering linkage that allows one or more wheels of vehicle 110A to rotate about an approximately vertical axis to change the angle of the plane of rotation of the wheel relative to the vehicle's longitudinal axis). In some implementations, a combination of powertrains and energy sources is used (e.g., in the case of electric / gas hybrid vehicles), and in some cases multiple electric motors are used as prime movers (e.g., dedicated to individual wheels or axles).

[0048] The directional control device 182 includes one or more actuators and / or sensors for receiving feedback from the directional or steering components so that the vehicle 110A follows a desired path. The powertrain control device 184 is configured to control the output of the powertrain 102 to control the speed and / or direction of the vehicle 110A, for example, by controlling the output power of the prime mover 194, controlling the gears of the transmission in the drivetrain 198, etc. The brake control device 116 is configured to control one or more brakes, for example, disc or drum brakes coupled to the wheels of the vehicle, to slow or stop the vehicle 110A.

[0049] Other vehicle types, including but not limited to off-road vehicles, all-terrain or tracked vehicles, construction equipment, etc., may require the use of different powertrains, drivetrains, energy sources, directional control devices, powertrain controllers, and brake controllers. Furthermore, in some embodiments, some of the components may be combined, for example, where vehicle directional control is primarily accomplished by varying the power output of one or more prime movers. Thus, embodiments disclosed herein are not limited to the specific application of the technology described herein to autonomous wheeled land vehicles.

[0050] Various levels of autonomous driving control for vehicle 110A are embodied in vehicle control system 120, which includes one or more processors 122 and one or more memories 124, each configured to execute program code instructions 126 stored in memory 124. The processor(s) may include, for example, graphic processing unit(s) (“GPU(s)”) and / or central processing unit(s) (“CPU(s)”).

[0051] The sensors 130 include various sensors adapted to collect information from the vehicle's environment for use in controlling the vehicle's operation. For example, the sensors 130 include a radar sensor 134, a LIDAR sensor 136, a 3D positioning sensor 138, such as an accelerometer, a gyroscope, a magnetometer, or any one of a number of satellite navigation systems, such as the Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou Navigation Satellite System (BDS), Galileo, Compass, etc. The 3D positioning sensor 138 is used to determine the vehicle's position on Earth using satellite signals. The sensors 130 include a camera 140 and / or an inertial measurement unit (IMU) 142. The camera 140 may be a monographic or stereographic camera and may record still and / or video images. The IMU 142 includes multiple gyroscopes and accelerometers that detect the linear and rotational motion of the vehicle in three directions. One or more encoders (not shown), such as wheel encoders, are used to monitor the rotation of one or more wheels of the vehicle 110A. Each sensor 130 outputs sensor data at a different data rate than the data rates of the other sensors 130.

[0052] The outputs of the sensors 130 are provided to a set of control subsystems 150, including a localization subsystem 152, a planning subsystem 156, a recognition subsystem 154, and a control subsystem 158. The localization subsystem 152 functions to accurately determine the position and orientation (sometimes referred to as "pose") of the vehicle 110A within its surrounding environment and approximately within some frame of reference. The autonomous vehicle's position is part of a labeled autonomous vehicle data generation and is compared to the positions of additional vehicles in the same environment. The recognition subsystem 154 functions to detect, track, determine, and / or identify objects within the environment surrounding the vehicle 110A. Machine learning models are utilized to track the objects. The planning subsystem 156 functions to plan a path for the vehicle 110A over a given time frame relative to a desired destination and stationary and moving objects within the environment. Machine learning is utilized to plan the vehicle's path. Control subsystem 158 functions to generate appropriate control signals to control various controllers in vehicle control system 120 to implement the planned path of vehicle 110A. Machine learning models are utilized to generate one or more signals to control the autonomous vehicle to implement the planned path.

[0053] It should be understood that the collection of components illustrated in FIG. 1a for vehicle control system 120 is merely exemplary in nature. Individual sensors may be omitted in some embodiments. Additionally or alternatively, in some embodiments, multiple sensors of the type illustrated in FIG. 1a are used for redundancy and / or to cover different areas around the vehicle, although other types of sensors may be used. Similarly, different types and / or combinations of control subsystems may be used in other embodiments. Also, while subsystems 152-158 are shown as being separate from processor 122 and memory 124, in some embodiments, some or all of the functionality of subsystems 152-158 may be implemented by program code instructions 126 resident in one or more memories 124 and executed by one or more processors 122, and such subsystems 152-158 may in some cases be implemented using the same processor(s) and / or memory. The subsystems may be implemented at least in part using various dedicated circuit logic, various processors, various Field Programmable Gate Arrays (FPGAs), various Application-Specific Integrated Circuits (ASICs), various real-time controllers, etc. As noted above, many subsystems may utilize circuits, processors, sensors, and / or other components. Additionally, the various components within vehicle control system 120 may be networked in various ways.

[0054] In some embodiments, vehicle 110A also includes an auxiliary vehicle control system (not shown) that is used as a redundant or backup control system for vehicle 110A. The auxiliary vehicle control system fully operates autonomous vehicle 110A when an adverse event occurs in vehicle control system 120, but in other embodiments, the auxiliary vehicle control system may only have limited functionality, such as performing a controlled stop of vehicle 110A in response to an adverse event detected by main vehicle control system 120. In still other embodiments, the auxiliary vehicle control system is omitted.

[0055] Numerous different architectures, including various combinations of software, hardware, circuit logic, sensors, networks, etc., may be used to implement the various components illustrated in FIG. 1a. Each processor may be embodied, for example, as a microprocessor, and each memory may refer to not only a random access memory (RAM) device including the main storage location, but also additional levels of memory, such as cache memory, non-volatile or backup memory (e.g., programmable or flash memory), read-only memory, etc. Each memory may also be considered to include memory storage devices physically located elsewhere within vehicle 110A, such as any cache memory within the processor, as well as any storage capacity used as virtual memory, such as that stored on a mass storage device or other computer controller. One or more of the processors illustrated in FIG. 1a, or entirely separate processors, may be used to implement additional functions within vehicle 110A other than those for autonomous driving control, such as controlling an entertainment system, operating doors, lights, convenience features, etc.

[0056] Additionally, for further storage, vehicle 110A may include one or more mass storage devices, such as, among others, a portable disk drive, a hard disk drive, a direct access storage device (DASD), an optical drive (e.g., a CD drive, a DVD drive, etc.), a solid state drive (SSD), a network attached storage, a storage area network, and / or a tape drive.

[0057] Additionally, vehicle 110A includes a user interface 164, such as one or more displays, touchscreens, voice and / or gesture interfaces, buttons and other tactile controls, etc., to enable vehicle 110A to receive multiple inputs from and generate outputs for a user or operator. Alternatively, user inputs may be received via other computers or electronic devices, such as via an app or web interface on a mobile device.

[0058] Vehicle 110A also includes one or more network interfaces (e.g., network interface 162) adapted to communicate with one or more networks 170 (e.g., a local area network (LAN), a wide area network (WAN), a wireless network, and / or the Internet, among others) to enable communication with other computers and electronic devices, including, for example, a central service such as a cloud service from which vehicle 110A receives environmental and other data for use in controlling its autonomous driving. Data collected by one or more sensors 130 is uploaded via network 170 to computing system 172 for further processing. A timestamp is added to each instance of vehicle data before being uploaded. Further processing of autonomous vehicle data by computing system 172 according to many embodiments is described with respect to FIG. 2.

[0059] 1a and various additional controllers and subsystems disclosed herein generally operate under the control of an operating system that executes or otherwise relies on various computer software applications, components, programs, objects, modules, data structures, etc., as described in more detail below. In addition, various applications, components, programs, objects, modules, etc. may also execute on one or more processors in other computers coupled to vehicle 110A via network 170, e.g., in a distributed cloud infrastructure or client-server computing environment, whereby the processing required to implement the functionality of a computer program is allocated across multiple computers and / or services via the network.

[0060] The routines executed to implement the various embodiments described herein, whether embodied as part of an operating system, a specific application, component, program, object, module, or sequence of instructions, or even a subset thereof, are referred to herein as "program code." Program code includes one or more instructions resident at various times in various memory and storage devices, and when read and executed by one or more processors, performs the steps necessary to perform the steps or elements embodying various aspects of the present disclosure. Also, while embodiments are described below in the context of fully operational computers and systems, the various embodiments described herein may be distributed as a program product in a variety of forms, and the embodiments are embodied regardless of the particular type of computer-readable medium used to actually distribute them.

[0061] Examples of computer-readable media include, among others, non-transitory media in tangible forms such as volatile and non-volatile memory devices, floppy disks and other portable disks, SSDs, hard disk drives, magnetic tape, and optical disks (e.g., CD-ROMs, DVDs, etc.).

[0062] Furthermore, various program code elements described below are identified based on the application in which they are implemented in a particular embodiment. It should be appreciated that any specific program nomenclature used herein is used merely for convenience, and thus the present disclosure should not be limited to use with any specific application identified / implied by such nomenclature. Additionally, given the myriad ways in which computer programs are typically organized into routines, procedures, methods, modules, objects, etc., and the various ways in which program functions are allocated among the various software layers present in a typical computer (e.g., operating systems, libraries, APIs, applications, applets, etc.), it should be appreciated that the present disclosure is not limited to the specific organization and allocation of program functions described herein.

[0063] The environment illustrated in Figure 1a is not intended to limit the embodiments disclosed herein, and in fact, other alternative hardware and / or software environments may be used without departing from the scope of the embodiments disclosed herein. 2. FM LIDAR for automotive applications

[0064] The truck includes a LIDAR system (e.g., vehicle control system 120 of FIG. 1a, LIDAR system 300 of FIG. 3a, LIDAR system 350 of FIG. 3b, etc.). In some embodiments, the LIDAR system uses frequency modulation to encode an optical signal and scatters the encoded optical signal into free space using an optical system. By sensing the frequency difference between the encoded optical signal and a return signal reflected from an object, the frequency modulated (FM) LIDAR system determines the object's location and / or accurately measures the object's velocity using the Doppler effect. FM LIDAR systems use continuous wave (referred to as "FMCW LIDAR" or "coherent FMCW LIDAR") or quasi-continuous wave (referred to as "FMQW LIDAR"). The LIDAR system uses phase modulation (PM) to encode an optical signal and scatters the encoded optical signal into free space using an optical system.

[0052] FM or phase-modulated (PM) LIDAR systems offer significant advantages over conventional LIDAR systems for automotive and / or commercial trucking applications. First, in some cases, an object (e.g., a pedestrian wearing dark clothing) has low reflectivity because it reflects only a small amount (e.g., less than 10%) of the light striking the object back to the FM or PM LIDAR system's sensor (e.g., sensor 130 in FIG. 1a). In other cases, an object (e.g., a lit road sign) has high reflectivity (e.g., greater than 10%) because it reflects a large amount of the light striking the object back to the FM LIDAR system's sensor.

[0053] Regardless of the object's reflectivity, FM LIDAR systems can detect (e.g., classify, recognize, locate, etc.) objects at greater distances (e.g., 2x) than conventional LIDAR systems. For example, FM LIDAR systems can sense low-reflectivity objects at distances of 300 meters or more and light-reflectivity objects at distances of 400 meters or more.

[0054] To achieve this improvement in detection capabilities, FM LIDAR systems use sensors (e.g., sensor 130 in FIG. 1a). In some embodiments, these sensors are sensitive to single photons, meaning they can detect the smallest amount of light possible. FM LIDAR systems use infrared wavelengths (e.g., 950 nm, 1550 nm, etc.) in some applications, but are not limited to infrared wavelength ranges (e.g., near-infrared: 800 nm to 1500 nm, mid-infrared: 1500 nm to 5600 nm, and far-infrared: 5600 nm to 1,000,000 nm). By operating an FM or PM LIDAR system at infrared wavelengths, the FM or PM LIDAR system broadcasts a stronger light pulse or beam while meeting eye safety standards. Conventional LIDAR systems are sometimes not sensitive to single photons or operate only at near-infrared wavelengths, limiting their high power output (and distance sensing capabilities) for eye safety reasons.

[0055] Thus, by sensing objects at greater distances, FM LIDAR systems have more time to react to unexpected obstacles. Indeed, even a few milliseconds of additional time can improve safety and ride comfort, especially for large vehicles (e.g., commercial trucking vehicles) traveling at highway speeds.

[0056] Another advantage of FM LIDAR systems is that they provide accurate instantaneous velocity for each data point. In some embodiments, velocity measurements are made using the Doppler effect, which shifts the frequency of light received from an object based on at least one of the radial velocity (e.g., the direction vector between the detected object and the sensor) or the frequency of the laser signal. For example, for on-road velocities of less than 100 meters per second (m / s), such a shift corresponds to a frequency shift of less than 130 megahertz (MHz) at a wavelength of 1550 nanometers (nm). Such a frequency shift is too small to be directly detected in the optical domain. However, using coherent detection in FMCW, PMCW, or FMQW LIDAR systems, the signal is converted to the RF domain, where the frequency shift can be calculated using various signal processing techniques. This allows autonomous vehicle control systems to process collected data more quickly.

[0057] Calculating instantaneous velocity makes it easier for an FM LIDAR system to determine distance, identify rare data points as objects, and / or track how such objects move over time. For example, if an FM LIDAR sensor (e.g., sensor 130 in FIG. 1a) receives only a few returns (e.g., hits) for an object at a distance of 300 m, but these returns provide interesting velocity values ​​(e.g., moving toward the vehicle at a speed >70 mph), the FM LIDAR system and / or the autonomous vehicle control system can weight each of them toward the probability of the object being detected.

[0058] Faster identification and / or tracking of an FM LIDAR system gives an autonomous vehicle control system more time to steer the vehicle. With a better understanding of how fast an object is moving, an autonomous vehicle control system can better plan a response.

[0059] Another advantage of FM LIDAR systems is that they are less static than traditional LIDAR systems. That is, traditional LIDAR systems, which are designed to be more sensitive to light, typically perform poorly in bright light. Such systems are also prone to performance degradation due to crosstalk (e.g., when sensors are cross-talked by each other's light pulses or light beams) and self-interference (e.g., when a sensor is cross-talked by its own previous light pulse or light beam). To overcome these shortcomings, vehicles using traditional LIDAR systems sometimes require additional hardware, complex software, and / or more computing power to manage this "noise."

[0060] In contrast, FM LIDAR systems do not encounter these types of problems because each sensor is specifically designed to respond only to its own light characteristics (e.g., light beam, light wave, light pulse). If the returning light does not match the timing, frequency, and / or wavelength of the originally transmitted light, the FM sensor filters (e.g., removes or ignores) the corresponding data point. As a result, FM LIDAR systems produce (e.g., generate, derive) more accurate data using fewer hardware or software requirements, enabling a safer and smoother ride.

[0061] Finally, FM LIDAR systems are easier to scale than traditional LIDAR systems. As more autonomous vehicles (e.g., cars, commercial trucks, etc.) appear on the road, vehicles powered by FM LIDAR systems can avoid interference issues caused by sensor crosstalk. FM LIDAR systems also use less optical peak power than traditional LIDAR sensors. Thus, some or all of the optical components for an FM LIDAR can be fabricated on a single chip, which creates its own advantages as discussed herein. 3. Commercial transport trucks

[0062] FIG. 1B is a block diagram illustrating an example system environment for autonomous commercial trucking vehicles, according to some embodiments. The environment 100B includes a commercial truck 102B for transporting cargo 106B. In some embodiments, the commercial truck 102B includes a vehicle configured for long-haul freight, regional freight, intermodal freight (i.e., transportation in which a road-based vehicle is used as one of multiple transportation modes to move cargo), and / or any other road-based freight application. The commercial truck 102B may be a flatbed truck, a refrigerated truck (e.g., a reefer truck), a vented van (e.g., a dry van), a moving truck, etc. The cargo 106B may be goods and / or products. The commercial truck 102B may include a trailer for transporting the cargo 106B, such as a platbed trailer, a lowboy trailer, a step deck trailer, an extendable platbed trailer, a sidekit trailer, etc.

[0063] The environment 100B includes an object 110B (shown as another vehicle in FIG. 1b) within a distance range of 30 meters or less from the truck.

[0064] The commercial truck 102B includes a LIDAR system 104B (e.g., an FM LIDAR system, the vehicle control system 120 of FIG. 1a, the LIDAR system 300 of FIG. 3a, the LIDAR system 350 of FIG. 3b, etc.) for determining a distance to or measuring the velocity of the object 110B. While FIG. 1b shows one LIDAR system 104B mounted on the front of the commercial truck 102B, the number of LIDAR systems and the mounting areas of the LIDAR systems on the commercial truck are not limited to any particular number or area. The commercial truck 102B may include any number of LIDAR systems 104B (or components such as sensors, modulators, coherent signal generators, etc.) mounted on any area of ​​the commercial truck 102B (e.g., the front, rear, sides, top, bottom, underside, and / or bottom) to facilitate detection of objects in any free space relative to the commercial truck 102B.

[0065] As shown, LIDAR system 104B in environment 100B is configured to detect objects (e.g., other vehicles, bicycles, trees, street signs, depressions, etc.) at a short distance (e.g., 30 meters or less) from commercial truck 102B.

[0066] 1c is a block diagram illustrating an example system environment for an autonomous commercial trucking vehicle, according to some embodiments. Environment 100C includes the same components included in environment 100B (e.g., commercial truck 102B, cargo 106B, LIDAR system 104B, etc.).

[0067] Environment 100C includes object 110C (shown as other vehicles in FIG. 1c) within a distance range of (i) more than 30 meters and (ii) less than or equal to 150 meters from commercial truck 102B. As shown, LIDAR system 104B in environment 100C is configured to detect objects (e.g., other vehicles, bicycles, trees, street signs, depressions, etc.) within a fixed distance (e.g., 100 meters) from commercial truck 102B.

[0068] 1d is a block diagram illustrating an example system environment for an autonomous commercial trucking vehicle, according to some embodiments. Environment 100D includes the same components included in environment 100B (e.g., commercial truck 102B, cargo 106B, LIDAR system 104B, etc.).

[0069] Environment 100D includes object 110D (shown as another vehicle in FIG. 1d) within a distance range of greater than 150 meters from commercial truck 102B. As shown, LIDAR system 104B within environment 100D is configured to detect objects (e.g., other vehicles, bicycles, trees, street signs, depressions, etc.) within a certain distance (e.g., 300 meters) from commercial truck 102B.

[0070] In commercial trucking applications, effective detection of objects at all ranges is important due to the increased weight and resulting longer stopping distances required for such vehicles. FM LIDAR systems (e.g., FMCW and / or FMQW systems) or PM LIDAR systems are highly suitable for commercial trucking applications due to the advantages described above. As a result, commercial trucks equipped with such systems have an improved ability to safely transport people and goods over short or long distances, improving the safety of not only the commercial truck but also surrounding vehicles. In various embodiments, such FM or PM LIDAR systems are used in semi-autonomous driving applications, in which the commercial truck has a driver and some functions of the commercial truck are operated autonomously using the FM or PM LIDAR system, or in fully autonomous driving applications, in which the commercial truck is operated entirely by the FM or LIDAR system, either alone or in conjunction with other vehicle systems. 4. Continuous Wave Modulation and Quasi-Continuous Wave Modulation

[0071] In a LIDAR system using CW modulation, the modulator continuously modulates the laser light. For example, if the modulation cycle is 10 seconds, the input signal is modulated for the entire 10 seconds. In contrast, in a LIDAR system using quasi-CW modulation, the modulator modulates the laser light to have both active and inactive portions. For example, for a 10-second cycle, the modulator only modulates the laser light for 8 seconds (sometimes referred to as the "active portion") and does not modulate the laser light for 2 seconds (sometimes referred to as the "inactive portion"). In this way, the LIDAR system can reduce power consumption for the 2 seconds because the modulator does not need to provide a continuous signal.

[0072] In Frequency Modulated Continuous Wave (FMCW) LIDAR for automotive applications, it is advantageous to operate the LIDAR system using quasi-CW modulation that employs FMCW measurement and signal processing methodologies, even when the optical signal is not always on (e.g., activated, powered, transmitting, etc.). In some embodiments, the quasi-CW modulation has an operating cycle of 1% or more and up to 50%. If energy is wasted during the actual measurement time in the off state (e.g., inactive, powered down, etc.), there may be a boost to the signal-to-noise ratio (SNR) and / or a reduction in signal processing requirements to consistently integrate all energy over longer time scales. 5. LIDAR system using the depolarization ratio of the return signal

[0073] FIG. 2 is a block diagram illustrating an example computing system according to some implementations.

[0074] 2, the illustrated exemplary computing system 172 includes one or more processors 210 in communication with memory 260 via a communication system 240 (e.g., a bus), at least one network interface controller 230 having a network interface port for accessing a network (not shown), and an input / output ("I / O") component interface 450 for accessing other components, such as a display (not shown) and input devices (not shown). Generally, the processor(s) 210 execute instructions (or computer programs) received from memory. The illustrated processor(s) 210 integrate with or are directly coupled to a cache memory 220. In some cases, instructions are read from memory 260 into the cache memory 220 and executed by the processor(s) 210 from the cache memory 220.

[0075] More specifically, the processor(s) 210 are any logic circuitry that processes instructions, e.g., instructions fetched from memory 260 or cache 220. In some implementations, the processor(s) 210 are microprocessor units or special-purpose processors. The computing device 400 is based on any processor or set of processors that operate as described herein. The processor(s) 210 are single-core or multi-core processor(s). The processor(s) 210 are multiple separate processors.

[0076] Memory 260 is any device suitable for storing computer-readable data. Memory 260 may be a device having a fixed storage location or a device for reading portable storage media. Examples include all forms of non-volatile memory, media, and memory devices, semiconductor memory devices (e.g., EPROM, EEPROM, SDRAM, and flash memory devices), magnetic disks, magneto-optical disks, and optical disks (e.g., CD-ROM, DVD-ROM, and Blu-Ray® disks). Computing system 172 may include memory 260 and any number of memory devices.

[0077] Cache memory 220 is a form of computer memory that is located in close proximity to processor(s) 210 for approximately fast read times. In some implementations, cache memory 220 is part of or on the same chip as processor(s) 210. In some implementations, there are multiple levels of cache 220, for example, an L2 and L3 cache hierarchy.

[0078] The network interface controller 230 manages data exchange through the network interface (sometimes referred to as a network interface port). The network interface controller 230 handles the physical and data link layers of the OSI model for network communication. In some embodiments, some of the network interface controller's work is handled by one or more of the processor(s) 210. In some embodiments, the network interface controller 230 is part of the processor 210. In some embodiments, the computing system 172 has multiple network interfaces controlled by a single controller 230. In some embodiments, the computing system 172 has multiple network interface controllers 230. In some embodiments, each network interface is a connection point to a physical network link (e.g., a Cat-5 Ethernet link). In some implementations, the network interface controller 230 supports wireless network connectivity, and the interface port is a wireless (e.g., radio) receiver / transmitter (e.g., for IEEE 802.11 protocol, Near Field Communication (NFC), Bluetooth, ANT, or any other wireless protocol). In some implementations, the network interface controller 230 implements one or more network protocols, such as Ethernet. The computing device 172 exchanges data with other computing devices over a physical or wireless link, typically via a network interface. The network interface couples to other devices directly or through intermediate network devices, such as a hub, bridge, switch, or router, thereby coupling the computing device 172 to a data network, such as the Internet.

[0079] Computing system 172 includes or provides an interface to one or more input or output ("I / O") devices. Input devices include, but are not limited to, keyboards, microphones, touchscreens, foot pedals, sensors, MIDI devices, and pointing devices such as mice or trackballs. Output devices include, but are not limited to, video displays, speakers, refreshable Braille terminals, lighting, MIDI devices, and 2D or 3D printers.

[0080] Other components include I / O interfaces, external serial device ports, and additional co-processors. For example, computing system 172 includes interfaces (e.g., universal serial bus (USB) interfaces) for connecting input devices, output devices, or additional memory devices (e.g., portable flash drives or external media drives). In some implementations, computing device 172 includes additional devices such as co-processors. For example, a mathematical co-processor assists processor 210 with highly precise or complex calculations.

[0081] 3a is a block diagram illustrating an example of a LIDAR system according to some embodiments. In some embodiments, the LIDAR system 300 is the LIDAR sensor 136 (see FIG. 1a). The LIDAR system 300 includes a laser 302, a modulator 304, a circulator optics 306, a scanner 308, a polarizing beam splitter (PBS) 312, a first detector 314, a second detector 316, and a processing system 318. The PBS is a polarizing beam splitter / combiner (PBSC). In some embodiments, the LIDAR system 300 is a coherent FMCW LIDAR.

[0082] In some implementations, laser 302 emits a laser output (LO) signal that is carrier 303. A splitter (not shown) splits the unmodulated LO signal into carrier 303, LO signal 321, and LO signal 323 that are in the same polarization state (referred to as the first polarization state).

[0083] In some embodiments, the modulator 304 receives a carrier wave 303 and phase- or frequency-modulates the carrier wave 303 to generate a modulated optical signal 305 in a first polarization state. The modulator 304 is a frequency-shifting device (acousto-optic modulator). The modulated optical signal 305 is generated using a time delay of a waveform modulation of a local oscillator. The modulator 304 uses frequency modulation (FM) so that (FM) LIDAR systems encode an optical signal and scatter the encoded optical signal into free space using an optical system. FM LIDAR systems use continuous wave (referred to as "FMCW LIDAR") or quasi-continuous wave (referred to as "FMQW LIDAR"). The modulator 304 uses phase modulation (PM) so that (PM) LIDAR systems encode an optical signal and scatter the encoded optical signal into free space using an optical system. In some embodiments, the modulator 304 uses polarization modulation.

[0084] In some embodiments, the modulated optical signal 305 is processed via circulator optics 306 into signal 307, which is input to scanner 308. Circulator optics 306 may be, but is not limited to, an optical circulator (e.g., a fiber optic coupled circulator). For example, circulator optics 306 may be an optical isolator. In some embodiments, circulator optics 306 is a free-space pitch catch optic (e.g., pitch optics and catch optics).

[0085] In some embodiments, a transmit signal 309 is transmitted through a scanner 308 to illuminate an object 310 (or region of interest). The transmit signal 309 is in a first polarization state. In some embodiments, the scanner 308 includes scanning optics (not shown), such as a polygonal scanner having multiple mirrors or facets. The scanner 308 receives a return signal 311 reflected by the object 310. The return signal 311 includes a signal portion in the first polarization state and / or a signal portion in a different polarization state (referred to as a "second polarization state"). The first polarization state is orthogonal to the second polarization state. The scanner 308 redirects the return signal 311 into a return signal 313.

[0086] In some implementations, return signal 313 is further redirected by circulator optics 306 into return signal 315 that is input to PBS 312. PBS 312 splits and polarizes return signal 315 into a first polarized optical signal 317 having a first polarization state and a second polarized optical signal 319 having a second polarization state. Circulator optics 306 and PBS 312 are integrated into a polarizing beam splitter / combiner (PBSC).

[0087] In some embodiments, the first detector 314 is a single paired or unpaired detector, or a one-dimensional (1D) or two-dimensional (2D) array of paired or unpaired detectors. The first detector 314 receives the LO signal 321 as a reference signal. The first detector 314 is an optical detector configured to detect an optical signal. The first detector 314 detects a first polarized signal (e.g., a first polarized optical signal) and outputs or generates a first electrical signal 325.

[0088] In some embodiments, the second detector 316 is a single paired or unpaired detector, or a one-dimensional (1D) or two-dimensional (2D) array of paired or unpaired detectors. The second detector 316 receives the LO signal 323 as a reference signal. The second detector 316 is an optical detector configured to detect an optical signal. The second detector 316 detects a second polarized signal (e.g., a second polarized optical signal) and outputs or generates a second electrical signal 327. The two electrical signals represent respective individual images of the object in different polarization states.

[0089] In some embodiments, processing system 318 has a configuration similar to computing system 172 (see FIG. 2). Processing system 318 is one or more computing systems having a configuration similar to computing system 172. Processing system 318 receives first electrical signal 325 and second electrical signal 327 and calculates a depolarization ratio (defined below) based on the electrical signals. First electrical signal 325 represents an image of an object in a first channel exhibiting a first polarization state, and second electrical signal 327 represents an image of the object in a second channel exhibiting a second polarization state. Processing system 318 processes first electrical signal 325 in the first channel and processes second electrical signal 327 in the second channel.

[0090] In some implementations, the processing system 318 (or its processor) is configured to compare the signal with two polarization states (e.g., electrical signals 325, 327) to estimate how much the object (or target) depolarized the return signal (by calculating a depolarization ratio, as defined below). In response to detecting the two polarization signals, one or more detectors (e.g., first and second detectors 314, 316) generate two corresponding electrical signals (e.g., electrical signals 325, 327 in FIG. 3a) in separate channels so that the electrical signals can be independently processed by the processing system 318. A LIDAR system (e.g., LIDAR system 300) provides two receive / digitizer channels (e.g., first and second channels) for independent signal processing of two beams having different polarization states (e.g., first polarized optical signal 317 and second polarized optical signal 319 in FIG. 3a). In this manner, the system processes independent streams from points in the cloud in two independent channels.

[0091] In some implementations, the processing system 318 (or its processor) is configured to calculate the depolarization ratio by calculating the ratio of reflectance between the first polarized optical signal 317 and the second polarized optical signal 319. The LIDAR system calculates the depolarization ratio by calculating the ratio of reflectance between separate images of an object (e.g., images represented by the first and second electrical signals) in different polarization states. The LIDAR system calculates the depolarization ratio according to the following equation:

[0092]

number

[0093] In some embodiments, the depolarization ratio is calculated using the ratio of average reflectance between two images with different polarization states, where average reflectance is defined for a certain region of space (e.g., for a certain voxel). The average reflectance of multiple samples is calculated for one or more voxels. The average reflectance of multiple samples for one or more voxels is affected by the selection of parameters, such as spatial resolution or precision (e.g., voxel order). Generally, averages over smaller voxels contribute less to the overall image average of an object, while contributing more to contrast, making objects more distinct. Selecting appropriate parameters for calculating the average reflectance (e.g., voxel order, number of samples, etc.) can add another dimension to a pure reflectance measurement, thereby enhancing the value of the data. The characteristics of the calculated average reflectance may differ from those of measured reflectance values. For example, individual measurements are uncorrelated, but averages are. In some embodiments, voxels of 5 cm x 10 cm order are used. The number of samples for averaging is less than 100. For example, 5 or 12 samples are used.

[0094] In some embodiments, the processing system 318 performs post-processing on the electrical signals 325 and 327 in each channel to generate respective images of the object in each polarization (e.g., a first image in a first polarization state and a second image in a second polarization state). The processing system 318 calculates the average reflectance of each image of the object across multiple samples. The processing system 318 performs spatial averaging per voxel. For example, the processing system 318 includes a hash-based voxelizer to efficiently generate multiple voxels representing the object in its polarization state. Using the hash-based voxelizer, the processing system 318 quickly searches for voxels. The processing system 318 calculates the average reflectance within each voxel of the multiple voxels across multiple samples. The processing system 318 calculates the ratio of the average reflectance within each voxel between two channels (e.g., the electrical signal 325 of the first channel and the electrical signal 327 of the second channel).

[0095] In some implementations, a LIDAR system includes a transmitter (e.g., scanner 308 in FIG. 3 a) configured to transmit a transmit signal (e.g., transmit signal 311 in FIG. 3 a) from a laser source (e.g., laser 302 in FIG. 3 a), a receiver (e.g., scanner 308 in FIG. 3 a) configured to receive a return signal (e.g., return signal 313 in FIG. 3 a) reflected by an object (e.g., object 310 in FIG. 3 a), one or more optical systems (e.g., circulator optical system 306, PBS 312 in FIG. 3 a), and a processor (e.g., processor of processing system 318 in FIG. 3 a). The one or more optical systems are configured to generate a first polarization signal of the return signal (e.g., first polarization optical signal 317 in FIG. 3 a) having a first polarization and a second polarization signal of the return signal (e.g., second polarization optical signal 319 in FIG. 3 a) having a second polarization orthogonal to the first polarization. The transmitter and receiver are a single transceiver (eg, a single scanner 308 in FIG. 3a).

[0096] In some embodiments, the one or more optical systems include a polarizing beam splitter (PBS 312 in FIG. 3a), a first detector (e.g., detector 314 in FIG. 3a), and a second detector (e.g., detector 316 in FIG. 3a), thereby providing a polarization-sensitive LIDAR (e.g., LIDAR system 300 in FIG. 3a). The PBS is configured to split the return signal into a first polarization signal and a second polarization signal such that the first polarization signal and the second polarization signal of the return signal are detected independently. The PBS is configured to polarize the first polarization of the return signal to generate a first polarization signal (e.g., first polarization optical signal 317 in FIG. 3a) and polarize the second polarization of the return signal to generate a second polarization signal (e.g., second polarization optical signal 319 in FIG. 3a). The first detector is configured to detect the first polarization signal. The second detector is configured to detect the second polarization signal.

[0097] In some implementations, the signals split into the two polarization states are compared (by calculating the reflectivity ratio between them) to estimate how much the object (or target) depolarized the return signal. In response to detecting the two polarization signals, one or more detectors generate two corresponding electrical signals (e.g., electrical signals 325 and 327 in FIG. 3a) in separate channels so that the electrical signals can be independently processed by a processing system (e.g., processing system 318 in FIG. 3a). The system has two polarization-sensitive LIDAR beams (e.g., first polarization optical signal 317 and second polarization optical signal 319 in FIG. 3a), each of which has two receive / digitizer channels (e.g., first and second channels) for independent signal processing (e.g., signal processing by processing system 318 in FIG. 3a). The system processes independent streams from points in the cloud in the two independent channels.

[0098] FIG. 3b is a block diagram illustrating another example of a LIDAR system according to some embodiments. In some embodiments, the LIDAR system 350 is the LIDAR sensor 136 (see FIG. 1a). Referring to FIG. 3b, the laser 302, modulator 304, circulator optics 306, scanner 308, and polarizing beam splitter (PBS) 312 of the LIDAR system 350 each have the same or similar configurations as those described with reference to FIG. 3a. The LIDAR system 350 further includes a detector 354, a shifter 356, and a processing system 368.

[0099] In some embodiments, detector 354 is a single paired or unpaired detector. First detector 354 receives LO signal 355 from laser 302 as a reference signal. Detector 354 is an optical detector configured to detect an optical signal. First detector 354 detects first polarized optical signal 317 and outputs or generates first electrical signal 359.

[0100] In some embodiments, shifter 356 is a frequency shifter or frequency shifting device. For example, shifter 356 is an acousto-optic frequency shifter. Shifter 356 receives second polarized optical signal 319 and generates a frequency-shifted optical signal 357 that is directed to detector 354. Detector 354 detects frequency-shifted optical signal 357 and outputs or generates second electrical signal 361.

[0101] In some embodiments, processing system 368 has a configuration similar to processing system 318 (see FIG. 3a). Processing system 368 receives first electrical signal 359 and second electrical signal 361 and calculates a depolarization ratio (as defined above) based on the electrical signals. First electrical signal 359 represents an image of an object in a first channel exhibiting a first polarization state, and second electrical signal 361 represents an image of the object in a second channel exhibiting a second polarization state. Processing system 368 processes first electrical signal 359 in the first channel and processes second electrical signal 361 in the second channel.

[0102] In some implementations, a LIDAR system (e.g., LIDAR system 350 of FIG. 3b) includes a single detector (e.g., single detector 354 of FIG. 3b) configured to detect two polarization signals (e.g., first and second polarization optical signals 317, 319) from the return signals (e.g., signals 311, 313, 315) using a splitter (e.g., PBS 312) and a phase shifter (e.g., shifter 356 of FIG. 3b) configured to shift the phase of the second polarization signal (e.g., second polarization optical signal 319). The single detector is configured to detect the first polarization signal (e.g., second polarization optical signal 317) and detect the phase-shifted second polarization signal (e.g., second polarization optical signal 319).

[0103] 3a and 3b show examples of coherent LIDAR systems that locate objects by mixing light reflected from the object with light from a local oscillator (LO). This disclosure is not limited in this respect, and in some embodiments, a direct-detection (or pulsed) LIDAR system may be used to calculate the depolarization ratio. In some embodiments, the hardware configuration of a direct-detection LIDAR system for measuring the polarization / depolarization ratio differs from that of a coherent LIDAR system. For example, a direct-detection LIDAR system requires the addition of additional optics to specifically polarize the outgoing pulse and perform measurements specific to different polarization states of the return signal. Using additional optics, a direct-detection LIDAR system polarizes the return signal. In contrast, in a coherent LIDAR system, the measured signal is essentially the portion of the return signal that is in the same polarization state as the local oscillator (LO).

[0104] 4a-4j are images illustrating various examples of depolarization ratio data according to some embodiments.

[0105] Figure 4a shows an image illustrating a color configuration table 401 used for the depolarization ratio images shown in Figures 4b-4j. According to the color configuration table, redder colors indicate that the object is more polarized (i.e., the depolarization ratio is closer to 0), while bluer colors indicate that the object is more depolarized (i.e., the depolarization ratio is closer to 1).

[0106] Figure 4b shows a color image 411 of the original scene, an image 412 showing the reflectance of the original scene, and an image 413 showing the depolarization ratio of the original scene. Figure 4b shows that the asphalt road and gravel have similar reflectance and are therefore not clearly distinguishable in the reflectance image 412, whereas the asphalt road 414 and the gravel 415 have different depolarization ratios and are therefore clearly distinguishable in the depolarization ratio image 413. Figure 4b also shows that, according to the color chart of Figure 4a, the asphalt road 414 remains more polarized while the gravel 415 becomes more depolarized.

[0107] Figure 4c shows a color image 421 of the original scene, an image 422 showing the reflectance of the original scene, and an image 423 showing the depolarization ratio of the original scene. Figure 4c shows that the asphalt road and lane markings have similar reflectance and are therefore not clearly distinguishable in reflectance image 422, whereas the asphalt road 424 and lane markings 425 have different depolarization ratios and are therefore clearly distinguishable in depolarization ratio image 423. Figure 4c also shows that, according to the color configuration table of Figure 4a, the asphalt road 424 remains more polarized, while the lane markings 425 become more depolarized.

[0108] FIG. 4d shows a color image 431 of the original scene and an image 432 showing the depolarization ratio of the original scene. FIG. 4d shows that asphalt road 433 maintains higher polarization while grass 434 is more depolarized, which is useful for detecting paved roads. FIG. 4d also shows that plant 435 maintains higher polarization while plant 436 is more depolarized. The inventors have found that such a low depolarization ratio for plant 435 was calculated incorrectly due to a split-pixel processing artifact. In general, rare objects or targets (e.g., bushes, tree branches, etc.) can cause erroneous measurements due to split-pixel processing artifacts.

[0109] To address this split-pixel processing artifact issue, in some implementations, such rare objects are detected and discarded using principal component analysis (PCA). A system (e.g., processing system 318, 358 in FIGS. 3a and 3b) calculates or obtains the distribution of points within a voxel. Based on the point distribution, the system performs PCA to determine whether the points on the plane are (1) flat, spread out, or solid, or (2) rare or linearly arranged. In response to determining that the points on the plane are (2) rare or linearly arranged, the system discards the corresponding points in the measurement or calculation of the depolarization ratio. In performing PCA, the system identifies eigenvalues ​​from the covariance matrix of the distribution and determines the sparsity of the object (or point) based on the largest eigenvalues.

[0110] 4e shows images 441 and 442, respectively, illustrating the depolarization ratio of a city scene. In image 441, the metal back surface and pole of a stop sign 444 maintain a higher polarization, while the city sign (or billboard) 443 above it is shown to be more depolarized. In image 442, the asphalt road 445 (e.g., black asphalt) maintains a higher polarization, while the lane markings 446 (e.g., white paint on the road) are shown to be more depolarized.

[0111] 4f shows images 451 and 452, respectively, illustrating the depolarization ratio of a city scene. Sign post 454 (in image 451) and sign post 457 (in image 452) are shown to maintain higher polarization, while the front of sign 456 (in image 452) is shown to be more depolarized. Asphalt road 455 (in image 452) maintains higher polarization, while grass 453 (in image 451) and concrete walkway or curb 458 (in image 452) are shown to be more depolarized.

[0112] 4g shows a color image 461 of the original scene and an image 462 showing the depolarization ratio of the original scene. In image 462, a metal pole 467 is shown to be more polarized, while the front of a sign 465 is shown to be more depolarized. Also, an asphalt road 463 is shown to be more polarized, while a concrete walkway or curb 464 is shown to be more depolarized. Also, a vehicle's license plate 466 is shown to be more depolarized, while other surfaces of the vehicle (e.g., headlights) are shown to be more polarized.

[0113] 4h shows a color image 470 of the original scene and images 471 and 472 showing the depolarization ratio of the original scene. The sign post 474 and the rear face of the sign (in image 471), the sign post 478 (in image 472), and the chain-linked fence 477 (in image 472) are shown to remain more polarized, while the front face of the sign 479 (in image 472) is shown to be more depolarized. Also, trees 473, 475, and grass 476 (in image 472) are shown to be more depolarized.

[0114] Figure 4i shows a color image 481 of an original scene and an image 482 showing the depolarization ratio of the original scene. In image 482, different building materials (e.g., building surfaces 483, 484, 485) are shown to have different depolarization ratios.

[0115] Figure 4j shows images 491 and 492, respectively, showing the depolarization ratio of an image of a person. It can be seen that the skin (e.g., face 494, 497, arms 493, 496) remains more polarized, while hair 499 and clothing 495, 498 become more depolarized. Similar depolarization ratios are obtained from images of animals.

[0116] Based on the observations from Figures 4a to 4j, object types are determined or detected based on the depolarization ratio. In some embodiments, object detection based on the depolarization ratio clarifies various key scenes. For example, (1) asphalt maintains higher polarization, while grass, rough concrete, or gravel are more depolarized; (2) metal poles maintain higher polarization, while trees or utility poles are more depolarized; (3) metal surfaces (or the rear surface of a sign) maintain higher polarization, while retro-signs (or the front surface of a sign) are more depolarized; (4) road surfaces maintain higher polarization, while lane markings are more depolarized; (5) vehicle license plates are more depolarized, while other surfaces of the vehicle maintain higher polarization; (6) skin (of a person or animal) maintains higher polarization, while hair and clothing are more depolarized.

[0117] This disambiguation technique is useful for recognizing specific road markings, signs, pedestrians, etc. In some embodiments, the depolarization ratio detects or recognizes rare features of an object (e.g., features with relatively smaller areas), which are easily registered for disambiguation of different objects. For example, rare features of a person (e.g., skin features identified based on a low depolarization ratio) are useful for detecting or recognizing pedestrians.

[0118] In some implementations, disambiguating different objects or materials based on the depolarization ratio helps a recognition system (e.g., recognition subsystem 154 of vehicle control system 120 in FIG. 1a) or a planning system (e.g., planning subsystem 156 of vehicle control system 120 in FIG. 1a) more accurately detect, track, determine, or classify objects in the vehicle's environment (e.g., using artificial intelligence techniques). The recognition subsystem 154 calculates the depolarization ratio of an image of an object acquired from a LIDAR system (e.g., the LIDAR system shown in FIGS. 3a and 3b) or receives the depolarization ratio calculated by the LIDAR system. The recognition subsystem 154 classifies objects based on (1) the acquired depolarization ratios and (2) the relationship between key feature surfaces (e.g., asphalt, grass, rough concrete, gravel, metal poles, trees, utility poles, sign surfaces, lane markings, vehicle surfaces, vehicle license plates, skin, hair, etc.) and the depolarization ratios described above. Machine learning models or techniques are utilized to classify objects. Such machine learning models or techniques include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, regression algorithms, instance-based algorithms, normalization algorithms, decision tree algorithms, Bayesian algorithms, clustering algorithms, artificial neural networks, deep learning algorithms, dimension reduction algorithms (e.g., PCA), ensemble algorithms, support vector machines (SVMs), etc.

[0119] 1a, the recognition subsystem 154 performs functions such as detecting, tracking, determining, and / or identifying objects in the environment surrounding the vehicle 110A based on the results of the object classification. The planning subsystem 156 performs functions such as planning a path for the vehicle 110A over a given portion of a time frame relative to a desired destination and stationary and moving objects in the environment based on the results of the object classification. The control subsystem 158 performs functions such as generating appropriate control signals for controlling various controllers in the vehicle control system 120 to implement the planned path for the vehicle 110A based on the results of the object classification.

[0120] In some implementations, an autonomous vehicle control system (e.g., vehicle control system 120 of FIG. 1a) includes one or more processors (e.g., processor 122 of FIG. 1a, processor of processing system 318 of FIG. 3a, processor of processing system 358 of FIG. 3b). The one or more processors are configured to cause a transmitter (e.g., scanner 308 of FIG. 3a) to transmit a transmission signal from a laser source (e.g., laser 302 of FIG. 3a). The one or more processors are configured to cause a receiver (e.g., scanner 308 of FIG. 3a) to receive a return signal (e.g., return signal 311 of FIG. 3a) reflected by an object (e.g., object 310 of FIG. 3a). The transmitter and receiver are a single transceiver (e.g., scanner 308 of FIG. 3a). The one or more optical systems are configured such that the one or more optical systems (e.g., circulator optical system 306 and PBS 312 in FIG. 3a) generate a first polarization signal of the return signal having a first polarization (e.g., first polarization optical signal 317 in FIG. 3a) and generate a second polarization signal of the return signal having a second polarization orthogonal to the first polarization (e.g., second polarization optical signal 319 in FIG. 3a).

[0121] In some embodiments, the one or more processors are configured to cause a polarizing beam splitter (PBS) (e.g., PBS 312 in FIG. 3a) of the one or more optical systems to polarize the return signal to a first polarization to generate a first polarization signal. The one or more processors are configured to cause the PBS to polarize the return signal to a second polarization to generate a second polarization signal. The one or more processors are configured to cause a first detector (e.g., first detector 314 in FIG. 3a) of the one or more optical systems to detect the first polarization signal. The one or more processors are configured to cause a second detector (e.g., second detector 316 in FIG. 3a) of the one or more optical systems to detect the second polarization signal.

[0122] In some embodiments, the first and second detectors are a single detector (e.g., single detector 354 in FIG. 3a). The one or more processors are configured to cause a phase shifter (e.g., shifter 356 in FIG. 3b) to shift the phase of the second polarization signal. The one or more processors are configured to cause the single detector to detect the first polarization signal and detect a phase-shifted second polarization signal (e.g., phase-shifted second polarization signal 357 in FIG. 3b).

[0123] The one or more processors are configured to operate the vehicle based on a reflectivity ratio (e.g., a depolarization ratio) between the first polarization signal and the second polarization signal. In some implementations, the first polarization signal represents a first image of an object having a first polarization, and the second polarization signal represents a second image of the object having a second polarization. The one or more processors are configured to calculate the reflectivity ratio by calculating (e.g., using Equation 1) a ratio between an average signal-to-noise ratio (SNR) value of the first image and an average SNR value of the second image.

[0124] One or more processors are configured to determine an object type based on the calculated reflectance ratio. For example, the recognition subsystem 154 (see FIG. 1a) classifies objects based on (1) the acquired depolarization ratio and (2) the relationship between key feature surfaces (e.g., asphalt, grass, rough concrete, gravel, metal poles, trees, utility poles, sign surfaces, lane markings, vehicle surfaces, vehicle license plates, skin, hair, etc.) and these depolarization ratios. The one or more processors are configured to control the vehicle path based on the object type. For example, the planning subsystem 156 (see FIG. 1a) performs functions such as planning a path for the vehicle 110A (see FIG. 1a) based on the results of the object classification. The control subsystem 158 (see FIG. 1a) performs functions such as generating appropriate control signals for controlling various controllers in the vehicle control system 120 (see FIG. 1a) to implement the planned path of the vehicle 110A based on the results of the object classification.

[0125] In some embodiments, the one or more processors are configured to determine the type of the object as one of asphalt road, lane markings, rough concrete road, grass, or gravel. The one or more processors are configured to determine that the object is an asphalt road based on the calculated reflectance ratio. For example, the recognition subsystem 154 classifies the object as one of asphalt road, lane markings, rough concrete road, grass, or gravel (e.g., by applying machine learning techniques to images acquired from a LIDAR system without using a depolarization ratio) and determines whether the depolarization ratio of the object is less than a predetermined critical value. In response to determining that the depolarization ratio of the object is less than the predetermined critical value, the recognition subsystem 154 determines that the object is an asphalt road.

[0126] In some embodiments, the one or more processors are configured to determine the type of the object as one of a metal pole, a tree, or a utility pole. The one or more processors are configured to determine that the object is a metal pole based on the calculated reflectance ratio. For example, the recognition subsystem 154 classifies the object as one of a metal pole, a tree, or a utility pole (e.g., by applying machine learning techniques to images acquired from a LIDAR system without using the depolarization ratio) and determines whether the depolarization ratio of the object is less than a predetermined critical value. In response to determining that the depolarization ratio of the object is less than the predetermined critical value, the recognition subsystem 154 determines that the object is a metal pole (or poles).

[0127] In some embodiments, the one or more processors are configured to determine the type of the object as one or more people. The one or more processors are configured to determine the skin and clothing regions of the one or more people based on the calculated reflectance ratios. For example, the recognition subsystem 154 classifies the object as one or more people (e.g., by applying machine learning techniques to images acquired from a LIDAR system without using depolarization ratios), determines whether a depolarization ratio of a first portion of the object is less than a predetermined first critical value, and determines whether a depolarization ratio of a second portion of the object is greater than a predetermined second critical value. In response to determining that the depolarization ratio of the first portion of the object is less than the predetermined first critical value, the recognition subsystem 154 determines that the first portion of the object is skin. In response to determining that the depolarization ratio of the second portion of the object is greater than the predetermined second critical value, the recognition subsystem 154 determines that the second portion of the object is hair or clothing.

[0128] 5 is a flowchart illustrating an example methodology for controlling a vehicle's path based on a depolarization ratio, according to some implementations. In this example methodology, the process begins in step 510 with one or more processors (e.g., processor 112 of FIG. 1a, a processor of processing system 318 of FIG. 3a, a processor of processing system 358 of FIG. 3b) determining an object type based on one or more images of the object acquired from a LIDAR system (e.g., sensor 136 of FIG. 1a, LIDAR systems 300, 350 of FIGS. 3a and 3b).

[0129] In some implementations, in step 520, the one or more processors determine whether the object is an asphalt road, lane markers, rough concrete road, grass, or gravel. In response to determining that the object is an asphalt road, lane markers, rough concrete road, grass, or gravel, in step 550, the one or more processors determine that the object is an asphalt road based on the calculated reflectance ratio. For example, the recognition subsystem 154 classifies the object as one of an asphalt road, lane markers, rough concrete road, grass, or gravel (e.g., by applying machine learning techniques to images acquired from a LIDAR system without using a depolarization ratio) and determines whether the depolarization ratio of the object is less than a predetermined critical value. In response to determining that the depolarization ratio of the object is less than the predetermined critical value, the recognition subsystem 154 determines that the object is an asphalt road.

[0130] In step 530, the one or more processors determine whether the object is a metal pole, a tree, or a utility pole. In response to determining that the object is a metal pole, a tree, or a utility pole, in step 550, the one or more processors determine that the object is a metal pole (or metal poles) based on the calculated reflectance ratio. For example, the recognition subsystem 154 classifies the object as a metal pole, a tree, or a utility pole (e.g., by applying machine learning techniques to images acquired from a LIDAR system without using the depolarization ratio) and determines whether the depolarization ratio of the object is less than a predetermined critical value. In response to determining that the depolarization ratio of the object is less than the predetermined critical value, the recognition subsystem 154 determines that the object is a metal pole (or metal poles).

[0131] In step 540, the one or more processors determine whether the object is one or more people. In some embodiments, in step 550, the one or more processors determine regions of skin and clothing of the one or more people based on the calculated reflectance ratios. For example, the recognition subsystem 154 classifies the object as one or more people (e.g., by applying machine learning techniques to images acquired from a LIDAR system without using depolarization ratios), determines whether a depolarization ratio of a first portion of the object is less than a predetermined first critical value, and determines whether a depolarization ratio of a second portion of the object is greater than a predetermined second critical value. In response to determining that the depolarization ratio of the first portion of the object is less than the predetermined first critical value, the recognition subsystem 154 determines that the first portion of the object is skin. In response to determining that the depolarization ratio of the second portion of the object is greater than the predetermined second critical value, the recognition subsystem 154 determines that the second portion of the object is hair or clothing.

[0132] In step 560, the one or more processors control the path of the vehicle based on the type of object (e.g., the type of asphalt road or metal pole determined in step 550) or the area of ​​the object (e.g., the area determined in step 550, or skin, hair, or clothing). For example, the planning subsystem 156 performs functions such as planning a path for the vehicle 110A relative to static objects in the environment based on the determined type of object (e.g., the type of asphalt road or metal pole). The planning subsystem 156 also determines paths for dynamic objects (e.g., one or more people) based on the determined area of ​​the object (e.g., the area of ​​one or more people, or skin, hair, or clothing), and determines a path for the vehicle 110A relative to dynamic objects in the environment based on the paths of the dynamic objects.

[0133] 6 is a flowchart illustrating an example methodology for operating a vehicle based on a depolarization ratio, according to some implementations. In this example methodology, the process begins in step 620 by transmitting a transmit signal (e.g., transmit signal 309 in FIG. 3a) from a laser source (e.g., laser 302 in FIG. 3a) and receiving a return signal (e.g., return signal 311 in FIG. 3a) reflected by an object (e.g., object 310 in FIG. 3a). In some implementations, transmitting the transmit signal and receiving the return signal are performed by a single transceiver (e.g., scanner 308 in FIG. 3a).

[0134] In some embodiments, in step 640, a first polarization signal of the return signal having a first polarization (e.g., first polarization optical signal 317 in FIG. 3a) is generated by one or more optical systems (e.g., circulator optical system 306 and PBS 312 in FIG. 3a). In generating the first polarization signal, the return signal is polarized to the first polarization by a polarizing beam splitter (PBS) in the one or more optical systems (e.g., PBS 312 in FIG. 3a) to generate the first polarization signal. The first polarization signal is detected by a first detector in the one or more optical systems (e.g., first detector 314 in FIG. 3a).

[0135] In some embodiments, in step 660, a second polarization signal of the return signal having a second polarization orthogonal to the first polarization (e.g., second polarized optical signal 319 in FIG. 3a) is generated by one or more optical systems. In generating the second polarization signal, the return signal is polarized to the second polarization by the PBS to generate the second polarization signal. The second polarization signal is detected by a second detector (e.g., second detector 316 in FIG. 3a) in the one or more optical systems.

[0136] In some embodiments, the first and second detectors are a single detector (e.g., single detector 354 in FIG. 3b). In generating a first polarized signal, the first polarized signal is detected by the single detector. In generating a second polarized signal, the phase of the second polarized signal is shifted by a phase shifter (e.g., shifter 356 in FIG. 3b). The phase-shifted second polarized signal (e.g., first phase-shifted second polarized signal 357 in FIG. 3b) is detected by the single detector (e.g., single detector 354 in FIG. 3b).

[0137] In step 680, the vehicle (e.g., vehicle 110A of FIG. 1a) is operated by one or more processors (e.g., processor 122 of FIG. 1a, processor of processing system 318 of FIG. 3a, processor of processing system 358 of FIG. 3b) based on a reflectivity ratio (e.g., depolarization ratio) between the first polarization signal and the second polarization signal. In some implementations, the first polarization signal represents a first image of an object having a first polarization, and the second polarization signal represents a second image of the object having a second polarization. In calculating the reflectivity ratio, a ratio between an average signal-to-noise ratio (SNR) value of the first image and an average SNR value of the second image is calculated (e.g., using Equation 1).

[0138] In the step of operating the vehicle based on the reflectance ratio, the type of object is determined based on the calculated reflectance ratio. For example, the recognition subsystem 154 (see FIG. 1a) classifies the object based on (1) the acquired depolarization ratio and (2) the relationship between the depolarization ratio and key feature surfaces (e.g., asphalt, grass, rough concrete, gravel, metal poles, trees, utility poles, sign surfaces, lane markings, vehicle surfaces, vehicle license plates, skin, hair, etc.). In some embodiments, the vehicle's path is controlled based on the object's type. For example, the planning subsystem 156 (see FIG. 1a) performs functions such as planning a path for the vehicle 110A (see FIG. 1a) based on the results of the object classification.

[0139] In some implementations for determining the type of object, the type of object is determined to be one of asphalt road, lane markings, rough concrete road, grass, or gravel. The object is determined to be an asphalt road based on the calculated reflectance ratio. For example, the recognition subsystem 154 classifies the object as one of asphalt road, lane markings, rough concrete road, grass, or gravel (e.g., by applying machine learning techniques to images acquired from a LIDAR system without using the depolarization ratio) and determines whether the depolarization ratio of the object is less than a predetermined critical value. In response to determining that the depolarization ratio of the object is less than the predetermined critical value, the recognition subsystem 154 determines that the object is an asphalt road.

[0140] In some embodiments, in determining the type of object, the type of object is determined to be one of a metal pole, a tree, or a utility pole. The object is determined to be a metal pole based on the calculated reflectance ratio. For example, the recognition subsystem 154 classifies the object as one of a metal pole, a tree, or a utility pole (e.g., by applying machine learning techniques to images acquired from a LIDAR system without using the depolarization ratio) and determines whether the depolarization ratio of the object is less than a predetermined critical value. In response to determining that the depolarization ratio of the object is less than the predetermined critical value, the recognition subsystem 154 determines that the object is a metal pole (or metal poles).

[0141] In some embodiments, in determining the type of the object, the type of the object is determined to be one or more people. Based on the calculated reflectance ratios, respective regions of skin and clothing of the one or more people are determined. For example, the recognition subsystem 154 classifies the object as one or more people (e.g., by applying machine learning techniques to images acquired from a LIDAR system without using a depolarization ratio), determines whether a depolarization ratio of a first portion of the object is less than a predetermined first critical value, and determines whether a depolarization ratio of a second portion of the object is greater than a predetermined second critical value. In response to determining that the depolarization ratio of the first portion of the object is less than the predetermined first critical value, the recognition subsystem 154 determines that the first portion of the object is skin. In response to determining that the depolarization ratio of the second portion of the object is greater than the predetermined second critical value, the recognition subsystem 154 determines that the second portion of the object is hair or clothing.

[0142] The above description is provided to enable those skilled in the art to practice the various embodiments described herein. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein apply to other embodiments. Accordingly, the claims herein are not intended to be limited to the embodiments disclosed herein, but are to be accorded the full scope consistent with the claims as they are expressed in language, and references to elements in the singular herein are intended to mean "one" and not "only one" unless specifically stated otherwise, and conversely, "one or more." The term "some" means one or more, unless specifically stated otherwise. All structural and functional equivalents to the elements of the various embodiments described throughout the foregoing description that are known or later become known to those skilled in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Furthermore, the entire disclosure herein is not intended to be provided solely to the public, regardless of whether such disclosure is expressly recited in the claims. No claim element should be construed as a means or function unless expressly recited using the phrase "means for."

[0143] It should be understood that the specific order or hierarchy of the blocks in the disclosed processes is an example of an exemplary approach. Based on design preferences, it should be understood that the specific order or hierarchy of the blocks in the processes may be rearranged while remaining within the scope of the previous description. The accompanying method claims present elements of the various blocks in a sample order and are not meant to be limited to the specific order or hierarchy presented.

[0144] The foregoing description of the disclosed embodiments is provided to enable one skilled in the art to make or use the disclosed subject matter. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the spirit or scope of the description. Thus, the description is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0145] The various examples illustrated and described are provided only as examples to illustrate various features of the claims. However, features illustrated and described with respect to any given example are not necessarily limited to that example and may be used or combined with other examples illustrated and described. Furthermore, the claims are not intended to be limited by any single example.

[0146] The method descriptions and process flow charts set forth above are provided merely as illustrative examples and are not intended to require or suggest that the blocks of the various examples be performed in the order presented. As one skilled in the art would understand, the order of the blocks in the above examples may be performed in any order. Words such as "continuously," "then," "next," etc. are not intended to limit the order of the blocks. These words are merely used to guide the reader through the method description. Additionally, any singular references to the claims, such as using the articles "a," "one," or "an," should not be construed as limiting the relevant element to the singular.

[0147] The various illustrative logic blocks, modules, circuits, and algorithm blocks described with respect to the examples disclosed herein may be embodied as electronic hardware, computer software, or a combination thereof. To clearly illustrate this interchangeability of hardware and software, the various illustrative components, blocks, modules, circuits, and blocks have been described above generally in terms of their functionality. Whether such functionality is embodied in hardware or software may depend on the particular application program and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in various ways for each particular application, and such implementation decisions should not be interpreted as departing from the scope of the present disclosure.

[0148] The hardware used to implement the various exemplary logic, logic blocks, modules, and circuits described in connection with the examples disclosed herein may be embodied or performed by a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof, designed to perform the functions disclosed herein. While the general-purpose software may be a microprocessor, the processor may alternatively be any conventional processor, controller, microcontroller, or state machine. A processor may also be embodied as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Alternatively, some blocks or methods may be performed by circuitry specific to a given function.

[0149] In some illustrative examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented as software, the functions may be stored as one or more instructions or code on a non-transitory computer-readable or processor-readable storage medium. Blocks of methods or algorithms disclosed herein may be embodied in processor-executable software modules that may reside on a non-transitory computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable storage medium is any storage medium that can be accessed by a computer or processor. By way of non-limiting example, such non-transitory computer-readable or processor-readable storage media include RAM, ROM, EEPROM, flash memory, CD-ROM, or other optical disk storage devices, magnetic disk storage devices, or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. As used herein, disks (or discs) include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks generally reproduce data magnetically, while discs reproduce data optically with a laser. Combinations of the above are also included within the scope of non-transitory computer-readable and processor-readable storage media. Additionally, the operations of a method or algorithm may reside as one or any combination or set of code and / or instructions on a non-transitory processor-readable storage medium and / or computer-readable storage medium embodied in a computer program product.

[0150] The foregoing description of the disclosed examples is provided to enable those skilled in the art to embody or use the present disclosure. Various modifications to these examples will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to some examples without departing from the spirit or scope of the present disclosure. Thus, the present disclosure is not intended to be limited to the examples shown herein, but is to be accorded the widest scope consistent with the following claims and the principles and novel features disclosed herein.

Claims

1. 1. A Light Detection and Ranging (LIDAR) system for a vehicle, comprising: a laser source configured to output a beam; a transmitter configured to transmit a transmit signal generated based on the beam; a receiver configured to receive a return signal reflected by an object in response to the transmitted signal; one or more optical systems configured to generate a first signal and a second signal based on the return signal, the first signal and the second signal having different polarizations; a processor configured to determine a type of the object by processing a signal-to-noise ratio (SNR) value of the first signal and an SNR value of the second signal; The LIDAR system, wherein the type of the object includes at least one of a type of road, a type of object proximate to the road, or a characteristic of a part of a person proximate to the road.

2. The one or more optical systems include:

10. The LIDAR system of claim 1, comprising a polarization beam splitter (PBS) configured to polarize the return signal with a first polarization to produce the first signal and to polarize the return signal with a second polarization to produce the second signal.

3. 3. The LIDAR system of claim 2, further comprising a first detector configured to detect the first signal and a second detector configured to detect the second signal.

4. 3. The LIDAR system of claim 2, further comprising: a phase shifter configured to shift a phase of the second signal; and a detector configured to detect the first signal and the phase-shifted second signal.

5. 1. An autonomous vehicle control system including one or more processors, the one or more processors comprising: A laser source outputs a beam; causing a transmitter to transmit a transmission signal generated based on the beam; causing a receiver to receive a return signal reflected by an object in response to said transmitted signal; causing one or more optical systems to generate a first signal and a second signal based on the return signal, wherein the first signal and the second signal have different polarizations; determining a type of the object by processing a signal-to-noise ratio (SNR) value of the first signal and an SNR value of the second signal; configured to operate a vehicle based on the type of the object; The autonomous vehicle control system, wherein the type of the object includes at least one of a type of road, a type of object proximate to the road, or a characteristic of a part of a person proximate to the road.

6. the one or more processors: causing a polarizing beam splitter (PBS) of the one or more optical systems to polarize the return signal to a first polarization to generate the first signal; The autonomous vehicle control system of claim 5 , further configured to cause the PBS to polarize the return signal to a second polarization to produce the second signal.

7. the one or more processors: causing a first detector of the one or more optical systems to detect the first signal; The autonomous vehicle control system of claim 6 , further configured to cause a second detector of the one or more optical systems to detect the second signal.

8. the one or more processors: causing a phase shifter to shift the phase of the second signal; The autonomous vehicle control system of claim 6 , further configured to cause a detector to detect the first signal and the second signal that is phase shifted.

9. the first signal represents a first image of the object having a first polarization; the second signal represents a second image of the object having a second polarization; the one or more processors: calculating a reflectance ratio by calculating a ratio between an average SNR value of the first image and an average SNR value of the second image; determining the type of the object based on the calculated reflectance ratio; The autonomous vehicle control system of claim 5 , further configured to control a trajectory of the vehicle based on the type of the object.

10. the type of the object includes a type of road; the one or more processors: determining the type of the road as one of asphalt, lane markings, rough concrete, grass, or gravel; The autonomous vehicle control system of claim 9 , further configured to determine that the object is an asphalt road based on the calculated reflectivity ratio.

11. the type of the object includes a type of object that is close to a road; the one or more processors: determining the type of the object proximate to the road as one of a metal pole, a tree, or a utility pole; The autonomous vehicle control system of claim 9 , further configured to determine that the object is a metal pillar based on the calculated reflectivity ratio.

12. the type of object includes characteristics of a part of a person in proximity to a road; the one or more processors: determining the type of the object as the characteristics of the portion of the person proximate to the road; The autonomous vehicle control system of claim 9 , further configured to determine portions of the person's skin and clothing that are proximate to the roadway based on the calculated reflectance ratio.

13. An autonomous vehicle, a laser source configured to output a beam; a transmitter configured to transmit a transmit signal generated based on the beam; a receiver configured to receive a return signal reflected by an object in response to the transmitted signal; a light detection and ranging (LIDAR) system including one or more optical systems configured to generate a first signal and a second signal based on the return signal, the first signal and the second signal having different polarizations; at least one of a steering system or a braking system; determining a type of the object by processing a signal-to-noise ratio (SNR) value of the first signal and an SNR value of the second signal; a vehicle controller including one or more processors configured to control operation of the at least one of the steering system or the braking system based on the type of the object; The autonomous vehicle, wherein the type of the object includes at least one of a type of road, a type of object proximate to the road, or a characteristic of a part of a person proximate to the road.

14. the one or more processors:

14. The autonomous vehicle of claim 13, further configured to cause a polarizing beam splitter (PBS) of the one or more optical systems to polarize the return signal to a first polarization to generate the first signal, and to cause the PBS to polarize the return signal to a second polarization to generate the second signal.

15. the one or more processors: causing a first detector of the one or more optical systems to detect the first signal; The autonomous vehicle of claim 14 , further configured to cause a second detector of the one or more optical systems to detect the second signal.

16. the one or more processors: causing a phase shifter to shift the phase of the second signal; The autonomous vehicle of claim 14 , further configured to cause a detector to detect the first signal and the second signal that is phase shifted.

17. the first signal representing a first image of the object having a first polarization, and the second signal representing a second image of the object having a second polarization; the one or more processors: calculating a reflectance ratio by calculating a ratio between an average signal-to-noise ratio (SNR) value of the first image and an average SNR value of the second image; determining the type of the object based on the calculated reflectance ratio; The autonomous vehicle of claim 13 , further configured to control a trajectory of the autonomous vehicle based on the type of the object.

18. the type of the object includes a type of road; the one or more processors: determining the type of the road as one of asphalt, lane markings, rough concrete, grass, or gravel; The autonomous vehicle of claim 17 , further configured to determine that the object is an asphalt road based on the calculated reflectivity ratio.

19. the type of the object includes a type of object that is close to a road; the one or more processors: determining the type of the object proximate to a road as one of a metal pole, a tree, or a utility pole; The autonomous vehicle of claim 17 , further configured to determine that the object is a metal pillar based on the calculated reflectivity ratio.

20. the type of object includes characteristics of a part of a person in proximity to a road; the one or more processors: determining the type of the object as the characteristics of the portion of the person proximate to the road; 18. The autonomous vehicle of claim 17, further configured to determine portions of the person's skin and clothing, respectively, that are proximate to the roadway based on the calculated reflectance ratio.

Images